Assessment


Honestly, the last thing I thought I’d be doing this week is posting in every spare moment on technology and its influences on literacy. Help me.

Or humor me. The mysterious commenter Dave (at dave@dave.dave, apparently) kindly provides this fast and super fun article at Slate from just last week, on the actual means by which we read on line differently from paper text. Michael Agger, who has won my heart with his snarky use of Net-bold type alone:

Humans are informavores. On the Internet, we hunt for facts. In earlier days, when switching between sites was time-consuming, we tended to stay in one place and dig. Now we assess a site quickly, looking for an “information scent.” We move on if there doesn’t seem to be any food around.

Sorry about the long paragraph. (Eye-tracking studies show that online readers tend to skip large blocks of text.)

Also, I’m probably forcing you to scroll at this point. Losing some incredible percentage of readers. Bye. Have fun on Facebook.

Take or leave his wordplay, but I’m going to be be thinking all summer about the ramifications of the Net reading meta-approach this discusses. Could it be– could it– treated as a new genre of reading, unto itself?

1) Why all educators should tar and feather anyone who mentions schools and “competitiveness in the 21st century” in the same breath.

2) Why “high flyer” schools that defy poverty might not be doing anything of the sort.

3) Tell me NCLB measures something concrete. Tell me standardized exam scores reflect teacher prowess. And then tell me why New York State gives its English exam in January, smack between two teachers’ interaction with a cohort of kids. Please. Tell me. I’ve been asking for seven years now.

Dear Lupe,

Did you ever wish that you could save the world?

Awhile back a student of mine, in seventh grade English, turned in an assigned poem. I loved its simplicity, its rhythm, the way the lines broke on the page. What made my heart even more glad was that it was from a kid I’ve been trying to reach for several years now.

Anyway. I was so proud of him that I posted the poem on my teacher blog earlier this week. And that’s where I found out he hadn’t written it at all. He had plagiarized “Kick, Push,” and confirmed that he had done it deliberately when I asked him about it. And not knowing your rap until this week, I had no idea.

It’s been an interesting journey, these past few days. I’ve cried once or twice. I’ve rethought how I give and support assignments for second language kids. I’ve been surfing your sites, pulling up your stuff on Youtube. I’ve fallen in love with your work. And I’ve rejected completely the punitive coercion that could serve as the consequence for plagiarism in my school. That stuff won’t work. This kid is too smart.

The only thing that will work, I’ve come to realize, is if, somehow, he talks to you.

This might strike you as overkill. What is plagiarism, after all, next to cheese heroin addiction, or teenage pregnancy, or gang bangs? But I would argue that it’s just this kind of tiny, critical choice, and how it’s handled by the adults involved, that can tip the balance in a pre-adolescent kid. Towards a life that is ruled by a living sense of the dignity of human beings, or suffocated with the stale mediocrity of selfishness. Towards a life fortified against amorality, or one that invites it in—in small ways now, and perhaps much worse ones later.

So now is the time.

I don’t really know what I am asking you for. Five minutes on speakerphone would do it. Perhaps a letter. Something that makes you real to my kid. Something that it is not yet, or may never be, within my power to do– try as I might.

Because you see, it’s not enough that you’re like a god to him; it’s not enough that he listens to you constantly and can recite your raps with passion and accuracy from memory in the middle of class. None of it matters—not the poetry, the positive role model, or the message—unless he internalizes it enough to know that in the destructive habit of taking the short, easy way out, he cheats everyone. You. All of us. And most importantly, himself.

I can’t guarantee that this will save the world. Maybe not even this kid. But it might. Will you bank on hope, with me?

Please give me a call.

~~~~~~~~~~~~~~~~~~~~

I’ll be sending this via snail and email to Lupe’s production company, 1st and 15th Entertainment. Anyone else got any bright ideas? Contacts I can use? How much does a full page ad in the Chicago Times cost?

I was driving into work today listening to Steve Inskeep on NPR finish up a report on presidential candidates (focus Mike Huckabee). And what transitional music do they play in between that report and the next? The first two notes– and only the first two notes– of the initial guitar lick of Depeche Mode’s “Your Own Personal Jesus.” I laughed all the way to school.

It got me thinking about the school improvement plan we were handed yesterday at our faculty meeting (I might post about this fascinating document later), a key component of which was addressing “weakness in background knowledge at all grade levels.” I can’t help but reflect that is only my odd, twisted, totally unique combination of “background knowledge” that let me in on an NPR auditory joke that I would be surprised if anyone else actually comprehended.

So a widely-administered test on understanding this joke, I’m sure you’d agree, would be unfair. And yet isn’t this exactly what we are doing in our listening and reading comprehension passages on a standardized exam?

E.D. Hirsch’s work here, and related articles in the same American Educator, have perhaps the most well-sourced argument I have seen for reading comprehension being tied almost completely to background knowledge. There’s also the old related standby that the “background knowledge” required to pass a standardized reading exam inevitably is reflective of socioeconomic status.

But I think I’m going one step further here. I heard NPR’s joke NOT only because I’ve heard some Depeche Mode, and NOT only because I happen to listen to NPR regularly, BUT because I personally happen to have some musical sense. My “background knowledge” is tied intimately not only to my SES, but to that pattern of exposure, internalization, predilection and talent that everyone has, and never repeats, like snowflakes.

Can we test the reading ability of a snowflake? Sure. (Give it a book about phases of matter and watch the magic.) But can we do it properly with a sole arbitrary reading exam that we administer to approximately 2,800,000 kids in New York State alone? We need different data than that. And we need better.

I had a conversation with one of my English colleagues this past week. We both agreed that we would eventually wander the earth rejected by both neo-hippies and corporate executives alike, since we fundamentally disagreed with the content and the administration of the high-stakes establishment exam we were giving in two weeks, but were giving it anyway.

This blog post won’t be unique or brilliant, but I hope to use it in upcoming years to remind myself of why my friend and I don’t really belong in Purgatory.

First of all, I have to dismiss the idea that participating in an institution is de facto an unacceptable compromise of integrity. My fear is that because I have dearly held principles that are at odds with those of my workplace, and yet I continue to participate in that work, I have at best subsumed myself in passivity– or at worst, screwed up irredeemably. I have to move beyond this fear.

This is not easy. We are a nation born of plucky, underdog resistance to an unjust institution, and the glorification of individualized resistance is in our myths, our ads, our movies, the water—particularly for those Thoreau-worshippers (me) for whom resistance, in general, seems like a great idea. And for Thoreau-worshippers who are also teachers, there’s no arguing that our natural tendencies are given plenty of fuel in the current climate of one-size-fits-NCLB.

But unexamined radicalism is unexamined radicalism. In otherwords, entrenched institutionalists and Thoreau-worshippers can both spout junk. So I must examine my tendencies at their logical conclusion. Should we abandon our schools because we believe that moral compromise is indigenous to them as institutions? Should we teachers go it alone—say, as free market hedge fund managers? Certainly there are some former educators out there who have done so—I know a few. But I’ve yet to meet a thoughtful teacher, sticking it out, who is just a hedge fund manager in disguise.

These teachers know two things about the myth of The All-Mighty Individual, for starters.

First, they know that the concept of somehow achieving moral purity through individualism is false. The struggle to act rightly in schools is born of our humanness– and thus indelibly replicated in the microcosm of our selves. The 10X15 house beside Walden Pond won’t save you or me. (It sure didn’t save Ted Kaczynski.)

Secondly, teachers committed to schools know that the healthy individual is indivisible from community. (Thoreau knew this too, actually, and received friends and visitors nearly every day.) In addition, a mass of psychological research has demonstrated that individualism (the myth) and autonomy (the healthy reality where we are nurtured as individuals in balance with communities) are two entirely different things. I’ll be talking more about this soon when I (finally) get to Parts II and III on “Why We Do What We Do.”

And there’s one last theory I have as to why thoughtful teachers stick it out. I’ve started to think of it as “soft math.”

Soft math is the cost-benefit analysis of participating in institutions which we know are fundamentally flawed. It isn’t quantitative or qualitative, but a yeasty mix of both. Soft math asks the question: does the amount of good that I do—in my college, in my cubicle, my classroom—outweigh the amount of damage I do by enabling a broken system?

In my clear-eyed moments, I know that it is soft math that must be my guide. Soft math does not rule out either change or stability. It leaves room for multiple responsible reactions to one’s teaching circumstances. It requires vigilant observation and reflection in a way that ideological despotism does not.

And, perhaps most importantly, soft math rests on hope. It is accountable hope, to be sure: a hope that must borrow the hardness of diamonds from logic, from precision, from fact. But this hope is also feathered, and perches on the soul.

The poet Adrienne Rich has something to say here too.

Oh you,

who love clear edges

more than anything…

watch the edges that blur.

(To borrow a phrase from Alexander Russo’s blog: I geek out on this stuff so you don’t have to.)

When I realized that Alfie Kohn’s highly touted book “Punished by Rewards” has well over twenty footnotes from the research of Ed Deci and Rich Ryan, I decided to follow the river back to the spring. I found one of those books that will come to my desert island: Ed Deci’s “Why We Do What We Do: Understanding Self-Motivation” (1995). What follows is Part One of Three on this theory and its applications to teaching.

I’ve never tried this in-depth kind of blogging before, but I am inspired by the lively and thorough prose of Eduwonkette (here’s her latest post), and I am feeling the need to document how important I think this theory is for teachers. But if your interest flags, worry not. Read another blog. I’d suggest the folks we really should be listening to: kids.

So here we go:

I was given an envelope yesterday morning that contained a pencil, a bookmark, and a colorful little eraser. I was told that at some point I should stand out in the hallway and give these out to whatever student was carrying around a book to read.

Motivation, in otherwords.  How can people motivate others? This is the question every good teacher should be asking, right?

Except that it isn’t.

WHAT IS THE PROPER QUESTION?

The proper question, Deci says, is this: “How can people create the conditions within which others will motivate themselves?” This reframing is very important, because it puts aside that idea that “motivation” is something that we can “do” to other people. Instead, motivation, in order to be truly effective, becomes something that must come entirely from inside the people we work with. It is tapping into their innate vitality: the natural need to engage and be engaged by the world.

WHAT IS THE ANSWER?

Deci’s answer is that the fastest, most productive, most enduring way to create intrinsically motivating circumstances for a human being is to support that person’s autonomy. Well-aware of the touchy-feely dangers and self-help gurus that populate this topic, he uses nearly twenty-five years of comprehensive, rigorous psychological research to back up this answer. He calls it “empirical humanism.”

DOES IT WORK?

Deci also conducts research to determine whether intrinsic motivation is all that it’s cracked up to be. It is. Intrinsically motivated tasks result in greater conceptual understandings, superior retention of information, technical expertise, and creativity.

HOW DO WE DO IT?

Autonomy support, Deci asserts, means addressing the three fundamental psychological needs of people—something like the air, food and water of the psyche. They are the need to feel self-determined; the need to feel competent; and the need to be connected to others. In every activity we undertake, if we consciously address these three needs, then this creates a situation where intrinsic motivation can thrive.

In his experiments, Deci addressed these needs in three concrete ways: providing a rationale for tasks (self-determination); acknowledging feelings (connection); and minimizing pressure (competence). In every case, intrinsic motivation occurred where these three things took place; and proportionately less intrinsic motivation occurred where they were absent.

And here’s a kicker: “Autonomy support,” Deci writes, “is a crucial context for maintaining intrinsic motivation…it also turns out to be essential for promoting motivation for uninteresting, although important, activities.” More on this next post—although I can’t resist commenting here on how telling it is that in this part of the book, all Deci talks about is school.

WHAT ELSE IS THERE?

Deci also discusses the range of behaviors in humans which are not autonomous (the gold standard). There is, of course, compliance, or doing something simply to do it. You might call this the “I don’t believe in this, but I don’t want to get in trouble” approach.

The opposite, one which teachers also know well, is defiance. Deci is very clear (and perhaps for us liberal-minded folk, this might come as a shock) that defiance for defiance’s sake is not autonomous, either. This might be termed the “Screw you, no matter what you’re saying” approach—or, more subtly on the adult level, “All change is good.” Defiance that is defined entirely by external control, in otherwords, is merely reaction, not autonomy.

But there is a middle ground as well. Deci calls this introjection. “These people,” says Deci, “do a behavior in spite of not feeling free, not enjoying it, and not believing it was personally important. They had swallowed the thought that they should do it, and they plodded forward, rather like sheep to the slaughter.” I’ll be talking more about how this  applies to school in my next post, but suffice it say for the moment that these people are tough nuts to crack. They comply; and they will even say, perhaps with great conviction, that they WANT to comply. But their motivation for doing so is not, at bottom, true to their authentic selves.

WHAT DOESN’T WORK?

Here is the list. For those of you who may not be familiar with this, it’s a little shocking. Remember that this is backed up by twenty-five years’ worth of data.

1)      In general, rewards of any kind: money, certificates, or colorful little erasers.   

2)      Pressure.

3)      Competition.

4)      Threats/demands.

5)      Surveillance.

6)      Critical evaluations.

7)      Summative tests and quizzes.

8)      Grades.

Deci puts it best: “Not only do controls undermine intrinsic motivation and engagement with activities, but—and here is a bit of bad news for people focused on the bottom line—they have clearly detrimental effects on performance of any tasks that require creativity, conceptual understanding, or flexible problem solving.”  

And the costs don’t stop there. Very strong extrinsic aspirations in adults– the same ones we encourage in our students–were always—always– associated with poorer psychological health.

Lots to think about…

In Self-Determination Theory Part Two, I’ll be addressing some foundational questions that arise from this work, and also how this theory applies specifically to teaching and learning.

I realized recently that I’ve been playing guitar for nearly twenty years now. (Have I been alive for twenty years?)

I’m a total hack– never took a lesson. I fake complexity in several nutty “Eastman School of What?” ways, including “alternate tunings.” Although that term implies gravitas that I don’t deserve. I basically just mess around with the pegs until something sounds good to me.

One of these tunings is formal, though: Open D. You tune your lowest and highest string to the note “D,” and when you strum without any fingering, out comes this lovely, resonant, deep thing, like a monk’s chant. It’s a dangerous tuning, though, has a life of its own. One misplaced finger when you do add chords, and the whole thing can fall apart– or, conversely, can take you on a wild musical trip you hadn’t planned at all.

Playing is a lot like that in general. Yeah, you practice, ostensibly so that you “get it right.” And you feel nervous at the idea of “messing up.” But this presupposes that your song is a static, unmoving body of knowledge. The fact is, though, that your “mistakes,” your missed strings, your off-beat strum, can actually end up being more interesting, more captivating, more truthful than what you were trying to do in the first place.

I’m not sure exactly what I’m trying to say here. That standardized tests and other school practices rob our kids of this organic kind of response to the world? That knowledge is just as much what happens in the unrepeatable moment as anything else? That “predictive validity” isn’t really validity at all?

Despite all these brave words, though, I have played for my students over the years exactly twice. But I hope to do something with one class mid-week, and if it flies I’ll unleash it on everyone else after the break. Working on it.

OK, grades are done, and it’s time to write about something positive. I stumbled randomly onto something this year that solved so many writing problems in one sweep that it’s near magical. What is this silver bullet?

Choice. Kids write a mini-essay– that is, three paragraphs, no more, no less– once a week, on whatever topic they choose.

I did this primarily to help motivate kids to write– that is, merely to get some pleasure out of producing words. I had no idea it would make them write so much better. They are regularly turning in work that is creative, lively, well supported with detail, and organized. My current favorite example is the kid who wrote his last essay describing, in five paragraphs of hilarious irony, how he couldn’t think of anything to write his essay about.

And I don’t think I will ever forget my student K., a reluctant writer at best, who shocked me by bursting out spontaneously in class the other day, “You’re like my favorite teacher ever. You give us…like… options.”

Now, this is not to say that I now have 85 Hemingways on my hands, but the writing problems I hear about in faculty meetings are not ones I consistently have. “They’re so dry. They have no voice.” “Their use of dialogue is awful.” “Why can’t they get their thoughts organized?” And I sit in the back and think: Who are these children?

These results are supported by a strong body of research conducted by Ed Deci at the University of Rochester, my new hero– I’ll be blogging on his stuff next post. But here’s a quote to whet your appetite: “…The performance of any activity requiring resourcefulness, deep concentration, intuition, or creativity is likely be impaired (italics mine) when external controls are the reason for their behavior.”

Oh my.

It seems to me that the most interesting thing about this is the implications it has for assessment. I would love it if kids were regularly given were opportunities to polish and share their own writing in addition to imposed assignments (which we do need to do, and can do well). This guarantees more of a multiple measures approach. But if a child’s writing portfolio is composed of nothing but demanded pieces, then what are we actually seeing?

A monkey hitting typewriter keys at random may eventually produce Shakespeare. But given my experience so far this year, I’d rather see what he turns out if we let him write about bananas.

Because I have no shame about this stuff anymore I emailed Lowell Monke today with a thank you and a note that I cited him in the last blog entry. Imagine my shock when he wrote back immediately.

To be included in an essay under a headline that pays homage to Pink Floyd and Albert Einstein is a great (and totally new) honor. Thank you for getting in touch. I am heartened that you pulled that particular item out of my article. Everyone else seems to ignore it, but I think it is really the key to changing the way we look at childhood and teaching.

Wow.

My colleague Joe likes to quote “question the hell out of everything” (and thank goodness). And so, when we attended a meeting recently where we were trained to be community portfolio reviewers for a constructivist high school in the area, it shouldn’t have surprised me to have him say as we left: “This is great. But really, though– how do we really know what our kids know? No matter what measures we use?”

He’s right, of course. The challenge teachers contend with every day is that we cannot (yet) pop our kids’ heads open like a Coke can and see what’s inside. We settle for the next best thing, which is to find valid and reliable means of assessing our students’ behaviors. But no matter how clever and penetrating this assessment is, it still stops short of truly knowing another’s mind. And for the entirety of my teaching career, I have treated this as a problem to be solved.

But perhaps it isn’t a problem at all. Perhaps, in the end, it is exactly the way things should be.

Einstein put it this way: “The most beautiful thing we can experience is the mysterious. It is the source of all true art and all science. He to whom this emotion is a stranger, who can no longer pause to wonder and stand rapt in awe, is as good as dead: his eyes are closed.”

Lowell Monke, in his article “Unplugged Schools” in the October 2007 Orion Magazine, then applies this idea to our students. “Some facet of a child’s inner life must remain sacred– off limits to our machinations,” he writes. “It should be viewed not as new territory for scientific investigation and technical manipulation but simply with awe and reverence and our own best, most human, expressions of support.” And this is exactly the fact that keeps our innocent hero Andy alive in the amazing movie “Shawshank Redemption” (check out how here).

Now, we’re not all going to be imprisoned in a 1960’s penitentiary falsely accused of murder. But we are going to be faced with pressures to act immorally, destructively, or simply in a way which is not in accordance with our authentic selves. And the fact that we all possess this autonomous core may be the only thing that saves us. 

Can I then let my assessment do what it does well, and then trust the rest? Trust that honoring the core selves of my students will bear good fruit– perhaps precisely because they know I trust them? Should I risk intruding on this core self simply because I want to figure out if a kid really knows what the central metaphor of The Great Gatsby is?

This is not a plea to abandon assessment. Is is, I think, a challenge to accept assessment’s limitations– and even to consider those limitations as healthy and appropriate. 

Is one of the best things we can do as teachers is– well– leave those kids alone?

Next Page »