A love-hate relationship with measurement
This post first appeared on my original site Nov. 14, 2017
There were so many levels of irony at this week's school board meeting, it was a challenge to untangle them. It took me a few days to percolate a blog post, and I'm not convinced I'm going to capture what I really want to say.
Here's my takeaway: board members were openly self-congratulatory about how little they care about test scores and how it's so important to only concentrate on un-measurable types of learning. This was while they were reviewing a report that they had requested consisting primarily of test scores. Requested because it is called for under a governance model that leaves the board very little room to consider, measure, reward or act on anything *but* data.
Scratching your head? So was I.
Our board is on the one hand facing the central question I think most educational institutions are facing, particularly in K-12 education: our educational system, thanks to the corporate culture spreading everywhere, places a very high value on measurement and data, and yet parents, employers, higher education and our communities in many instances are demanding that kids be equipped with critical thinking skills, job readiness, the ability to work in a team and to problem solve rather than the memorize and repeat back rote information. As one teacher pointed out during the meeting, mere facts can be collected at any moment from a smartphone in someone's pockets, so being able to regurgitate facts from memory is suddenly a lot less valuable than it was when we adults were in school.
In a related dilemma -- one the school board knows well, so many parents, community members and even educators love to hate standardized tests. Yet when we move to a new place, we search for a school district ranked high based on its test scores. We pay for expensive test preparation courses and books to help our high school kids prepare for the ACT or SAT, because those scores remain an indicator of academic mastery that colleges use to decide whether to admit those kids. Businesses locate in communities with the best test scores, and politicians point to test scores as evidence of improvement or failure in educational reform.
Shorewood School District leadership is facing these questions along with educators and administrators across the country and beyond. There are a few challenges unique to us that worry me:
First, our affection for dismissing standardized test scores runs counter to a very data-drive policy governance structure that the school board adopted. The board insists it will only focus on setting policy and measuring "outcomes." During this week's board meeting, the limitations of that model -- indeed, it's incompatibility with what the "authentic learning" Shorewood leadership loves to hold up as its ideal -- were brought into clear relief.
As a parent, I don't want my kids tested constantly, but I do want the school to run regular standardized tests in each subject at least once a year, and I am OK with the loss of some instruction time if it means we have a clearer picture of growth, gaps and (hopefully) excellence. It also means that taxpayers see that they're getting the excellent schools many of them moved here to be near, and that the taxes they pay every year are put to use making sure there we don't have a town full of clueless young people. I don't think a standardized test tells you much about an individual kid -- I do think a standardized test allows for aggregate data that is very useful to spotting trends.
The newest data available to us to help us measure those things is from the Forward exam administered to 3rd, 8th and 10th graders in spring of this year. The board looked at the Forward exam, ACT scores and course grades for Science and Social Studies in grades 7-12 this week, starting with Science.
Context: This year, the middle and high schools have been adopting a new science curriculum based on Next Generation Science Standards. This was welcome by many parents who rightly noticed that science at SIS was a weak spot. The latest test results don't reflect the shift because the Forward exam tests were conducted in the last school year.
The hope is that these new standards will address both low performance in general and the gaps in performance: African-American students, and males had much lower Forward exam results (8th grade), final grades of "C" or better in Science, and lower ACT scores (11th grade) than white students and girls. Shorewood 8th grade boys did worse than the state average on the Forward Exam by % proficient or advanced (41.5% versus a state average of 47.6% proficient and advanced) and African-American students' scores had an even wider gap -- 33.3 scored proficient or advanced.
But it's not clear that test scores are going to rise as a results of adopting this new curriculum. Teachers were open about how difficult the transition has been, because it's less "content-heavy" (e.g. learning the periodic table) and more focused on understanding and practicing scientific inquiry. At the same time, everyone seemed very confident that the new curriculum would narrow gaps by gender and race by offering better "engagement" and attention to "identity safety." I sincerely hope it does, but I would feel more optimistic if they had cited research showing that those things have worked elsewhere.
Next, as the board looked at its report and heard from teachers on Social Studies in grades 7-12, the mood was what I would describe as proud but conflicted or even befuddled. On the one hand, I share that pride in one thing the teachers talked about -- bringing the district's professional development Raceworks project and its Teaching Tolerance work into the classroom by implementing a social justice curriculum.
"That has been a whole new lens to put on everything we do," longtime SHS Social Studies teacher John Jacobson [edited to correct his name, I am so sorry for the error] said of the social justice curriculum. He admitted how skeptical, even cynical, he was about this professional development work, but it was clear he'd been 100% won over and legitimately changed his teaching and relationships with students as a direct result of that work. When someone who has taught for decades in the district is joyfully describing his own growth as a professional, that's pretty great to see.
"The teaching Tolerance and Raceworks piece for us is changing what we do, even as we are still teaching the same courses -- ten of them -- it's changing what we do and how we look at those courses," he said. "Which hopefully brings us back around to this a conversation like this in a year or two or three when we can look at that gap and see the data we all want to see."
Along with Mr. Jacobson, Sarah Kopplin, a great coach and teacher who teaches Geography at SIS, also was thrilled about bringing social justice curriculum into her work. She talked about new "authentic learning" she was focused on bringing her students -- very cool stuff, like having kids propose their own solutions to major global challenges, and hosting a simulated refugee crisis. That work really engaged students and got them excited, she said, which she said hopefully would "narrow that (achievement) gap, and probably not to any test scores improving. I don't know -- we'll see," she laughed.
I should say here that we are well above state averages on the Forward Exam for Social Studies, given in 4th, 8th and 10th grades, with 66.4% proficient and advanced compared to a state average of 49.6%. So this isn't an area like science where we are really in need of attention to why we're not doing better across the board, but there are some significant gaps in test scores by race and gender here as well.
As Mrs. Kopplin continued, and the board responded, I felt things got a little bit too far into the "test scores are meaningless" territory, and even further into celebration of the idea that kids shouldn't be saddled with learning just facts, but should instead be exploring the world as historians and problem-solvers, because that's "authentic learning." For example, Mrs. Kopplin said first that she no longer has any map tests (i.e., "label these countries on a map") because she doesn't see them as valuable in her World Geography class.
This is a point where I think reasonable people can disagree, and I absolutely don't mean disrespect to Mrs. Kopplin, but I actually like the idea of kids being asked to label countries on a map. When our president is making up fake countries in Africa, clearly we could all use a basic understanding of where things are located, before we dive in to cool things like brainstorming ways to tackle global climate change.
Board members were very pleased with what Mr. Jacobson and Mrs. Kopplin were saying. I understand why they were excited -- it's thrilling to see teachers who are so clearly dedicated and full of joy, optimism (and commitment) to and about their work. But they struggled with their love of "authentic learning" and the report in front of them, a direct product of their data-driven governance policies.
Board member Pablo Muirhead brought the point home: "I think what's going on in the classroom is incredibly rich, and I don' t know that this data captures it," he said. "This report doesn't capture the amazing things that are happening."
He asked the teachers for their input on ways the board might measure the great work they are doing. They got a thoughtful pause, then Mr. Jacobson joked, "How late can we stay? .... This profession is in transition, necessarily so, from content to process. That's the world we're getting these kids ready for, that's the world we live in ... I struggle mightily with that question -- how do we put data points on that process?"
Folks, I could just continue to quote Mr. Jacobson, because he's clearly an AMAZING teacher and human. Because he was unknowingly becoming my new favorite person, Mr. Jacobson quoted a piece by poet Wendell Berry (no relation, but one of my favorite writers) that he'd recently assigned his Political Theory students.
"We are now almost at a point when we find it embarrassing when we cannot reflect the results of some endeavor with a number or a technical term." I love that thought -- how clearly crazy it is that we have devalued the things that matter most. Everyone in the room looked pensive.
So, here's the thing: apparently most Shorewood teachers, administrators and school board members, along with other parents all agree that data is extremely limited and insufficient as a way to gauge whether we're delivering a quality education. We're on board with Wendell Berry. But in the meantime, our district leadership -- specifically our board, operates in a way that relies on the kinds of reports that Dr. Muirhead pointed out leave out pretty much everything they've decided is important.
Board Vice President Rodney Cain saw the dilemma, at least in part: "I think this is something we're going to continue to struggle with, because there's a great desire to measure ... demonstrate that we're doing this. This isn't just true in this area. We struggle with this in 'wellness,' we struggle with 'healthy habits' and so forth."
When it comes to social justice or wellness, those intangible things, he said, it may be the district just has to ensure kids are "exposed to things." I'm not sure that is sufficient for actual content, though -- kids do need to be able to identify the Bill of Rights and differentiate Ghana from Senegal. There are some things we need to make sure they just know -- plain old facts. Again, now that we're in an age when facts are brought into question every day, it seems foolish to dismiss the value of learning and retaining actual knowledge.
The two of us aren't always on the same page, but as the conversation wound down, Superintendent Bryan Davis backed up to where I feel more comfortable, in the middle ground. He pointed out that the district does set expectations for standardized tests, for content understanding, along with other things the district values like social justice. "I think it's an 'and/both'" he said.
I wish our board could get closer to an "and/both" place with its own work -- to see beyond its almighty Reports, its carefully numbered policies and its Operating Expectations, and just have real conversations in public with people, to talk to teachers without fear of breaking rules, to stop being afraid to actually ask questions about what's happening in classrooms, playgrounds, fields and hallways. I want to see them dive in to the operations of our district beyond what administrators write up, graph and score for them. Not everything that matters can be measured, and not everything measured matters.