• 

  RESEARCH | CIRGE Study

  The results of CIRGE's national study of doctoral education and careers, Social Science PhDs—Five+ Years Out (SS5) are being widely disseminated...

  MORE

• 

  Research Conference | Faculty

  These videos from the 2008 Research Conference highlight the range and depth of research activity at the College of Education ...

  MORE

•  

Application Deadlines

Scholarships & Funding

Courses

At A Glance

Calendar

Giving

home > research > rtm_07

Research

That Matters

A Test for the Test

It has been the job of UW professor Catherine Taylor, on special assignment to the Office of the Superintendent of Public Instruction as principal investigator for the WASL, to test the test, study different populations’ responses to problems, and ensure that the test is fair to all students.

Her research has pointed to the need for WASL adaptations for special-education students, versions for the blind and, translations for English language learners. It has also uncovered surprising data on who benefits from traditional math testing.
The multiple-choice test questions, preferred in much math testing, seem straightforward. They typically have four answers. One is correct: A, B, C or D. The others are incorrect. But on the math portion of the WASL, the A, B, C, D items don’t fairly assess certain groups of kids.

Taylor’s research shows that, although males tend to perform well on multiple-choice questions, females do not. The test questions that are most effective for non-Asian minorities and females are conceptual math problems — open-ended items that may require students to draw graphs, create tables, write comparison statements, or show how they arrived at their solutions.

These “performance-based” questions offer partial credit for partial understanding and recognize that students may have unique ways of solving problems — ways that may have little to do with traditional algorithmic mathematics teaching.

“There are many routes to the right answer in problem-solving. We always think when we write test questions that we know the reasons behind the answers kids choose. We don’t,” says Taylor, who had 10 years experience as a professional test developer before coming to the UW in 1991.

“The WASL is based on the idea that education is supposed to be for all students, not just the college-bound,” says Taylor. “The kind of algorithmic math traditionally taught in middle and high school might make sense — with no further explanation — to future theoretical mathematicians, but it seems a fairly elitist thing to push algorithmic math as mathematics instruction for all students.”
“Most kids are not going to become mathematicians, but they are still going to need to use mathematical ideas. What happens is that the largely abstract mathematics instruction becomes a turn-off for many students so they drop out of mathematics. We are one of the very few nations in the world where it’s acceptable to say, ‘I don’t do math.’”

To ensure that every student has a chance to engage with mathematical thinking and use mathematics to solve problems, the WASL asks students to put mathematical procedures into action in ways that are useful in daily life. The test questions — all written by Washington State math educators — may require students to figure out which cell phone plan gives them the most calls for the least money, or what the length and cost of a fence for a given rectangular area would be.

About one-third of the math WASL items are performance-based.

“The goal is to make sure that all of these questions on the WASL are really about applications of math, not just abstract mathematics with little relevance for the students,” says Taylor, whose studies show that when test questions are placed in more meaningful contexts, students are more likely to attempt them. “Students can’t show what they know if they don’t try to answer the test questions.”
Washington State was one of the first states to incorporate performance-based questions along with multiple-choice questions on a state test, and their inclusion helped stoke the fires of back-to-basics discontent with the WASL. Why were students drawing and writing on a large-scale math test? What kind of test was this? If teachers were teaching to the test, what kinds of math instruction were Washington State students getting? Most importantly, why were so many students failing a test that could determine whether or not they graduate from high school?
More than 25 percent of the class of 2008 have yet to pass the math portion of the WASL. Even with the inclusion of performance-based items, low-income and non-Asian minority students show significantly lower pass rates than their peers. It’s a high-stakes test that churns students’ stomachs, grates on their nerves, and, too often, angers their parents.

It’s easy to blame the messenger. “If there is a measure that shows kids slipping through the cracks, that measure becomes very threatening,” says Taylor. “It’s like deciding to get rid of the thermometer because it shows people have a fever. And people in poverty have an even higher fever.”
A place to look for change, she suggests, is not in the testing, but in the classroom. Many students in mathematics classes are still not being brought up to the modern content standards tested in the WASL — standards that require them to learn to think analytically and logically, and use experience and knowledge to solve problems. Taylor notes, “Research demonstrates significant improvement in WASL scores in schools that have adopted ‘standards-based’ curricula.”
Why push for deeper mathematical understanding and problem solving? The answer is again equity. Historically, rote drill, memorized algorithms, and procedural step-by-step math instruction worked for top math students who needed few explanations, but left a segment of the student population flailing. Many dropped math at the first opportunity, then bombed on large-scale tests. “These kids, slowly over time, see an algorithmic test and think ‘I’m too stupid for this. I won’t even try,’” says Taylor.

Under the federal No Child Left Behind act, that scenario is no longer acceptable.
“If we were still practicing medicine the way people did 100 years ago, the human race wouldn’t be living as long as we are. Bacteria would be killing us daily,” says Taylor. “So why would we think that mathematics instruction from the 19th century is what we still should be doing today?”

The debate over the WASL will continue as educators, administrators, politicians, and business leaders consider how best to educate a broad populace to perform mathematically in a new millennia. Meanwhile, WASL math scores continue on a slow rise. Almost 20 percent more 10th graders are passing the test on their first try than five years ago. At the 4th-grade level, almost three times as many students are passing. Taylor stated that, “Nationally, Washington State is one of the highest performing states on the National Assessment of Educational Progress — and Washington State is one of very few states that is closing the achievement gap for African American students. Washington State is seeing real growth in mathematics achievement for the first time in many decades.”
And, this time, the students who do not perform well on A, B, C, D multiple-choice problems are in the count. “It has been remarkable to see how the test is making it possible for students who have not done well with traditional assessments to show the things they have learned,” says Taylor.

For more information on Taylor’s work with state assessments see:

Taylor, C. S. (2002). Incorporating classroom based assessments into large-scale assessment programs. In G. Tindall & T. Haladyna (Eds.), Large Scale Assessment Programs for All Students. Erlbaum.

For more information on the research base of the WASL see:
Taylor, C.S., Hirsch, T., & Cammaert, R. (2004). 2003 Grade 10 Technical Report for the Washington Assessment of Student Learning. Olympia, WA: Office of the Superintendent of Public Instruction.