The Latest News

Measuring Up on November 4, 2011
View Article from Source


Psychometricians might sound like villains from Superman comics, but they soon will have a significant impact on teachers around Maryland. And for some teachers, the psychometricians might prove to be like kryptonite.

That's because psychometricians, who create and study ways to measure qualities such as intelligence, soon will craft new methods in Maryland for measuring student progress and how individual teachers affect it. The judging of teacher effectiveness based in large part on student test performance has the support of some educators and the opposition of others.

The psychometricians' work is part of the state's proposed education reforms, for which it recently was awarded $250 million through the federal Race to the Top initiative.

To show student improvement or decline on tests, schools can use a simple growth model. The most comprehensive form of growth models, called value-added assessments and developed by psychometricians, track student progress on tests over time, but they also predict how individual students will perform on a test, controlling for various factors such as socioeconomic background. Teachers are then measured against those predictions to isolate their impact on student performance.

Proponents argue that value-added assessments accomplish a task previously thought difficult or impossible.

"Value-added judges teachers by what they did, what they accomplished for a student, over a period of time," said Jeanne Allen, a Bethesda resident and president of the Center for Education Reform in Washington, D.C., which advocates for objective measures of student proficiency and teacher performance.

Prince George's County Public Schools created a program last year that paid $1.1 million in bonuses to 279 teachers who volunteered for performance-based pay. Only 21 teachers who volunteered failed to meet requirements for the bonuses (the teachers who didn't qualify for bonuses were not disciplined, because the pilot program was voluntary).

A tentative agreement reached this month between the Baltimore Teachers Union and city schools would link teacher pay to student performance, but the details of the arrangement have not been finalized, said union spokeswoman Jessica Aldon.

But the media's focus on how individual schools fare on certain tests often ignores other important factors inside schools, said Clara Floyd, president of the Maryland State Education Association. Floyd also noted that she isn't opposed in principle to using the growth models as one of many measures of teacher performance.

As part of Race to the Top, Maryland will implement growth models beginning in the 2012-2013 school year, after recommendations for the assessments are submitted to officials by a 20-member state Council for Educator Effectiveness by Dec. 31.

"Everyone wants their profession to be protected, and at the same time no one wants the bad apples that bring their profession down," said Del. Anne R. Kaiser (D-Dist 14) of Burtonsville, a member of the council.

The bad apples can have a huge impact on students.

Academically, students with teachers ranked in the bottom 15 percent are on average one year behind students with teachers ranked in the top 15 percent, said Jane Hannaway, director of the Education Policy Center at the Urban Institute in Washington, D.C.

"This is putting some objective information on the table, with all the warning signals around it, and telling management to factor this into your decisions," Hannaway said.

Growth models

may not be ‘super'

Even when a formula to evaluate teachers based on student performance is developed and chosen, how much it should count already is a source of conflict.

The 2010 Maryland Education Reform Act, which smoothed the way for Race to the Top, says growth models will make up a "significant" part of educator evaluations. Legislators stripped out language that required half of a teacher's evaluation to be based on growth models.

But in Maryland's Race to the Top application, student growth and objective standards created by the state and school districts make up half of a teacher's total evaluation. The remaining 50 percent uses classroom environment, planning and fulfilling professional responsibilities as factors.

Meanwhile, skeptics worry that statistical errors could lead to good teachers losing their jobs and question if they really measure teacher performance well.

A July report from Mathematica Policy Research, which designed a growth model for District of Columbia Public Schools, showed that with typical growth models using data over three years, about one in four average teachers are misclassified as high-performing.

Still, Hannaway said, being 75 percent certain that a teacher is average is useful information.

Teachers who initially look successful also might disappoint down the road. An Aug. 29 report by the Economic Policy Institute noted that in a study of growth models in five urban school districts, two-thirds of teachers who ranked in the top 20 percent one year fell out of that category the next year, and one-third of them sunk to the bottom 40 percent.

Floyd said factoring in a school's relationship between teachers and administrators, as well as professional development options in individual schools, helps create a truly complete picture of a teacher's ability to help students. "If you haven't provided (for) those other factors, then it's very difficult to compare Student X with Student Y from one school system to another school system," she said.

Is a little knowledge


Del. John A. Olszewski Jr. (D-Dist.6) of Dundalk, a resource teacher in Baltimore County Public Schools, said there's a danger in drawing straightforward conclusions from test scores.

He used as an example people assuming a high school teacher who brought students' reading from a third-grade level to a sixth-grade reading level is a failure because students still will be below grade level at the end of the year. School-to-school or class-to-class comparisons of growth model results should be avoided, he said.

"Teachers aren't afraid of evaluations," Olszewski said. "I think that all they're looking for is for it to be fair."

While teachers are constantly challenged to make their classes interesting, the state's standardized tests, like the Maryland School Assessment, stand out largely for how boring they are, said Katie Esmark, a fifth-grade teacher at Arrowhead Elementary School in Upper Marlboro.

"No one's asking the kids either, and I think that's a big deal, too," said Esmark, who has been teaching at Arrowhead for four years. "I think people are just telling us what's going to be done and not asking us what should be done."

In the state's Race to the Top application, however, the proposed new evaluation system includes use of the Maryland School Assessment for reading, math and science. Additional tests will be developed by the National Psychometric Council and other experts.

In the state's application, officials also wrote, "Teachers and principals who do not meet at least the Effective standard on the student-growth portion of their evaluations cannot be rated Effective overall and will thus be deemed Ineffective."

Jennifer McBeth, the math department's instructional leader at Oakland Mills High School in Columbia, said teachers might simply base what they teach on the test used in growth models. Other concerns, she said, include how to measure the separate impacts of two teachers working in one class, how to evaluate students who start school in the middle of an academic year and how to recognize that educators may feel pressure to alter data.

"I would rather have more money to put into having people who are qualified to evaluate teachers and support teachers in other ways than standardized tests," McBeth said.

Add a Comment!

Our Team!

the blog

2014 End of Session Report
By: on April 14, 2014

Dear Friend,

Now that this year’s 2014 legislative session has officially drawn to a close, I am pleased to provide you with my Annual End of Session Report.  Sending these updates is important to me because it reflects ...

Read more on this topic! See more topics!
Latest News

State Senate Candidates Working Out Details Of Two-Debate Proposal
Ben Boehl, The Dundalk Eagle on September 23, 2014

It is rare for a candidate regarded as a front-runner to seek out debates; such events are traditionally regarded as ...

read more!

New Battle Acre Gates Set To Arrive In Time For Bicentennial Event
Ben Boehl, The Dundalk Eagle on September 23, 2014

As The Eagle went to press on Tuesday, it appeared that the pedestrian gates would be in place for Thursday’s ...

read more!

Democrats Propose Annual Pass System For Key Bridge Tolls
Ben Boehl, The Dundalk Eagle on September 23, 2014

It took a while, but it appears that the Democratic state legislative candidates in the 6th District are on the same ...

read more!