Thomas Scott, from the Education Collaborative, chats with Sally Dias, superintendent of Watertown Public Schools. Photo by Jon Chase.

Being responsible for children, thousands of children, and discovering that many of them didn't do well on a state-mandated test -- well, that can keep you up nights, say school superintendents in the Boston area.

But more than a dozen superintendents who gathered at the Graduate School of Education (GSE) recently were anything but passive while grappling with concerns about the exam known as the MCAS (Massachusetts Comprehensive Assessment System). They engaged in a lively discussion with experts on testing -- including the GSE's Paul Reville -- regarding how MCAS scores compare to those of other tests with which they had more experience.

"This is the first time out for the MCAS; think of it as a test of the test," said Reville, a lecturer on education and co-director of the Pew Forum on Standards-Based Reform.

In a profession where much of one's time is spent on the "three B's" -- buildings, budgets and buses -- school superintendents hunger for something more: a chance to talk with peers about problems and ideas, an opportunity to network, and space to listen to other professionals share their research.

For leaders of 16 school districts and a former superintendent who works as an educational consultant, the Greater Boston Superintendents' Roundtable provides all of these things. Once a month, these school leaders from Boston and Weston, Quincy and Waltham, and other towns gather at the Graduate School of Education. Over coffee, croissants, and boxed lunches, they listen to and ask questions of professors from the GSE, other Harvard schools, and other institutions. They learn about research that is relevant to the daily life of public schools and they come away recharged.

"As superintendents, we are advocates of professional development for our teacher colleagues, but we often don't make the time for reflection and to improve our own practice," says Thomas Payzant, superintendent of the Boston Public Schools. "The Roundtable provides that opportunity."

Holding Schools Accountable

Two superintendents, Jeffrey Young of Newton and Sally Dias of Watertown, approached GSE Academic Dean Susan Moore Johnson two years ago with the idea of creating a series of seminars that would be held at, and sponsored by, the School of Education. Johnson loved the idea, says Peggy Kemp, who is director of the Office of School Partnerships.

"Susan suggested that I collaborate with Linda Greyser, associate director of the Ed School's Programs in Professional Education to come up with a plan," Kemp says. "Not long after that, we had our first two presentations."

Richard Murnane, a professor at the School of Education who studies connections between education and economics, led those initial sessions along with Frank Levy, an M.I.T. economist with whom he had written Teaching the New Basic Skills. Several seminars during that first year were also led by Richard Elmore, a GSE professor who specializes in making school reform issues accessible to local school districts.

Superintendents' desires to focus on themes of accountability and assessment this year have led Kemp and Greyser to invite speakers not only from the GSE, but also from the Business School and other institutions. Two experts on testing from Boston College, George Madaus of the Center for the Study of Testing, Evaluation and Educational Policy, and Irwin Blumer, a research professor, joined Reville earlier this month to discuss the meaning of the MCAS test scores.

Many students who, on the MCAS, had placed in the "needs improvement" or failure category had previously scored as proficient on other tests, emphasized Madaus. Through a series of graphs and charts, he showed how the MCAS overlapped with exams like the PSAT, the Iowa, the ACT (American College Testing), and the ERB (Educational Record Bureau).

"A huge disconnect exists in some instances," Madaus said. "Other standardized tests show Massachusetts to be among the 14 states that do better in science than most modern countries except Singapore. Yet 40 percent of Massachusetts students failed the science part of the MCAS."

While the goal of raising student achievement is important, many youngsters are actually doing better than the MCAS scores would lead one to believe, said Blumer.

"Many kids in the 'needs improvement' category have actually mastered basic skills," Blumer said. "But what the press does is label these youngsters, who are academically average, as failures."

The MCAS tests, however, "are on the right track" in measuring the comprehensive skill of students, the Boston College professors stressed. And like any other test, they added, the MCAS is "eminently coachable."

Reville agreed. "On the whole, the MCAS represent a step forward in testing. We're setting the bar higher for students, and that's the direction in which we need to move."

Responding to superintendents' concerns about the amount of time devoted to testing, Reville stated that the test should be viewed as a "performance demonstration" -- as a culminating event, much as a skater's Olympic performance represents the culmination of years of building skills and knowledge.

Sally Dias, one of the roundtable's originators, left the session on testing "very satisfied."

"We deal with so many issues and problems on a daily basis as superintendents," she said. "The opportunity to concentrate on an important issue like testing with colleagues and university researchers is invaluable."