TECHNOS

July 20, 2008

TECHNOS QUARTERLY Winter 1992 Vol. 1 No. 4

Assessment: Let's See What Our Kids Can Do

By Eva L. Baker

Assessment is a hot topic an American education today, especially when national standards are included in the discussion. UCLA's Eva Baker is at the forefront of research into student assessments that will support educational reform in this country. But, before we find new ways of testing U.S. students' capabilities, Baker warns, we must address some basic questions, one of the most important being that of the quality of schooling.

Although the autumn of an election year is usually not the optimal time for national unity of spirit, the Presidential candidates agreed on at least one (and perhaps only one) educational issue. Both (even all three) went on record in support of a “national system of examinations.” In effect, this year's candidates for President said: Let's see what our kids can do.

To those in schools, the idea of a national system of assessment may seem redundant. American children already are tested frequently by standardized and other tests, some claim more than children in other countries. Various assessments and tests are required by school district boards, by states and by federal programs. Classroom teachers also make their own tests to evaluate student achievement. These tests are purportedly used to monitor system progress, to communicate goals, to enforce accountability, and to diagnose student needs. Why are we focused on testing again, and why now?

The answers reside in two facts: first, the perceived failures of the present American system; and second, the complexities of trying to reform all at once a national education system that has at its core local educational responsibility.

Let's start with the premise that many Americans are dissatisfied with their educational system. With only minor dissent, a large number of policymakers, business leaders, and educators assert that American students are not sufficiently challenged intellectually and that an across-the-board elevation of academic standards is required. They think school should do a better job with their children, and they are right. But there is no consensus about exactly how to fix education.

People also differ on who should be the prime beneficiaries of reform. Many contend that demography and morality require us to place most of our attention on children who are poor, who also may be members of minority groups, or non-native speakers of English. Others believe that our focus should be on teachers, and we must not neglect children disabled in different ways or the specially talented and gifted. Platforms for change splinter even more when individuals argue for educational reform for specific emphases, for work-force skills, for foreign languages, for the interpretive and performing arts, for interdisciplinary learning, for multiculturalism, for new curricula, and of course, for technology.

Great Expectations

What makes assessment so appealing is that it apparently can be applied across the board to all students and to all subject matter emphases. In January 1991, the National Council on Education Standards and Testing, a group sanctioned by the Bush administration, Congress, and the National Governors Association, released its report, Raising Standards for American Education. This report presented a proposal to reform education by focusing on national standards and a national system of assessment.

The envisioned assessment system attempts to support local educational control and to avoid a single national test. These goals would be accomplished by states or consortia in their development of assessments of national standards—a strategy used with success in the Federal Republic of Germany. In the full Council's deliberations, in the heated debate of the Council's Assessment Task Force, and in an independent report released by the Office of Technology Assessment, “Testing in American Schools: Asking the Right Questions,” it is clear that the public, educators, and policymakers have multiple expectations for assessment and make many assumptions about the validity of the information that tests and assessments provide.

The list of expectations is a long one:

Communication of standards
Impetus for curriculum reform
Diagnostic information to help students
Motivation of all participants
Clear benchmarks for progress
Management and accountability strategy
A basis for individual, teacher, school, and state comparison
A route to renewing confidence in the schools
A method to make good selections of students for jobs or higher education.

On the other hand, skeptics about assessment argue that its benefits are largely illusory. While they may support the claim that required multiple-choice tests lead to a fragmented, unchallenging curriculum, they also decry the jump in logic that assumes high-quality assessments inevitably will have a positive impact on teaching and curriculum. And they counter with uncomfortable questions about the quality of knowledge we have to achieve these goals. Do we know how to teach challenging tasks to most or to all of our students? If teachers could do so, critics of assessment believe, surely they would be doing so now.

Although the different sides argue with varying levels of passion and reason, very few proponents see assessment as the single cure for educational ills. Even fewer would suggest that any particular assessment can serve adequately all the assessment purposes (or, depending on your view, fantasies) set forth earlier. But assessment is a seductive policy option, since it appears to be less expensive than most alternatives. Its very indirectness is its particular charm. Better to use assessments to pull teachers toward appropriate instruction, materials toward appropriate content, children toward motivated learning, than to work on the relevant processes themselves. No one can fault the desire to be efficient in this period of tight resources. But there is as yet no evidence that assessments will have the impact desired. In fact, there is relatively little research information on a number of key assessment issues. However, progress is being made.

Important Issues

A case in point is the national attention to new forms of assessments of student performance, assessments that use projects, experiments, essays, and portfolios rather than more familiar types of answer-sheet testing. These assessments require complex thinking, tasks with many steps, open-ended and creative answers, and reliance on the best ideas of subject matter and interdisciplinary experts. My colleagues at the National Center for Research and Evaluation, Standards, and Student Testing (CRESST) will provide information on how valid some of these tests are for different uses. They also will provide guidelines to develop and validate similar measures. Already, we have results on history and science assessments that we believe will make performance assessment significantly more cost effective and valid.

But to begin to achieve the aspirations of the public for test information—and to assure for ourselves and others that testing supports rather than impedes student learning—additional questions require sustained attention. Questions such as:

Q: How fair are performance assessments to children of different backgrounds? How do we assure that economically disadvantaged or limited English proficient, or other special populations, such as the learning disabled, are fairly treated?

Q: How is fairness influenced by different administrative and scoring procedures? How should comparability of assessment design, administration, and test results be determined?

Q: What is the impact of various kinds of instruction on these new assessments? Do they measure the things we wish to teach? What kinds of important learnings can be measured? How general or transferable is performance from project to project or assessment to other important accomplishments? How do these assessments predict readiness for the work force or postsecondary education?

Q: Are such assessments less or more corruptible than traditional tests? How trustworthy are our findings, and what steps can increase our confidence in test performance?

Q: Which, if any, multiple purposes of assessment can be served simultaneously with validity? Can an assessment contribute to teaching and learning and accountability? What is the impact of new assessments on students and the quality of life in school?

Q: What are the best ways for teachers to be involved in the design and use of new assessments? What ways are cost-effective? What ways contribute directly to student learning? How can the design of new assessments build upon our knowledge of thinking and learning?

Q: How should we measure students' ability to integrate across subject-matter areas? How should we measure their efforts, thinking processes, and habits of mind?

Q: How should we combine the range of assessment options before us to provide the best information, with the most positive consequences, and with reasonable costs?

Beyond the solutions to these problems, we must develop improved ways to communicate the results of student assessments to parents, to teachers, to the students themselves, and to the public at large. Some researchers believe that testing has developed in the manner it did in part because of reporting issues. Policymakers became used to easy, seemingly clear answers that depended on numbers such as an average score, a specific percentile, a grade level. Changing the basis of assessment to more concrete analyses of children's performance will change as well the kind of information policymakers and the public will be given.

The National Center for Research on Evaluation, Standards, and Student Testing (CRESST) is one of 25 national education research centers funded through the Office of Educational Research and Improvement. CRESST's mission is to improve methods of measuring student performance, focusing primarily on performance-based assessments such as portfolios, self-evaluations, student projects, student journals, and a variety of other non-standardized assessment techniques. The concept behind these new prototypes is to see how students actually perform, not just how well they test. For more information, write to UCLA SCE/CRESST, 300 Charles E. Young Drive North, GSE&IS Bldg. 3rd Floor/Mailbox 951522, Los Angeles, CA 90095-1522, 310-206-1532.

Research on how to communicate complex information to the public is essential if real improvements in assessment are to be possible. We need to find ways to show details and realities of what children can do, rather than to rely exclusively on abstractions such as scores and averages. We must find ways to reach out to parents who need to have good information about their schools in a form that can support their children's growth and accomplishments.

Quality Control

Most of the research questions I've listed here are under study now; some of them will take considerable time to answer. All share a focus on student outcomes. I believe we must make a substantial effort to address still another class of questions about educational reform. We must focus the attention of scholars and talented practitioners on how to assess the quality of schooling itself. Clearly, under certain conditions, we can infer something about the quality of instruction from student performance.

But many of us worry that we have not paid enough attention to the quality of description of student experiences. Why should we expect children to do well in school—and have high test scores or sparkling accomplishments—if their schools are not safe places, or if they must worry about the security of the route they take on the way to and from school? Why should we expect children to excel, when they do not have challenging textbooks or enough of them to go around? How can students do homework without enough paper or technological support? To understand the results from any tests, whether multiple-choice or performance-based, we need to be able to make accurate statements about what school experiences are like. We need to look at school quality in a number of ways, to document excellence where we find it, to condition our understandings of student performance, and to identify the expectations of quality for all schools.

Contextual Considerations

During the 1991 meetings of the National Council on Education Standards and Testing, issues of school experiences and environments were discussed in at least two different ways. First, they were cast as school and system “delivery standards,” a framing which implied that agreement is possible on certain attributes of schools. In this role, standards would be set for educational institutions parallel to those expected for individual students. Second, school experiences were emphasized through descriptive information about school contexts (such as transiency), inputs (such as resources), and processes (such as time for learning). These were discussed as part of needed conditions before assessments were used for high-stakes purposes, such as promotion or graduation.

Not surprisingly, the assessment of school processes was an extremely controversial topic during the Council meetings. To some, it implied a prescriptive or controlling function, a way to homogenize schools and classrooms from inside the beltway. To others, it conjured up horrors of more checklists, paperwork, and mandated but unread reports. But in intent—the need to link explicitly school processes with student outcomes—the direction of the discussion seemed appropriate to me. Both student outcomes and school processes must be documented: the first, to show the effects of our efforts; the second, to begin to assess responsibility for performance. Without a dual system, educational equity—a major goal of all reform—has little chance of success.

Technological Implications

Where does technology fit in the standards and assessment reform efforts? Technology offers us more than the chance to make our assessments efficient. Even beyond the use of sophisticated multimedia projects to stimulate student thinking or to create products as we have seen in some of the Apple Classroom of Tomorrow research, technology has a special role to play in assessment. It may be the only means we have available for exploring certain capabilities of students. For example, at UCLA with Apple Computer, Inc., support, we have been working on computer-based assessments that attempt to get at students' conceptual understandings without exclusively relying on their ability to write. Students construct HyperCard databases or concept maps to illustrate the relationships among principles, facts, and conclusions. Other studies focus on database search strategies used by students as a measure of their familiarity with subject matter. Such strategies are potentially extremely important for students who don't have a firm grasp of English language skills but who, nonetheless, know a good deal about particular content areas.

Research is also under way in a variety of settings to explore the limits of automated scoring of students' work, including maps and essays. Looking at team simulations in the work force readiness area provides another example of how technology can further new opportunities for assessment in exciting ways. We also are exploring the use of authoring systems to develop performance assessments as a means of allowing greater teacher participation.

What Now?

If rhetoric and political interest is the standard, the prospects for improving our educational performance have never been more encouraging. But our transformation still has a way to go. To punch through rhetoric, to create world-class student performance, we need to design world-class schools-schools with commitments to technology, high-quality teaching, engaging materials, and sensible assessment. The assessment enterprise seems to many like the best place to start; but the challenges we have are so complicated, we probably need to start anywhere we can find energy, commitment, and enthusiasm. The fundamental message of new assessments is one that is essential to our future. We now say to our schools, our teachers, our kids: Show us what you can do.

Illustration by Dave Coverly.

The work reported here was partially supported under the Educational Research and Development Center Program cooperative agreement number R117G10027 and CFDA catalog number 84.117G as administered by the Office of Educational Research and Improvement (OERI), U.S. Department of Education. The findings and opinions expressed in this report do not reflect the position or policies of OERI or the Department.

Eva Baker is professor of educational psychology and social methods in the Graduate School of Education at the University of California, Los Angeles (UCLA). She is director of UCLA's Center for the Study of Evaluation and codirector of the National Center for Research on Evaluation, Standards, and Student Testing. Baker has served as president of the Educational Psychology Division of the American Psychological Association. As a member of the National Council on Education Standards and Testing, Baker chaired the Assessment Task Force.