Performance Assessment:

Science Knowledge in Action

Computers are beginning to change the way we teach students; soon they will revolutionize the way we assess those students' understanding. Why? Because computers are an active medium and student understanding is best measured when it is being applied to active situations. Print materials can explain things to you; computers can guide you to learn by trial and error. Printed tests can ask you questions; computers can pose complex challenges and offer you alternative ways to achieve a goal. We can infer a great deal by observing how students rise to the challenge, what use they make of the tools at their disposal, how they react when their first idea doesn't pan out, and how they interpret their results.

No matter how well designed a test is, it is difficult to determine whether someone really understands a concept and can apply that understanding in a practical situation simply by asking a series of questions. The very act of answering questions on a test puts one into a special mindset, calls up specific associations, and prepares one to think about a situation in a particular way. For example, if we are teaching genetics, we will want to know whether our students know how dominant and recessive traits are inherited. Do they understand how the appearance of an organism is related to the specific forms of the genes it carries? Do they know that every sexual organism inherits half its genes from its mother and half from its father? If we simply ask those questions outright, however, many students will answer them correctly, even though most might be incapable of using their "school knowledge" in a practical context. The Horns Dilemma computer activity, developed by the Modeling Across the Curriculum project, tests students' knowledge of genetics and also assesses their ability to apply that knowledge. Here's how it works.

Dragons illustrate the problem

We start by describing a real-world situation involving a genetic disease that results from the inheritance of two copies of a relatively rare recessive allele. The story involves a young girl named Sara who has cystic fibrosis. The students are told that this disease is genetically inherited and that it is caused by only one gene. They are also told that neither of Sara's parents, nor any of her siblings or relatives, has any hint of the disease. How can this be? The students are left to ponder that question as we switch to a seemingly unrelated one involving the not-so-real world of dragons.

The crux of the Horns Dilemma activity. Note that Aleph, the father dragon, is selected, thus it is his chromosomes that we see. Since he has two dominant "H" alleles, he cannot have a hornless (hh) baby, and it will be necessary for the student to change one of his horns genes in order to solve the problem. The text makes it clear that this "genetic engineering" is allowed, but stops short of suggesting that the student do it. It is a reasonable inference, therefore, that students who go on to try to make a hornless offspring without altering the father's genes do not really understand how a recessive trait can be inherited from parents who do not exhibit it.

One of the traits of the dragon species, as we have programmed our BioLogica(tm) program, is the possession or absence of horns, with horns being dominant over hornlessness. Specifically, the horns trait is governed by a single gene that comes in two varieties, or alleles: the dominant "H" and recessive "h." A dragon with two recessive "h" alleles has no horns; any other combination of alleles (HH, Hh, or hH) will result in a dragon that has horns. We show the student two dragons-one male and one female-both with horns, and we ask whether it is possible for them to have a baby that has no horns. Initially, the genes of these dragons are not visible. Unless the student uses a special "chromosome tool," she can only see the dragons' physical traits and there is no way for her to tell whether these particular dragons can have a hornless baby.

In fact, one of the two dragons has two "H" alleles, making it impossible for this pair to have a hornless baby. (For them to have a baby with no horns, each parent dragon would have to have one "H" and one "h" so that each could donate a recessive "h" to the offspring.) Through the magic of computer simulation, the student can do more than just look at the dragons' chromosomes, she can alter them if she chooses, as long as each parent still has horns. And to achieve the goal of creating a hornless baby dragon, she will have to do just that-changing an "H" to an "h" to give both parents a recessive allele.

Monitoring student actions

For assessment purposes, we now have two actions to monitor, each of which bears significantly on the student's understanding: if she fails to look at the genes of the parent dragons, she is probably not thinking "genotypically," but considering only the outward appearance of the dragons. On the other hand, if she looks at the genes, but does not change them, it is a reasonable inference that she does not realize the importance of each parent carrying that recessive allele. Because the activity is run on a computer, we can easily monitor which choices the student makes-by recording mouse clicks or paths along the branching story line.

In later stages of the activity the student runs meiosis on each parent and fertilizes the resulting gametes to produce an offspring. If that offspring has horns the computer will offer minimal guidance, in the form of hints that become more and more explicit until the student succeeds in making a hornless baby dragon. Thus, the level of aid required becomes yet another useful indicator of the student's preparedness for the task.

But how do we know, in the end, whether the student has connected the manipulation of the dragon model to the real world of genetics? Once the hornless baby has been created, the computer returns to the story of Sara, and asks a series of questions, including the critical one: what does the hornless dragon activity have to do with Sara's cystic fibrosis? The student who realizes that hornlessness in dragons and cystic fibrosis in humans are both passed on the same way-through the inheritance of two copies of a recessive allele-has integrated "book learning" with a real-life situation, and is on the way to thinking and behaving like a geneticist.

Assessments of this kind, in which one infers the state of students' knowledge and understanding from their problem-solving actions, are called "performance assessments," and much of our current research is devoted to learning how to design them well. We are figuring out what kind of challenges to pose, how to provide timely feedback without being intrusive, what data to collect, and how to analyze that data. We still have a lot to learn before we can realize the full potential of our technology, but preliminary results hold much promise for science, technology, and vocational education. More good news: the technology for capturing student actions and measuring student understanding continues to improve. Our data collection and analysis software is now robust, powerful, and widely available. Over 260 schools have registered to use our modeling software and many of these have registered their students with us, enabling us to collect performance assessment data and generate reports for them. Let us know if you'd like reports for your students, too. Our modeling software is available for biology, physics, and chemistry.