Electronic Yields

Charting the Race to Artificially Gauge Intelligence

After 90 minutes of intense laboring at his keyboard, Jon Corey couldn't believe his eyes. A recent graduate of NYU's nursing division, he could nimbly bandage a bullet-wound victim. But when his computer went black during an electronically administered state licensing exam, resuscitation was beyond his reach.

"I panicked," he remembers. "The staff explained that the machines shut down when you've either answered so many wrong that you're going to fail, or when you've answered enough right to pass. Given that the written test used to take two days, I assumed I'd failed." Fortunately for Corey, the move to electronic testing means quickly tabulated results; within three days he was able to call a 900 number and receive the news that he would be certified. His entire experience took less than a week from start to finish, but for others it may take slightly longer since results are now mailed out.

Electronic exams began dotting the landscape several years ago, starting with multiple-choice tests that simply grade on a pass/fail basis. Soon, though, technology put the idea of a "global classroom" within reach, and it's no surprise that testing is being altered to keep pace—and becoming big business in the process. Eager to cash in on the explosion, companies are banking on complex interactive examinations administered remotely and transferred via Internet to home sites for grading as the next wave in intelligence assessment. The types of tests that can be given annually, ranging from licensing to entrance exams, number in the hundreds, carving a highly coveted niche for software developers and venture capitalists.

While licensing exams alone can provide an impressive annual return, the real cash cow of electronic testing is the academic world, which holds a huge captive market—teenagers preparing for higher education. As a point of comparison, over 460,000 people took the Graduate Record Exam in 1998 at $96 a pop, totaling at least $43 million. Tantalizing as these numbers are, they pale next to the 3 million students expected to take the Scholastic Assessment Test 1 this year. Currently, the cost of the SAT is $23, but many often choose to retake the exam—netting hundreds of millions of dollars in revenue.

With figures like these, many software companies are now contemplating the mundane world of examinations with mercenary zeal. One of the largest proponents of wide-scale electronic testing is Educational Testing Services, the New Jersey–based company behind the bête noire of all high schoolers—the SAT, as well as the AP, GRE, TOEFL, and the GMAT, among others. ETS nabbed the contracts to design, administer, and score these standardized tests about 50 years ago, giving them a significant edge over the competition.

And they are going to be hard to beat. To keep abreast of software innovation—and insure their strong hold on the market—ETS developed a three-part plan outlining the logistics of global electronic testing, which, if all goes as planned, will redefine the scope of large-scale examinations irrevocably. According to their highly touted blueprint, structured tests given on set schedules will be a thing of the past. Instead, exams will be interactive and student-driven, with multimedia software programs taken online and delivered in the comfort of a school library for instance, making the entire experience less regimented. Plus, test takers, who can register over the phone or Internet with a credit card, will be able to work exams into their schedules conveniently, and receive their results within days. The first phase, dubbed First-Generation by ETS, is already upon us.

In the 1997 academic year, roughly 1 million examinees took electronic tests in which the questions offered were based, in part, upon test takers' previous responses, thereby tailoring them to individual skill levels. This, in education circles, is known as an adaptive—or First-Generation—test. Theoretically, an adaptive exam does not test students beyond their abilities, since an incorrect answer bars advancement to a more difficult series. Throughout the test, the computer intersperses questions from all levels, supporters claim, ensuring test takers ample opportunity to demonstrate their knowledge. Despite concerns from academics that this method will change the "standard" of standardized tests, ETS counters that electronic exams measure the same skills as their paper predecessors since they draw from an identical pool of questions. The GRE is just one of many exams administered adaptively.

The success of First-Generation is crucial to the future of electronic testing because, although the method is still in its infancy, it is a good indicator of the feasibility of future generations. Yet one noticeable absence from the myriad of exams that have already gone adaptive is the SAT—and it's not likely to do so anytime soon. "One of the main concerns we have is cost," says Jeffrey Penn, spokesperson for the College Board, which regulates the SAT. "Who will pay for all this? How widely available is the technology? Everybody has access to a number two pencil, not everyone will be able to use a computer."

Some educators also fear that electronic testing will further exacerbate the existing correlation between household earnings and academic test scores. Bob Schaeffer, of FairTest, a national center for fair and open testing, worries about students who might not frequently use computers. "We already know one of the best indicators of how a student is going to fare on these exams is the parents' income," explains Schaeffer. "Owning a computer isn't a prerequisite for doing well, but clearly students who are comfortable with the machinery will have, at the very least, a psychological advantage."

Next Page »