Electronic Yields

Charting the Race to Artificially Gauge Intelligence

After 90 minutes of intense laboring at his keyboard, Jon Corey couldn't believe his eyes. A recent graduate of NYU's nursing division, he could nimbly bandage a bullet-wound victim. But when his computer went black during an electronically administered state licensing exam, resuscitation was beyond his reach.

"I panicked," he remembers. "The staff explained that the machines shut down when you've either answered so many wrong that you're going to fail, or when you've answered enough right to pass. Given that the written test used to take two days, I assumed I'd failed." Fortunately for Corey, the move to electronic testing means quickly tabulated results; within three days he was able to call a 900 number and receive the news that he would be certified. His entire experience took less than a week from start to finish, but for others it may take slightly longer since results are now mailed out.

Electronic exams began dotting the landscape several years ago, starting with multiple-choice tests that simply grade on a pass/fail basis. Soon, though, technology put the idea of a "global classroom" within reach, and it's no surprise that testing is being altered to keep pace—and becoming big business in the process. Eager to cash in on the explosion, companies are banking on complex interactive examinations administered remotely and transferred via Internet to home sites for grading as the next wave in intelligence assessment. The types of tests that can be given annually, ranging from licensing to entrance exams, number in the hundreds, carving a highly coveted niche for software developers and venture capitalists.

While licensing exams alone can provide an impressive annual return, the real cash cow of electronic testing is the academic world, which holds a huge captive market—teenagers preparing for higher education. As a point of comparison, over 460,000 people took the Graduate Record Exam in 1998 at $96 a pop, totaling at least $43 million. Tantalizing as these numbers are, they pale next to the 3 million students expected to take the Scholastic Assessment Test 1 this year. Currently, the cost of the SAT is $23, but many often choose to retake the exam—netting hundreds of millions of dollars in revenue.

With figures like these, many software companies are now contemplating the mundane world of examinations with mercenary zeal. One of the largest proponents of wide-scale electronic testing is Educational Testing Services, the New Jersey–based company behind the bête noire of all high schoolers—the SAT, as well as the AP, GRE, TOEFL, and the GMAT, among others. ETS nabbed the contracts to design, administer, and score these standardized tests about 50 years ago, giving them a significant edge over the competition.

And they are going to be hard to beat. To keep abreast of software innovation—and insure their strong hold on the market—ETS developed a three-part plan outlining the logistics of global electronic testing, which, if all goes as planned, will redefine the scope of large-scale examinations irrevocably. According to their highly touted blueprint, structured tests given on set schedules will be a thing of the past. Instead, exams will be interactive and student-driven, with multimedia software programs taken online and delivered in the comfort of a school library for instance, making the entire experience less regimented. Plus, test takers, who can register over the phone or Internet with a credit card, will be able to work exams into their schedules conveniently, and receive their results within days. The first phase, dubbed First-Generation by ETS, is already upon us.

In the 1997 academic year, roughly 1 million examinees took electronic tests in which the questions offered were based, in part, upon test takers' previous responses, thereby tailoring them to individual skill levels. This, in education circles, is known as an adaptive—or First-Generation—test. Theoretically, an adaptive exam does not test students beyond their abilities, since an incorrect answer bars advancement to a more difficult series. Throughout the test, the computer intersperses questions from all levels, supporters claim, ensuring test takers ample opportunity to demonstrate their knowledge. Despite concerns from academics that this method will change the "standard" of standardized tests, ETS counters that electronic exams measure the same skills as their paper predecessors since they draw from an identical pool of questions. The GRE is just one of many exams administered adaptively.

The success of First-Generation is crucial to the future of electronic testing because, although the method is still in its infancy, it is a good indicator of the feasibility of future generations. Yet one noticeable absence from the myriad of exams that have already gone adaptive is the SAT—and it's not likely to do so anytime soon. "One of the main concerns we have is cost," says Jeffrey Penn, spokesperson for the College Board, which regulates the SAT. "Who will pay for all this? How widely available is the technology? Everybody has access to a number two pencil, not everyone will be able to use a computer."

Some educators also fear that electronic testing will further exacerbate the existing correlation between household earnings and academic test scores. Bob Schaeffer, of FairTest, a national center for fair and open testing, worries about students who might not frequently use computers. "We already know one of the best indicators of how a student is going to fare on these exams is the parents' income," explains Schaeffer. "Owning a computer isn't a prerequisite for doing well, but clearly students who are comfortable with the machinery will have, at the very least, a psychological advantage."

The reluctance of the College Board to implement interactive testing may not bode well for ETS's long-term goals. Although it has successfully moved several exams to electronic platforms, with fees absorbing most of the cost, nothing compares to the vast numbers involved with the SAT. With millions of potential test takers a year, many of whom opt for multiple sittings, the reality of setting up enough secured test sites and ensuring access to all schools in all states is overwhelming—so much so that the Board is hesitant to embrace First-Generation, let alone future ones.

To help sidestep this obstacle, ETS contracted the Sylvan Technology Center, a Maryland-based company with secure test sites around the country, to provide them access to already-established locations. This partnership has allowed ETS to avoid the exorbitant cost of building their own sites, which would have rendered their long-term strategy near impossible. But, despite the successful integration of Sylvan Center, the Board still hasn't committed to switching gears and ETS can't move forward without its permission.

However, the College Board may be forced to confront the future sooner than it would like. The possibilities of savvy software have attracted investors as well as instructors, many of whom believe that standardized tests do not accurately assess student aptitude. According to an ETS report written by Randy Bennett, Speculations on the Future of Large-Scale Educational Testing, "Change should come first in the test design, which reflects an outmoded psychology." Schaeffer would also not bemoan the disappearance of the present format, which, he says, carries an innate bias. "Our current system is based on the Socratic methodidea of asking a question when you already know the right answer—and there are several cultures that are not accustomed to that style of interrogation."

Enter Next-Generation, the second phase of ETS's game plan, which will aggressively utilize multimedia and interactive features to gauge reasoning. To demonstrate what the future might hold, Bennett cites the AP exam in U.S. history, in which students could use excerpts from diaries, articles, and maps to formulate answers. An interactive question might run a newsreel or a reenactment of a political speech and ask students to discuss the events depicted, thereby measuring audio, visual, and reading comprehension skills simultaneously. Foreign language exams would lend themselves particularly well to such testing, with oral responses captured through computer- controlled microphones and cameras.

The final phase of ETS's brave new world involves long-distance learning, which won't be possible until Internet security becomes inviolable. When it is, students anywhere in the world wishing to take a GRE, for example, will proceed to a secure test area, have a retina or a palm scan, and begin an interactive examination that will be sent over the Web back to the home site. This, Generation "R," is the most ambitious phase of all and, ultimately, will include virtual reality classes that essentially allow a student to "attend" a university from halfway across the globe.

Whether or not ETS is able to proceed this far is undecided, but the arrival of Next-Generation is imminent. While software developers wrangle out the details, some who will be most affected by its onset aren't impressed. "There are kids who test well and always get good scores, even when they don't study," says Gidon Fisher, a senior at Manhattan Solomon Schechter High School. "The ones in my school who don't score well hire a tutor to teach them the tricks." When asked about the possibility of interactive testing eliminating such an advantage, Fisher remains unconvinced. "It might help to keep me awake during the test," he says, "but that's about it."

Show Pages
My Voice Nation Help