The recent 5-4 Supreme Court ruling on the New Haven Fire Department vocational advancement exam in Ricci v. DeStefano once again stirs the waters on the question of how to choose the best from among a crowd. (See our entry "The Outliers of Law–Embracing Heresy".) The "best" in this case was determined to be simply the highest scorers, even if those scores seem to imply discrimination against a particular group.
What’s Sonia Got to Do With It?
A lot of press has been devoted to parsing whether Sonia Sotomayor’s vote with the majority at the appeals court, which affirmed throwing out the test results, implies her personal position on affirmative action.
A look at Sotomayor’s own test scores gives an interesting gloss to the discussion. She was, by her own admission, an "affirmative-action baby" who did not do well on her SATs and LSATS, or at least not as well as her fellow students at Princeton and Yale. Yet she went on to graduate from Princeton with highest academic honors and has reached the upper echelons of law practice. As Walter Kirn said in a recent New York Times article about his own experience at Princeton, "the poorer and browner of my classmates — particularly the women — seemed to study twice as hard as I did, clocking endless hours in the library and forgoing weekend parties for late-night cram sessions. Maybe their SAT scores were lower than mine, but they ranked higher than I did on the effort scale. And on the bravery scale too."
So was this a case of retrospective justice-making by Ms. Sotomayor?
Regardless of what Sotomayor was doing in the public sector, the glaring lesson to be taken from her own story is that aptitude assessments are not the last word on potential for achievement.
The Texas Experiment
In 1997, Texas House Bill 588, better known as the "Top 10 Percent Law," was passed, guaranteeing high school graduates who ranked in the top 10% of their senior class, regardless of their SAT or ACT scores, admission to a state institution. While hotly contested at the time as risking the influx of less able students, it is a law that school administrators and legislators agree "by any measure of public policy is a success."
Not only did the 10% plan in Texas get more minority students into top public universities with race-neutral criteria, it spawned similar programs in California and Florida and the consideration of many other states. (Due to its immense popularity, last month the Texas Legislature agreed to limit to 75% of its freshman slots the number from the program that their flagship school, the University of Texas at Austin, had to admit.) According to the most recent issue of Inside Higher Ed, "every internal study that… the UT system conducted and every external study has shown that the 10 percent students, relative to others, have done better by any measure — lower attrition rates, graduate in shorter time periods," etc.
As Malcolm Gladwell wrote in his 2001 New Yorker article "Examined Life": "Critics of the policy said that it would open the door to students from marginal schools whose SAT scores would normally have been too low for admission to the University of Texas—and that is exactly what happened. But so what? The ‘top ten percenters,’ as they are known, may have lower SAT scores, but they get excellent grades. In fact, their college GPAs are the equal of students who scored two hundred to three hundred points higher on the SAT [emphasis added]. In other words, the determination and hard work that propel someone to the top of his high-school class—even in cases where that high school is impoverished—are more important to succeeding in college (and, for that matter, in life) than whatever abstract quality the SAT purports to measure. The importance of the Texas experience cannot be overstated."
Predicting the Best Lawyers
A number of studies have looked for what might predict eventual success as a practicing lawyer. Evidently LSAT scores, and not undergraduate grade point averages, are the best indicators of academic performance in the first year of law school, and academic performance in the first year of law school appears to be the best predictor of whether the new graduate will pass his/her state bar exam on the first attempt. There is also a very strong correlation between the personality attribute of pessimism and law school grades, i.e., the higher the pessimism, the higher the grades.
But none of these factors–undergraduate grades, LSAT scores, law school grades–gives us the key to determining who is likely to be at the top of the lawyering heap.
A New Kind of Test
Marjorie M. Shultz, a retired Berkeley law professor, together with the Law School Admissions Council which administers the LSAT and a university psychologist, are trying to remedy that, according to The New York Times. After the adoption of California’s Proposition 209, which banned consideration of race in admissions, they surveyed and interviewed judges, clients and law graduates throughout California about the qualities most important in the practice of law. Twenty-six characteristics were identified as "effectiveness factors," including the ability to write, manage stress, listen, research and solve problems. From this data, a test was developed in hopes of measuring raw lawyerly talent.
Scores on the first administration of the test to over 1,100 volunteers were coordinated with the participants’ LSAT sores and grades from college and law school. The results? The new test is no better than others indicators in predicting how well participants do in law school, but it is able to predict effectiveness in the practice of law. As a bonus, the test did not produce a gap in scores among racial or ethnic groups, as does the LSAT. Plans are underway to replicate the study in other parts of the country.
Learning From What We Know
Some contend that the Supreme Court decision in Ricci v. DeStefano will in fact have little practical relevance since many advancement decisions in both the public and private sectors are no longer based on written tests but on applicants’ performance in staged scenarios.
Meanwhile, the legal industry persists in betting all of its marbles on law school class standing–the measure we know for sure is not an accurate predictor of success.
But what, instead of class standing, should we be looking for in? The California factors, the hard work gene? And more critically, will we have the courage necessary to rise to the top of the heap when it comes to adopting sophisticated recruiting techniques?
Once we’ve mastered that selection process, what may be an even more daunting challenge is structuring our personal and professional lives so as to produce not only success in our legal practice but also success in our relationships. See David Brooks "The Way We Live Now" and our recent entry "Who Is the Best and the Brightest?"
So many possibilities.