With President Biden’s nomination for the next Supreme Court justice, standardized tests are once more in the news, along with the fallacies offered on both sides.
What both sides fail to admit, at least publicly, is that standardized tests are a tool, nothing more and nothing less. If the tool is poorly constructed, it won’t work well. Even if it’s decently constructed, if it’s applied poorly, the results may not be accurate.
Often overlooked is the fact that tests such as the ACT and the SAT were initially effectively designed to measure the qualities needed by white, predominantly male, upper middle class students to succeed in college. The tests have proved to be, despite claims to the contrary, moderately effective for determining collegiate success for that socio-economic group and for certain hard-working Asian minority students. They’re less effective for other socio-economic groups, for a number of reasons.
Well-designed standardized tests will measure certain results accurately, no matter what detractors claim. The problem is that the results they measure aren’t precisely what the proponents of such tests claim. Tests such as the ACT, SAT, LSAT, MCAT, or GRE measure not only certain types of knowledge, which is their stated purpose, but they also measure indirectly other abilities.
The tests measure the ability to read and comprehend quickly, to recognize and analyze patterns, and to quickly recall facts and techniques and to apply them to a situation, problem, or text presented in verbal or mathematical form.
That means that someone who takes the test who reads quickly and accurately has a tremendous advantage on timed tests, and that advantage effectively allows the test-taker more time and places more pressure on the test-taker who knows just as much if not more but who cannot read as fast. In addition, the tests often don’t measure depth of knowledge or the ability to solve complex and multi-faceted problems.
Tests given at the primary school level can reflect as much the students’ socio-economic backgrounds as their intelligence, because a student from a well-read and well-educated upper middle class background will often have greater exposure to the terms and structures of testing.
Such tests are biased, no matter what backers of the tests say, against individuals who do not read the test language quickly, against individuals from a differing socio-economic background who don’t know all the indirect cultural referents embodied in the test, and against those who have high intellectual levels but who do not process information quickly.
What that does mean is that the tests are generally more accurate in assessing the abilities of an upper-middle-class male who reads moderately quickly than in assessing actual intellectual abilities of someone who comes from a different background.
Such tests can be a useful indicator, but they shouldn’t be used as the sole indicator. Unfortunately, the problem today is that many of the other indicators used previously have become useless. Grade inflation has gotten to the point where there’s almost no statistical difference between students in many schools, and where class rank is often decided by a single bad mark in a single course in the ninth grade [FYI, this isn’t hyperbole]. Neither are outside activities.
Tests also don’t reflect the character and determination of the test-taker. Every year, my wife the professor sees students with good high school grades, high test scores, and good native ability flunk out because they were unwilling or unable, for other reasons, to do work that should be well within their capabilities.
But right now, standardized test scores, flawed and biased as they are, are the most accurate predictor of performance for their original target population, simply because there aren’t any other reliable measures.
For everyone else, whatever other yardsticks are being used to determine their abilities are in fact somewhere between estimates and guesstimates.