Why you got different results on two IQ tests

Two IQ tests usually disagree for five reasons: each result has a margin of error, the tests compare you with different groups, you got better at the puzzles the second time, an unusually high or low first result tends to move toward the middle, and the tests measure different mixes of abilities. A gap of up to about 10 points between two short online tests is usually noise, not a change in you.

1. Every result has a margin

A test result is your ability plus some noise. The standard error of measurement (SEM) says how big the noise usually is: about 2 points for a long clinical battery, about 7 points for a 20-question test. Two results from the same short test can differ by a few points with nothing about you changing.

To compare two results, look at their ranges, not their centers. If the ranges overlap, the tests agree as well as tests of that length can.

	Test A	Test B
Questions	20	40
Result center	112	120
Margin (one SEM)	±8	±5
Range	104 to 120	115 to 125
Overlap	115 to 120	115 to 120

Here the centers differ by 8 points, and the ranges overlap between 115 and 120. Both results are consistent with a true value in that band, so the difference tells you little. If two ranges don’t overlap at all, something other than noise is going on, and the reasons below are where to look.

2. The tests use different norms

An IQ result places you against a reference group. A clinical test uses a sample chosen to match the census; an online test usually uses the people who chose to take it, or the planned difficulty of its questions. The same raw performance can land in different places against different groups.

Norms also age. Across 285 studies, average scores on IQ tests rose about 2.3 points per decade, the Flynn effect, so a test normed 20 years ago tends to give higher results than one normed last year. Some recent online data show the trend slowing or reversing, which is one more reason two tests with different norm dates disagree.

3. You practiced

Taking a second test of the same kind raises your score. A meta-analysis of 122 studies and over 150,000 people found an average gain of about a third of a standard deviation from the first test to the second, around 5 points, with figure puzzles gaining as much as other content. The gain shrinks when the second test uses different questions and when more time passes, and it mostly stops after the third attempt.

That gain reflects familiarity with the format: knowing the rule types, the timing and the traps. It isn’t evidence that your reasoning changed. Our guide on practice and IQ goes through the research.

4. Extreme results drift toward the middle

If your first result was unusually high or low, the next one will probably be closer to average. This is regression to the mean: part of any extreme result is luck, a good guess or a bad day, and luck rarely repeats in the same direction. It is why people who retake a test because they scored low often do a little better, with no change in ability.

5. The tests measure different things

“IQ test” covers very different tasks. A visual matrix test samples one ability, fluid reasoning. A full clinical battery adds vocabulary, general knowledge, working memory and processing speed. Even among short online tests, one might use letter and number series and another only shapes. Abilities are related but not identical, so someone strong in vocabulary and average on matrices will get a higher result from a verbal test. See fluid vs crystallized intelligence for the two broad abilities most tests mix.

Can IQ really change?

Yes, but less than test results do. Long-term studies that retest the same people from childhood find that most of the change between two tests is small or unreliably measured, and real, large changes happen in a minority. In adults, schooling has a measurable effect: a meta-analysis found that each extra year of education is worth about 1 to 5 points. Day to day, though, a different number from a different test almost always comes from the five reasons above.

How to compare two results fairly

1. Use the ranges. Overlapping ranges mean the tests agree within their precision.
2. Check what each test compares you with. A population norm and a test-taker norm are different scales.
3. Note the order. The second test benefits from practice, especially if it used similar puzzles.
4. Check what each test measures. Compare a matrix test with a matrix test.
5. Ignore single points. A test that gives one exact number hides its margin, so you can’t compare it properly.

The Cogniva test reports a range, the estimate plus or minus one standard error of measurement, and names the norm version that scored you, so you can do this comparison with our result too. The methodology shows how.

Sources

• Scharfen, J., Peters, J. M., & Holling, H. (2018). Retest effects in cognitive ability tests: a meta-analysis. Intelligence, 67, 44–66. doi:10.1016/j.intell.2018.01.003
• Trahan, L. H., Stuebing, K. K., Fletcher, J. M., & Hiscock, M. (2014). The Flynn effect: a meta-analysis. Psychological Bulletin, 140(5), 1332–1360. doi:10.1037/a0037173
• Dworak, E. M., Revelle, W., & Condon, D. M. (2023). Looking for Flynn effects in a recent online U.S. adult sample. Intelligence, 98, 101734. PDF
• Streiner, D. L. (2001). Regression toward the mean: its etiology, diagnosis, and treatment. Canadian Journal of Psychiatry, 46(1). McMaster
• Ritchie, S. J., & Tucker-Drob, E. M. (2018). How much does education improve intelligence? A meta-analysis. Psychological Science, 29(8), 1358–1369. PMC6088505
• Canivez, G. L. (2010). Review of the WAIS-IV. The Eighteenth Mental Measurements Yearbook. PDF