How accurate are online IQ tests?

An online IQ test can give a useful estimate, but a short one can’t give a precise number. A full clinical battery with dozens of tasks has a margin of about 2 points; a 20-question online test has a margin of about 7 to 8 points, because each question carries more of the weight. That’s why an honest short test reports a range, publishes how its norm was built, and says what the result can’t be used for.

Why does a short test have a wider margin?

Every test result is the true ability plus some noise: a lucky guess, a misread figure, a distraction. Psychologists measure the noise with the standard error of measurement (SEM). It depends on the test’s reliability, which is how consistently it ranks the same people, and on the scale’s standard deviation of 15 points:

SEM = 15 × √(1 − reliability)

More questions means more reliability and a smaller SEM. The table below shows what published reliabilities imply.

Test	Questions	Reliability	SEM in points
WAIS-IV full-scale result, given in person	10 core subtests	.97 to .98	about 2
ICAR-16, an open research test taken online	16	.81	about 6.5
A 20-question matrix test (estimated)	20	about .79	about 7

The last row is our calculation. The 11 matrix items of the open International Cognitive Ability Resource had a reliability of .68 in a large online sample; the Spearman-Brown formula, which predicts reliability when a test gets longer, puts 20 similar items at about .79.

So a 20-question test is not useless. It is about three times less precise than a long clinical battery, and a result reported as one number hides that.

What does that mean for a range?

The standard way to show the margin of a single result is the estimate plus or minus one SEM. On a 20-question test that is about 7 to 8 points either way. A band that captures the true value 95% of the time would be about twice as wide.

This is how our own test reports results. The Cogniva test has 20 puzzles, and each range is the estimate plus or minus one standard error of measurement: 8 points for most results, 9 or 10 at the top of the scale, where there is less room to measure. A result of 17 out of 20 on the current norm reads “114 to 130”, never “122”. The methodology shows the current norm, its size and its date.

Who are you compared with?

An IQ estimate places your result against a reference group, called the norm. Clinical tests build their norms from samples chosen to match the census; the WAIS-IV norm has 2,200 adults, stratified to match the U.S. census.

Online tests usually can’t. Their norm comes from the people who chose to take them, and the research team behind the largest open online dataset, about 394,000 American adults tested between 2006 and 2018, says plainly that their sample is not nationally representative. Volunteers who look for IQ tests are not a random slice of the population.

That is why a careful online test says its result places you on its own scale. Ours does: until 300 people complete it, the Cogniva norm comes from the planned difficulty of each puzzle, and after that from completed tests, recalibrated at 300 and 1,000 completions.

Four checks for any online IQ test

1. Does it give a range or a single number? A single number from 20 questions claims a precision the test doesn’t have.
2. Does it say who the norm is? “Compared with the general population” needs a representative sample, and a test that has one says how it was drawn.
3. Does it publish its size and date? A norm with no sample size and no date can’t be checked.
4. Does it say what the result is not for? An honest online test says it isn’t for hiring, admissions or medical decisions.

A test that passes all four can still be wrong for you on a given day. It is unlikely to be pretending.

What makes your own result less accurate?

• Practice. People score about a third of a standard deviation higher the second time they take a similar test, around 5 points, and figure puzzles show the same gain. If you have solved many matrix puzzles, your result will run high. Our guide to practice covers the studies.
• Conditions. A phone in a noisy room is not a quiet testing office. Small screens also make figures harder to read.
• Guessing. On a six-option puzzle, a guess is right one time in six. A few lucky guesses move a short test more than a long one.
• A different test. Two tests built from different tasks measure slightly different mixes of abilities, so results can differ by more than their margins. We explain how to compare two results in why you got different results on two IQ tests.

So, are online IQ tests accurate?

Some are accurate enough for what they claim, and many claim too much. A short, well-built online test can tell you roughly where you stand on its own scale, give or take 7 to 8 points. It can’t give you a precise IQ, compare you with the whole population unless it has a representative norm, or replace a full assessment by a psychologist.

Sources

• Canivez, G. L. (2010). Review of the Wechsler Adult Intelligence Scale, Fourth Edition. In The Eighteenth Mental Measurements Yearbook. Buros Institute. PDF
• Condon, D. M., & Revelle, W. (2014). The international cognitive ability resource: development and initial validation of a public-domain measure. Intelligence, 43, 52–64. doi:10.1016/j.intell.2014.01.004
• Dworak, E. M., Revelle, W., & Condon, D. M. (2023). Looking for Flynn effects in a recent online U.S. adult sample: examining shifts within the SAPA Project. Intelligence, 98, 101734. PDF
• Scharfen, J., Peters, J. M., & Holling, H. (2018). Retest effects in cognitive ability tests: a meta-analysis. Intelligence, 67, 44–66. doi:10.1016/j.intell.2018.01.003
• Spearman-Brown prediction formula: Brown, W. (1910) and Spearman, C. (1910), British Journal of Psychology, 3. Overview
• Cogniva methodology: norm version, size, date and margin, updated when the norm changes.