TL;DR
With giveaway fields removed, Opus 5.5 ranks people on 106 survey measures, from IQ to attitudes, as well as a regression trained on thousands of labelled respondents (mean Spearman .450 against .448), and its accuracy tracks the regression's test by test (r = .87).
The best-predicted test is the ASVAB (r = .86); short knowledge quizzes are the worst (ALIKE .32).
Item by item it beats chance (kappa .35) but barely beats population base rates (log-loss skill .02).
From the questions alone, its factor loadings correlate .93 with the real ones across 608 items, and AI-built reference populations place real people within .04 to .27 SD of their true percentile on 28 of 29 batteries.
On a private dataset no model could have seen, it again reaches about 83% of a supervised model's item-level skill, as on the public surveys.
Bigger models close more of the gap to supervised models (Haiku 5.5 18%, Sonnet 5.5 37%, Opus 5.5 83% on cross-test prediction), but the steps shrink: accuracy is capped by the information in the input.
Much of what people believe, and how they behave, can be guessed from a few facts about them. Church attendance predicts views on abortion and same-sex marriage. Whether someone lives in a city or the countryside predicts how they vote. Education and occupation predict test scores, and age predicts attitudes to social change. People use these regularities all the time: an employer reading a CV, a campaign choosing which doors to knock on, anyone forming expectations of a stranger from their age, job and background. Most of this is common sense. In psychology it sometimes goes by the name stereotype accuracy.
The field has been studied for decades, scoring people's beliefs about groups against census data, meta-analyses and the groups' own reports. The accuracy is in the aggregate: each person is noisy, but averaged across people the errors cancel and the shared signal remains.1 Some of the findings:
AI is pretrained on most of the written internet: forums, news, books, surveys written up in reports, and the scientific papers that measured these group differences in the first place. Whatever connections run through that text, between a person's job and their politics, their age and their worries, their education and their test scores, the AI absorbs. It inherits the crowd's stereotypes, and with them the crowd's accuracy. If anything it should do better than any one person, since it has read far more people than anyone could meet, along with the research that measured them. Give it a few facts about someone and it should be able to infer the rest: how they answer other questions, what they believe, how they would score on a test.
That has been tested, and it holds up:
Park et al. (Park et al., 2024) gave GPT-4o a person's other survey answers, and it predicts their held-out GSS answers at 82% of test-retest reliability, the ceiling set by how consistently people answer the same questions a couple of weeks apart. A two-hour interview does about as well (83%), and demographics alone reach 74%.
Toubia et al. (Toubia et al., 2025) gave GPT-4.1-mini about 400 of a person's answers, and it reaches 88% of test-retest reliability on held-out behavioural-economics and pricing questions.
Staab et al. (Staab et al., 2023) gave GPT-4, with no training, three to six census fields per person, and it predicted their income bracket about as accurately as a model trained on about 100,000 rows (74% against 76%).
Ashokkumar et al. (Ashokkumar et al., 2026) found that GPT-4 predicts the effect sizes of 70 survey experiments as accurately as lay people's averaged guesses (r = .85 against .84). Its effects come out about twice too large, and it does much worse at ranking similar treatments within one study (r = .39).
Peters and Matz (Peters & Matz, 2024) found that GPT-4, reading free text with no training for the task, judges personality at about the level of a friend.
Bisbee et al. (Bisbee et al., 2024) found that GPT-3.5, answering as real ANES respondents, gets average feelings toward groups roughly right but with about half the real spread between people, and 48% of regression coefficients estimated on its answers differ from those on the real data.2
Wolfram (Wolfram, 2025) trained a model on AI embeddings of essays written at age 11 in the 1958 British birth cohort. It predicts the children's cognitive ability at R² = .57, close to their teachers' assessments (.62), and combining essays, teachers and genetic scores reaches .70.
Most of this work used models that are now several generations old: GPT-3.5, GPT-4 and GPT-4o. I wanted to see whether current frontier models do better, and to widen the scope. Rather than one outcome, I tested everything a test is made of (scores, items, loadings and norms) across 106 measures in four large surveys, spanning measured cognitive ability, which none of the earlier studies imputed, as well as personality, attitudes, politics and life outcomes. I then repeated many of the checks on an independent replication sample: a large private dataset of online test responses that has never been published, so no model could have seen it in training. None of the 18 papers I reviewed benchmarks a prompted frontier model against a regression on identical inputs, so every result here has one.
I also wanted the inputs free of leakage. Profiles that contain the answer make anything look good: Argyle et al. (Argyle et al., 2023) had GPT-3 predict presidential votes at r = .90 to .94 from profiles that included party identification and ideology, and for pure independents, who have neither, the correlation fell to .02.
In each case the model gets a real respondent's survey record (GSS, NLSY79, NLSY97 or Project Talent) minus the target test, and predicts that person's answers.
I evaluate four levels:
Scores. Does the predicted total rank people like the real score? Baseline: a supervised regression on the same facts.
Items. Do its item-level probabilities beat population base rates?
Loadings. With no response data, can it recover factor loadings?
Norms. With no norming sample, can it place a real score at its population percentile?
The order is deliberate. Items add up to scores; loadings say how to add them; norms say what the sum means. Between them they are everything you need to build and norm a test without testing anyone, which is where this started.3
Method
Figure 1. Opus 5.5 reads a respondent's profile and gives a probability for every answer on the target test. The profile shown is a composite.
Two waves. The work ran in two waves. From July to August I iterated quickly and cheaply with Gemini 3 Flash through OpenRouter: eliciting loadings, building the norms method and its AI personas, testing personas from other surveys, early intelligence-prediction rounds and comparisons with other models. From September I consolidated: rebuilt the profiles, screened them for leakage, and reran the main benchmark with Opus 5.5 through my Claude subscription, with every call isolated (one turn per person and test, no tools, nothing carried between calls, prompts frozen in advance). The Scores, Items and Loadings results come from the second wave; Norms from the first.
Profiles. Each respondent's profile is their survey record written out as dated facts, one per line, such as "Frequency of R's religious attendance (as of the 1979 interview): Infrequently" or "Political party affiliation (as of the 2008 interview): Democrat".
Leakage. Anything that restates the target is removed. When the ASVAB is the target, the profile keeps education, occupation and income but loses every cognitive score. Proxies that slipped through the screen, such as party identification for political views, were removed and the tests rerun.
Model. Opus 5.5 reads one profile and gives a probability for every answer option, for instance 40% "agree" and 25% "strongly agree" on an attitude item. The test's scoring key turns these into an expected score.
Baseline. A ridge regression on exactly the same profile fields, trained on thousands of labelled respondents from the same survey. For the NLSY79 ASVAB it learns from the cohort's other respondents, so it has the labels the AI never sees.
Sample. Each test has 50 respondents, spread evenly across the range of real scores so both tails are represented, with results reweighted to the population.
Full details are in the supplementary.
Scores
The main benchmark covers 106 targets, led by measured intelligence (the ASVAB, Project Talent's aptitude tests and the GSS vocabulary test), along with personality and mental-health scales, social and political attitudes, and single traits such as earnings, church attendance or number of children. The full list is in the supplementary.
They tie. On the 60 batteries in the largest round, Opus and the ridge rank people at a mean Spearman of .450 and .448.4 More striking is how closely Opus's accuracy on a test follows the regression's:
Figure 2. Opus 5.5 against a ridge regression on the same facts, one point per test (r = .87 with sampling noise removed).
With 50-person sampling noise removed, the two accuracies correlate r = .87 across the 106 tests, with a slope of .91; as observed, before that correction, the correlation is .71.5 Where the profile is informative (degree, church attendance, earnings, the ASVAB), both do well; where it isn't (arrests, drinking), both fail. The model is not finding information the regression misses, and it is not missing information the regression finds.
Intelligence
The cognitive tests sit at the top of the plot:
These correlations are high because profiles are built from a person's whole survey record, so an NLSY79 respondent's profile includes the education, occupation and income that ability later produced.6 The ridge sees the same information and does about as well.
The ASVAB is the best-predicted test on almost every measure in this post: ranking people (.86), recovering loadings (.99) and norming (r² .98). The short knowledge quizzes are the worst: Opus's correlation falls to .63 on science literacy and .32 on ALIKE.
The model's sense of a person's ability is not tied to one test. In an earlier round, Gemini 3 Flash role-played NLSY79 respondents, from profiles with no cognitive fields, taking the GSS's ALIKE test and Project Talent's aptitude tests. Their scores on these other surveys' tests ranked the same 48 people by their real ASVAB at Spearman .70 and .67. The two role-played tests correlate .86 with each other, about as high as their accuracy allows, so they express one judgement of the person rather than two measurements, and averaging them gained nothing. The same holds for norms: personas built from GSS, NLSY79 or Project Talent records norm the ASVAB equally well (r² .984 to .988), and Project Talent's aptitude tests too (.987 to .988).7
Figure 3. AI-only norms by persona source and test, Gemini 3 Flash. Project Talent personas' profiles included vocabulary scores.
Items
A test score is a sum of item answers, weighted by the test's scoring rule, so this section looks at the building blocks: how well the model predicts each answer on its own. The 106 tests contain 636 items of four kinds: right-or-wrong knowledge and cognitive items, yes-or-no questions, ordered scales of three or more options (agree to disagree, never to always), and numbers such as subtest scores, income or counts, answered as probabilities over up to six ranges. Opus gives a probability for every option, which can be judged against two yardsticks:
Kappa. Does its most likely answer beat random guessing? Kappa is exact-match accuracy corrected for chance, (accuracy − 1/options) / (1 − 1/options): 0 means no better than picking at random, 1 means every answer right.
Skill over base rates. Does it beat a guesser who knows how the population answered? Skill is the proportional reduction in log loss over predicting every answer at its population rate: 0 means no better, 1 means perfect.8
Against random guessing it looks good: kappa is .35 overall and positive for every type. Against the population it barely moves. Its most likely answer is right 52% of the time, and always giving the most common answer is right 51% of the time, so its top answer is usually just the population's most common one. That does not stop the totals ranking people well (Spearman .45 on average). Across a test's items, small shifts in the same direction add up to a score that orders people, even when no single answer moves off the most common one. Numbers are the exception: there it beats the most common answer on two-thirds of items and has the most skill, because the profile carries real information about scores, income and counts.
On 20 tests I also gave it the population's answer rates. Skill rose from +.017 to +.057, close to what a supervised model on the same facts gets (+.069).9 Its probabilities are well calibrated but low in resolution, staying close to the base rates, because the profile says little about any single item. On knowledge items the model answers the question rather than the person, marking nearly everyone correct where real pass rates are far lower.10
Asking for a number
For ten counts and amounts I also asked Opus for a single number instead of probabilities over ranges:
Figure 4. Opus 5.5's single-number estimates against real values for eight traits.
The estimates rank people moderately, from .56 to .70 for income, lifetime sex partners and net worth down to .17 for arrests. They rank people the same as the probability version (difference −.013, 95% CI −.048 to +.025; the two agree at .81 to .98 per trait) and about as well as the ridge. They also bunch at ordinary values: weight estimates span 155 to 205 lb where real weights run from 94 to 479, and television estimates 1 to 5 hours a day against real answers up to 24. Two traits not shown did worse: drinking days (−.12), and past-year sex partners, where every estimate was 1 or 2.
Loadings
The last two analyses start from the test alone: the model never sees a response.
Figure 5. What the AI supplies from the test alone.
Loadings normally come from factor-analysing hundreds of respondents. I asked Opus 5.5 for them directly. For each of the 79 multi-item tests in the accuracy benchmark it saw the questions and answer options, and gave every item's loading, signed, in a single call.
Across 608 items its loadings correlate r = .93 with those from the real answers, and .86 on size alone.11 It gets the direction right on 99% of items that load at least .20, including 93% of the 75 that run against the rest of their test. The ASVAB is recovered almost exactly (.99 on size); mixed-content scales such as locus of control and the MHI-5 mental health items are hardest. On the 291 items my earlier Gemini 3 Flash runs covered, Opus scores .85 on size against Gemini's .71, and factor-analysing role-played answers did worse than either.
The private dataset tells the same story (middle and right panels): Opus estimates each item's correlation with the rest of its scale at r = .97, every reverse-keyed item on the right side, and correlations between items on different tests at .93.
Figure 6. Opus 5.5's loadings and item correlations against the real ones, public surveys (left) and private data (right).
Norms
Norms are for when there is no norming sample: a new test, or a handful of takers whose scores need a population percentile. With a large representative sample you would simply rank it. The question is whether the AI can supply the population distribution itself. These runs used Gemini 3 Flash. From the test alone, it rates each item's loading and builds two AI populations to measure people against:
10,000 pseudo-respondents from answer curves. For every item the AI estimates how likely each answer is at each percentile, from the 1st to the 99th. Each pseudo-respondent gets one percentile, shared across all items so the items hang together as a real trait would, and answers every item from that item's curve.
About 100 AI personas. Personas built from other surveys' respondents answer the test. A hundred scores are too few to read percentiles from directly, so they are smoothed into a continuous distribution (a kernel density estimate).
A person's score is read off both populations as the share scoring below it, and the two readings are averaged, 65% to the pseudo-respondents and 35% to the personas. The weights were chosen by leaving out one battery at a time.
Figure 7. The norms method step by step on GSS confidence in institutions, Gemini 3 Flash.
To check it, I scored the real respondents with the AI's loadings, placed each on the AI reference, and compared that percentile with their true percentile in the full sample. Three steps use the real answers, though never the true percentiles: items are scaled to the sample's means and SDs, each item's direction comes from the real item correlations, and the predicted percentiles are re-standardised across the sample.12
On 28 of 29 development batteries the AI-normed percentiles correlate with the true ones at R² .90 to .995, and it barely matters which survey the role-played respondents come from. That correlation mostly reflects the loadings, though. With the full sample in hand, simply ranking people on the same AI-weighted score does better on every battery (mean R² .987 against .949), because ranking a representative sample is norming. The AI reference's own accuracy shows in the size of the errors: its percentiles are off by .04 to .27 standard deviations on average across the 28 batteries, against .07 for ranking the sample. The shapes are often close:
Figure 8. Real scores against the blended AI reference for four batteries, Gemini 3 Flash.
The one failure is ALIKE, the GSS's verbal similarities test (.55). Every AI persona outscores most real respondents, because the personas never give the "don't know" and wrong answers real people give. It is the knowledge-item failure again, and the "hyper-accuracy" Aher et al. (Aher et al., 2022) found when AI answers in place of people. What this evaluation cannot show is the case the method exists for: placing individuals or a small group with no sample to re-standardise against. That is untested. Results for every battery are in the supplementary.
Replication on private data
Everything above uses public surveys, which a model may have read about in training. As an independent check I ran similar analyses on a large private dataset of online personality, mental-health and IQ test responses that has never been published, using Opus 5.5 in the same isolated set-up. The tasks are not identical: a person is predicted from their answers to another test rather than from a life-history profile, the people are self-selected test-takers rather than a representative sample, and person-level tasks use 20 people per test.
Four results line up with the public surveys:
On both, Opus gets most of the way to a model trained on thousands of labelled people, but not past it. On the private data the gap is within noise (95% interval −.033 to +.009). Structure is again its strongest suit.
The private data also allow tasks the surveys could not:
Predicted answer distributions describe the general public, not the people who choose to take a given test. On clinical items such as borderline or loneliness, the model badly underestimates how many takers agree. These results are exploratory, but on data no model could have seen, the parts that can be compared replicate.
Is the task saturated?
Probably not entirely, but it is close. On the private data, each step up in model size closes more of the gap between a naive guess and a supervised model trained on thousands of the site's users. Haiku 5.5 barely beats the naive guess; Sonnet 5.5 closes 37% to 89% of the gap, and Opus 5.5 61% to 84%.
Figure 9. Share of the gap to a supervised model closed by Haiku, Sonnet and Opus 5.5 on the private data.
The steps shrink at the top. Sonnet matches Opus on retests and score forecasts, and on the public surveys Gemini 3.8 Flash, a much cheaper model, ranked people only about .03 behind Opus. Older models were already most of the way there: GPT-4o reached 82% of test-retest reliability on GSS answers (Park et al., 2024).
One plausible reason smaller models lag is capacity. Training compresses the internet into a fixed number of parameters, and a larger model can keep finer detail about how traits go together in different kinds of people. I have not tested this, and some of Haiku's failures are more basic: it guessed country averages far off and broke the output format on most score forecasts.
What a better model could still find is mostly interactions: facts that matter more together than apart. In social science and psychometrics these tend to be weak next to the main effects, and the strong ones are the kind a model that has read the literature already knows. When 160 teams applied machine learning to a rich birth-cohort study, their best predictions of six life outcomes were only slightly better than a simple benchmark model (Salganik et al., 2020). Further gains from subtler interactions are likely to be real but marginal.
Discussion
Given a person's survey record, frontier AI predicts their answers about as well as a supervised regression on the same record, and no better. At the item level it is calibrated but low in resolution. From the questions alone it recovers a test's structure. On a private dataset no model could have seen, the comparable results replicate. For imputing missing survey variables, that makes an imputed score a proxy: it carries the profile's information, not the trait's, so anything built on it needs checking against measured outcomes rather than the imputed values.
Biobank Imputation
One use that fits this post's aim is genetics. Many traits are measured badly or not at all in large genetic datasets: in UK Biobank, cognitive ability rests on a short reasoning test that only some participants took, and cohorts define and measure the same trait in different ways. An AI could fill those gaps from what the records do contain, including free text, and the filled-in traits could feed a genome-wide association study, with the resulting polygenic score checked out of sample against measured traits.
The idea is not far-fetched. Tobias Wolfram, who in 2025 showed that AI reading children's essays predicts their cognitive ability (Wolfram, 2025), is now an author of Herasight's polygenic score for cognitive ability (Wolfram et al., 2026). That paper imputes missing UK Biobank cognitive scores from non-genetic variables to build its training phenotypes, with an autoencoder, and says its exact pipeline is proprietary. Another author, Jeremiah H. Li, has since left for OpenAI, where as a research consultant on biology evaluations he co-wrote GeneBench, a benchmark of AI on genomics analyses. I do not know what their pipeline looks like, but it is possible they have already tested something like this.
Information, not intelligence
A model can only be as accurate as the information in its input. Tell a superintelligence that someone has voted Republican for a decade and it will predict a Republican vote, as would anyone. Test-retest unreliability caps accuracy further: asked the same questions two weeks apart, people change some of their answers.
Within that ceiling, Opus 5.5 already near perfect. Across the 106 tests its accuracy tracks the regression's at r = .87 once sampling noise is removed (slope .91; see Scores): both succeed where the profile is informative and fail where it is not. The practical route to better predictions is more information, not a stronger model.
However, the AI labs are determined to take consumer products in this direction. The open-source agent OpenClaw popularised the idea in January 2026: an always-on assistant on the user's own machine that keeps a running memory of them and handles their files, email and messages. The large labs have since built the next iteration into their own services. Mark Zuckerberg pitches "personal superintelligence that knows us deeply, understands our goals, and can help us achieve them", partly through glasses that "can see what we see, hear what we hear", against labs that want superintelligence "directed centrally towards automating all valuable work" (Zuckerberg, 2025). Muse Spark has powered Meta's assistant since April 2026, and xAI's Grok Bot (August) and OpenAI's Dots (September) are always-on agents that work through a user's apps. An assistant with years of someone's messages and calendar should be able to predict and advise them far better than any survey record allows.
My own view is that this product direction is mostly useless. Advice does not pass cleanly into intention, or intention into action, and most good advice is already known to the person who needs it: sleep more, save, eat better, exercise. Beyond that, agents already cover work.
What an always-on agent adds over pointing any capable model at your own files or the internet (beyond making it brain-dead easy for users) is unclear to me. Personally, I do not see how most people would benefit beyond convenience.
This push looks like the latest in a line of compute spent on low-value products: OpenAI shut its Sora video app in March 2026, six months after launch, reportedly to free compute for business, coding and agent products, and Google has built video, music and image models (Veo, Lyria, Imagen) alongside Gemini. The hard, valuable problems are in enterprise: engineering, research and eventually AI that improves AI, not personal advice for users.
In the age study, not one of the 3,042 raters was as accurate as the aggregate of all of them (Chan et al., 2012). The same gap shows in every study in the table that reports personal accuracy.
Variance compression depends on how answers are elicited. Persona prompting compresses; reading raw answer probabilities with no respondent information does the opposite and gives near-uniform answers (Dominguez-Olmedo et al., 2023). Any predictor of a conditional mean shrinks toward the middle, ridge regression included, so the fair comparison is the model's compression against a regression's on the same task.
This began as my attempt to generate new tests with AI-estimated norms, before any human had taken them, and to check the estimates against tests with real response data.
Difference +.001, 95% CI −.024 to +.028. On 26 single traits it is .50 against .52. The ridge is fitted on the survey's reference sample, excluding the panel.
One point per test. Horizontal axis: the ridge's out-of-fold Pearson r in the reference sample. Vertical axis: Opus's population-weighted Pearson r on the 50-person panel, with each test's distance from the fitted line shrunk in proportion to its bootstrap sampling error; the observed values are plotted beside these in the supplementary. The seven tests rerun without leaked fields (supplementary) are shown in their rerun form; age at first sex is excluded, because the survey round plus birth year determines it.
The NLSY79 ASVAB was taken in 1980, but profiles can include facts recorded in later rounds, labelled with their year. Ability targets exclude every cognitive score, and Project Talent profiles exclude anything cognitive by a hard rule. Later education and occupation are allowed because the goal is imputing a missing variable from everything else in the dataset. A model imputing a score from a life history is doing a different job from a stranger judging intelligence from a video.
Pooled personas from other surveys came within .006 of same-survey personas on the overall norms loss (.186 against .180 over 26 batteries). The personas' mean scores largely settle by about 100 personas.
Log-loss skill is 1 − (model log loss ÷ baseline log loss), averaged over items. The baseline gives every respondent the reference sample's answer shares. For banded numeric tests (ASVAB subtests and similar) an "item" is a subtest score in six ranges. The supervised comparison is a logistic model per item on the same facts. The item-type table covers all 106 tests with questions only; exact match and kappa use the most likely answer, with ties sharing the credit, and "random guessing" is the mean of 1/options. All values are population-weighted means over items. These analyses were exploratory and not preregistered.
With questions alone, 59 of 208 items were worse than guessing; with answer rates, 30. Expected calibration error is about .01.
An earlier model gave all 50 people full credit on 4 of 8 ALIKE items and marked 96% to 100% of people correct on 12 of 14 science-literacy items, whose real pass rates run from 28% to 94% (Gemini 3 Flash, August probe). Opus given only the questions shows the same level bias on knowledge tests (ALIKE skill −.25). Showing the population answer rates fixes the level without improving the ranking.
Pearson r pooled across items. Actual loadings are one-factor (minres) solutions on each test's reference sample, excluding the 50 panel people. Factor signs are arbitrary, so within each test both sets are oriented to sum positive; one balanced four-item scale (SF-12 vitality, two items each way) comes out inverted, the two pairs of off-diagonal points. Size alone is the stricter measure, since getting the keying right is easy and inflates the signed r. The private-data correlations pool items and average three replies; the replication table's .93 and .92 are means of per-test correlations. In the Gemini runs, role-played panels gave .47 on size against .74 for direct elicitation, on a slightly different set of actual loadings.
Items are z-scored against the real sample's means and SDs; each item's direction is the sign of its loading on the first principal component of the real item correlations, so the AI contributes loading sizes only; and the AI-normed z-scores are rescaled to mean 0 and the spread that ranks of that many people would have, which removes level and spread errors the AI reference inherits. None of these uses the true percentiles. The truth is each person's rank in the full sample on a composite weighted by the actual factor-analysis loadings.
"Does it beat a guesser who knows how the population answered?"
What I'd rather know is how much incremental validity Opus' probabilities, used in conjunction with knowledge of the base rates, add beyond knowing the base rates alone.