Worth Hiring an AI Tutor? StudentBench Finds GRE Learning Gains Match Human Tutoring

In September 2026, Handshake AI Research released the preprint StudentBench: a comparison of AI tutoring, expert human tutoring, and a no-tutoring control on GRE Quantitative and Verbal. The analysis covers about 2,383 adult participants across roughly 2,469 valid sessions, with more than 175,000 student–AI messages. The headline result: pooled AI tutoring is statistically equivalent to expert human tutoring for GRE learning gains (p=.015). Among individual tutors, Gemma 4 31B passed equivalence versus humans (p=.044) at about 918× lower cost per percentage point of learning gain under the paper’s cost definitions. The public platform is at studentbench.org.

Questions about whether an “AI tutor is worth hiring” often stall at anecdotes or single-product demos. StudentBench reframes the question as a checkable experiment: the same pre-/post-tests, the same one-hour teaching window, human and no-tutoring controls, and equivalence testing (TOST) rather than a simple “who scored higher” race.

How the study was designed

Each student first took a GRE-timed pre-test (Quantitative: 27 questions / 47 minutes; Verbal: 27 questions / 41 minutes), spent about five minutes reviewing mistakes, then was randomly assigned to one of three conditions:

1. AI tutoring — one of the LLM tutors handled lesson planning, practice-problem generation, and one hour of text chat;
2. Expert human tutoring — one-to-one live video;
3. No-tutoring control — educational videos unrelated to the GRE.

Students then took a post-test matched in length and concepts but with different items. Pre- and post-test forms were randomly swapped so form difficulty would not be mistaken for learning. Items were newly written by former ETS/Kaplan GRE creators; old public GRE items were avoided to reduce training-data contamination. AI tutors never saw the post-test.

Learning gain is post-test percent correct minus pre-test percent correct. Comparisons adjust for pre-test score and related covariates (ANCOVA-style). AI–human equivalence uses two one-sided tests (TOST) with primary bounds of ±0.25 pooled SD of learning gains (about ±4.09 percentage points for the combined GRE comparison), with a stricter ±0.20 SD bound also reported.

A second study asked 51 expert tutors to complete 2,028 pairwise rubric evaluations of AI-generated lesson plans and embedded practice problems—separating lesson planning, practice design, and conversational pedagogy.

Key results: pooled equivalence is not “every model is equal”

Pooled across AI tutors, combined Quantitative and Verbal learning gains were statistically equivalent to expert human tutoring (p=.015); the same comparison also met the stricter ±0.20 SD bounds (p=.023). The adjusted AI−human difference was about −0.58 percentage points (90% CI [−2.18, 1.03]).

Relative to the no-tutoring control, adjusted AI gains were approximately:

• Quantitative: +6.86 percentage points;
• Verbal: +5.47 percentage points;
• Combined: +6.15 percentage points (about 1.5–2 more correct answers out of 27).

Boundaries that must stay visible:

• Equivalence is a pooled result, not a claim that every model matches humans. Across seven GRE domains, the best AI tutor’s mean gain exceeded the human mean in five; models still differ, and individual equivalence tests are separate.
• Equivalence at ±0.25 SD also held for Quantitative; Verbal pooled did not establish equivalence the same way. On whole-section means, human tutoring retained the highest Verbal mean, while several AI tutors led human means in Quantitative domains.
• Gains are immediate pre-to-post measures—not long-term retention months later.

Cost, latency, and the engagement chain

The paper standardizes “USD per percentage point of learning gain” as mean session cost divided by mean learning gain. The human reference is a published $75/hour market rate; AI costs are inference for planning, practice, and the tutoring hour (about 97% of API calls reconstructed). Under that definition, Gemma 4 31B achieved GRE learning gains equivalent to expert human tutoring (p=.044) at about 918× lower cost per percentage point (paper example: roughly $0.0052 versus $4.81 per pp; mean inference about $0.067 for the full session). Five other AI tutors also passed individual equivalence tests. Session inference costs across the set ranged from about $0.067 to over $20.

On the Quantitative path, exploratory analysis traced a chain (all p<.002): faster AI replies ↔ more student messages ↔ more correct first-attempt practice ↔ larger learning gains. The same associations were not statistically significant in Verbal—Verbal students tended to send fewer, longer messages, so message count is not a interchangeable engagement metric across sections.

Expert pairwise reviews separately preferred some model families (notably Anthropic) for lesson plans and practice design; teaching-capability leaderboards do not map one-to-one onto learning gains—latency and practice volume may help explain why models that are not top-ranked on “teaching taste” can still reach human-level gains.

Limits the paper keeps—and this article keeps

1. No long-term retention follow-up — immediate gains do not automatically persist for months.
2. No “practice problems alone” control — the design cannot isolate conversational tutoring over unaided practice.
3. Population and setting — English-literate adults recruited via Handshake for a paid study; not multi-language, multi-device, or classroom deployment.
4. Cost multiples depend on definitions — AI inference versus a $75/hour human reference; other market rates or platform overhead would change the multiple.
5. Leaderboards score expert preference and dialogue behaviors, not learning gains directly.

The protocol was IRB-approved (BRANY). Data and code are open-sourced; the platform is free to use.

What this means for “should I hire an AI tutor?”

Within StudentBench’s setting—one hour, GRE Quant/Verbal, minimal software scaffolding, human experts and a no-tutoring control—pooled AI tutoring produced statistically equivalent immediate learning gains to humans, and some lightweight/open models open a large gap on unit cost. That is a checkable answer to Bloom’s two-sigma problem in its affordability form: if individualized tutoring is less limited by what a family can pay, the opportunity structure shifts.

It does not imply wholesale replacement of human tutors, nor that any chatbot teaches equally well. A steadier reading is three sentences:

1. Capability has crossed a checkable equivalence threshold (pooled; some models individually), especially on Quantitative;
2. Verbal means and long-term retention remain places where human strength and open questions coexist;
3. Latency, practice, and engagement may matter as much as how “good” a lesson plan looks to experts—benchmarks that ignore how many problems students get right will miss the gain mechanism.

StudentBench pulls “can AI teach?” from slogan back to the experiment desk: numbers public, limits stated, platform open. Whether an AI tutor is worth hiring is no longer a vibe—it is a conditional: for which section, under which cost definition, and whether immediate gains are the metric you accept for now.

Sources

• Curtis Northcutt, Inaara Hasmani, Kevin Feng, Trevor Khangi, Andreas Plesner, Jonas Mueller (Handshake AI Research): StudentBench: AI and human tutoring yield equivalent GRE learning gains (arXiv:2609.28470, September 2026)
• StudentBench platform: https://studentbench.org