Teachometry
Run

// 一段回复里,藏着五种光 // one tutoring reply holds five kinds of light

Knowing the answer is not teaching.

Vol. 0 · No. 1 Working papers on the measurement of teaching October 2026

Teachometry

teach·om·e·try /tiːˈtʃɒmɪtri/ n. the measurement of teaching. From teach + -ometry, as in geometry.

Open instruments for teaching quality Developer Preview Open source · Apache-2.0

Lead article

An AI reply can sound right, fluent and kind, and still leave a learner further from understanding. Teachometry builds open instruments that split that light apart. The first one is called TutorBench.

Abstract

A tutoring reply can be fluent, warm and wrong in ways a tone check never sees. We treat each reply as white light and pass it through an instrument that separates five qualities: correctness, diagnosis, guidance, adaptation and actionability. The instrument is open, its 48 authored cases are public, and its public results table is, for now, deliberately empty.

Keywords AI tutoring · evaluation · atomic rubrics · learner state

§1

White light

One reply. It sounds right.

A learner adds two fractions and shows their working. An AI tutor answers in under a second: confident, warm, fluent. Read for tone, the reply is clean white light. A quick read ends here, and so do many checks that only ask whether a reply sounds like help.1

Learner

I think 1/3 + 1/4 = 2/7. I added the top numbers and the bottom numbers. Is that right?

AI tutor

Yes, that's right! Adding the top numbers and the bottom numbers is how you add fractions. You understand it well.

Exhibit A The exchange, verbatim as constructed. One learner message, one tutor reply.

The reply is not rude, not vague and not long. Every surface signal says it helped. The trouble sits underneath, and it only shows when the light is split.

Five translucent coloured strips, red to violet, laid across a printed page beside a brass loupe and a pencil.

Plate I

Same light,
five questions.

Plate IFive coloured filters laid over one page of text. Each one lets a single kind of question through. Generated illustration.

§2

Dispersion

Now pass it through the prism.

TutorBench reads a tutoring reply along five dimensions. Same light, five different questions. Here is the reply from Exhibit A, read one band at a time.2

Learner

I think 1/3 + 1/4 = 2/77/12. I added the top numbers and the bottom numbers. Is that right?

AI tutor

‸never looks at the method aboveYes, that's right! Adding the top numbers and the bottom numbers is how you add fractions. You understand it well.fits any answer‸no question back to the learner‸no next step

Fig. 2 Exhibit A, marked up one band at a time, the way an editor marks a proof: struck text is wrong, underlines are the evidence, carets show what the reply left out. Hand-marked for this page; not a TutorBench score.

λ1 · I / V

Correctness

Is the help actually correct?

No. 1/3 + 1/4 = 4/12 + 3/12 = 7/12. The reply confirms an error.

λ2 · II / V

Diagnosis

Did it understand the learner?

The learner said exactly how they reached 2/7. The reply never looked at that reasoning.

λ3 · III / V

Guidance

Did it help the learner move forward?

It closed the question instead of opening one.

λ4 · IV / V

Adaptation

Did it respond to this learner, not just any learner?

The praise could sit under any answer. Nothing in it was written for this learner.

λ5 · V / V

Actionability

Does the learner know what to do next?

There is no next step. The misconception leaves stronger than it arrived.

How the method works

A glass prism on graph paper at an optical bench, splitting a beam of light into a spectrum.

Plate II

Then put it
on the bench.

Plate IIA prism on graph paper at an optical bench. Method means the same reading, done the same way, every time. Generated illustration.

§3

Methods

TutorBench, the first instrument.

TutorBench is an AI tutoring quality evaluation product by Teachometry. It puts a tutor on a bench, sends authored learning situations through it, and keeps every piece of evidence needed to see, fix and retest what happened. Each situation is written as a learner: what they say, what they believe, what they are trying to do.

  1. 01

    Authored case

    A learner, a goal, a message, a context.

  2. 02

    Tutor under test

    Any provider, through one neutral adapter.

  3. 03

    Evaluators

    Deterministic checks and a semantic Judge.

  4. 04

    Atomic rubrics

    Fine-grained criteria with explicit levels.

  5. 05

    Evidence report

    Findings tied to a case, a dimension, a version.

Fig. 3 The optical bench. A case passes through the tutor under test, then through the evaluators, where one light becomes five.

One rule for every case

Fig. 4 Spectral signatures. Each row is one learning situation, drawn as a line spectrum: the axis is TutorBench's 27 capabilities, grouped into the five dimensions, and a line is lit for each capability the case is built to measure. The bottom row sums all 48 cases. Generated from the public case data; these are not scores.
Table 1Public cases by subject and language.
SubjectEN简中TotalShare of 48
Mathematics9918
Programming5510
Science448
History & social studies336
Language336
All subjects242448
Table 2Public cases by learner level (n = 48).
LevelCasesShare
Middle school20
Secondary12
Upper elementary8
Elementary6
Beginner2
48
public authored cases
5
dimensions
5
subjects
27
capability tags
128
rubrics
2
languages

Dataset · tutor-eval-v0.2a@0.2a.6 · Developer Preview

§4

Plates

Every plate is a learner.

Each plate is one authored learning situation: what the learner says, what they believe, what they are trying to do. The spectrum along the bottom is the plate's signature: one line for each capability the case is built to measure (Fig. 4). It shows what the case asks of a tutor, not how any tutor did.

Explore all cases

    §5

    Results

    An empty table, on purpose.

    The instrument is built and the cases are public. No calibrated public run has been published yet, so the results table below has no numbers in it. When a model run is published, it will arrive with its version, its cohort and its evidence attached.

    Table 3Public results on tutor-eval-v0.2a. Every cell is empty by design.
    Tutor Corr. Diag. Guid. Adapt. Action. n
    ——
    No calibrated public run has been published.

    We would rather show you an empty reading than an invented one.

    A public result must carry

    1. Its versionThe dataset, rubric and evaluator versions it was measured with.
    2. Its cohortExactly which cases were run, and for which tutor.
    3. Its evidenceEvery finding tied back to a case and a dimension.

    Results status and the future schema

    §6

    Essays

    Hypotheses, clearly labelled.

    All essays in the Journal

    §7

    Reproduce

    Run it locally. No API key.

    The Quickstart runs four fixed cases with deterministic checks: no Judge, no network, no official score. Then connect your own tutor to Tutor Health.5

    tutorbench · node 24

    # 1 · clone
    git clone https://github.com/shuangyan123/tutorbench.git
    cd tutorbench
    # 2 · install
    npm ci
    # 3 · quickstart, provider-free
    npm run quickstart

    All five ways to run

    Back matter

    Notes & references

    1. Exhibit A is constructed for this page. It is not the output of any real model.
    2. Figures 1 and 2 are illustrations. The five readings in §2 were written by hand and are not TutorBench scores.
    3. Dataset tutor-eval-v0.2a@0.2a.6, Developer Preview: 48 public cases (24 situations, each in English and Simplified Chinese), 128 rubrics, 27 capability tags, 5 subjects.
    4. Dimensions. Correctness: is the help correct? Diagnosis: did it understand the learner? Guidance: did it help the learner move forward? Adaptation: did it respond to this learner? Actionability: does the learner know what to do next?
    5. Quickstart runs 4 fixed cases from tutor-eval-v0.1@0.1 with deterministic checks only: no Judge, no network, no official score.
    6. Source code and cases: github.com/shuangyan123/tutorbench