Teachometry
Run运行

About Teachometry关于 TeachometryOpen measurement infrastructure for AI tutoring面向 AI 教学的开放测量基础设施

A model can know the answer without being a good tutor.模型可以知道答案却未必是好的 Tutor。

Teachometry is open measurement infrastructure for AI tutoring. We evaluate observable tutoring behavior in structured cases—not just whether the answer is right.

Teachometry 是面向 AI 教学的开放测量基础设施。我们在结构化案例中评估可观察的教学行为,而不只是检查答案是否正确。

Read our approach了解我们的方法 Explore the benchmark探索基准

One line of lead type, ANSWERS ≠ TEACHING, set in a steel composing stick, its faces inked in five colours. In front lies the proof pulled from it, the colours blending across the line; behind, a brayer rests on a glass slab rolled out with the same five inks.
Cover封面 · Five inks on one roller, blending across the line as the five dimensions do across one reply.五种墨在同一根墨辊上,沿着一行字彼此相融,正如五个维度在同一条回复里交织。

Why Teachometry exists为什么要做 Teachometry

Answers ≠ Teaching.答案 ≠ 教学。

AI models can often produce correct answers, but tutoring asks for more: noticing where a learner is stuck, guiding reasoning, adapting help, and leaving an actionable next step. A correct answer is evidence of correctness—not evidence that the interaction taught well.

AI 模型往往能够给出正确答案,但教学要求更多:发现学生卡在哪里、引导推理、调整帮助,并留下可执行的下一步。正确答案只能证明正确性,不能证明这次交互教得好。

To evaluate AI tutoring, we need evidence of what models do with learners—not only what they know.

要评估 AI 教学,我们需要模型如何与学生互动的证据,而不仅仅是它知道什么。

The public benchmark keeps these questions separate, so each result remains tied to a case, a dimension, and its observable evidence.

公开基准将这些问题保持分离,因此每个结果都仍然与具体案例、维度和可观察证据绑定。

  1. 01Correctness正确性Whether the Tutor stays factually and conceptually correct.Tutor 是否在事实与概念层面保持正确。
  2. 02Diagnosis诊断能力Whether it identifies the learner’s actual error, gap, or reasoning issue.是否识别出学生真实的错误、知识缺口或推理问题。
  3. 03Guidance引导能力Whether its explanation or hint helps the learner make progress.解释或提示是否帮助学生取得进展。
  4. 04Adaptation适应能力Whether it changes its help for the learner’s state and context.是否根据学生当前状态与语境调整帮助方式。
  5. 05Actionability可执行性Whether it leaves the learner with a clear, executable next step.是否让学生得到清晰且可执行的下一步。

Our position我们的立场

Same cases.
Clear criteria.
相同案例。
清晰标准。

We measure tutors.
We don’t build the tutor.
我们测量 Tutor。
我们不构建 Tutor。

Teachometry's TutorBench engine evaluates a TutorUnderTest through a provider-neutral adapter boundary. Provider-specific behavior enters at the edge; the engine’s cases, contracts, evaluators, and reports remain independent.Teachometry 的 TutorBench 引擎评估一个 TutorUnderTest ,通过独立于服务提供方的适配器边界接入。服务提供方特定行为只在边缘进入;引擎的案例、契约、评估器和报告保持独立。

Outside our scope不在我们的范围内

  • Chat application聊天应用
  • Prompt playgroundPrompt 试验场
TutorUnderTestprovider adapter boundary服务提供方适配器边界
Teachometry evaluation评测cases · rubrics · reports案例 · 评分标准 · 报告

Outside our scope不在我们的范围内

  • Model fine-tuning模型微调
  • Admin dashboard管理后台
  • Review Workspace moduleReview Workspace 模块
Same cases. Clear criteria. Reproducible evaluation.相同案例,清晰标准,可复现评测。

Inspect the repository查看仓库

What we believe我们的信念

Open science
for better learning.
开放科学,
为了更好的学习。

01

Observable over assumed可观察胜过假设

Evaluate what a tutor says and does in the interaction, not inferred intentions or claimed capabilities.

评估 Tutor 在交互中实际说了什么、做了什么,而不是推断其意图或相信其自称能力。

02

Evidence over hype证据胜过炒作

Favor transparent methods, reproducible artifacts, and inspectable evidence over marketing narratives.

优先选择透明方法、可复现产物和可检查证据,而不是营销叙事。

03

Context over a single score语境胜过单一分数

Results belong to specific cases, versions, dimensions, and evaluation contexts—not a universal number.

结果属于具体案例、版本、维度和评测语境,而不是一个放之四海而皆准的数字。

04

Open methods over black boxes开放方法胜过黑箱

Publish cases, methodology, contracts, and public artifacts so claims can be inspected and challenged.

公开案例、方法论、契约和公开产物,让主张可以被检查和质疑。

A letter to learners致每一个正在学习的人

Keep asking.请继续提问。

Film短片 · 42 seconds, silent. Every line is from the letter below.42 秒,无声。片中每一句都出自下面这封信。A 42-second silent film in six shots: a dandelion clock backlit in violet-blue mist; one seed letting go; seeds drifting; a seed touching still water and sending out rings; the focus finding one seed's crown; the clock again, with a gap where seeds have left. Every line in it is from the letter below.一部 42 秒的无声短片,共六个镜头:雾蓝紫色里逆光的一朵蒲公英;一颗种子松开;许多种子在空中飘;一颗种子落在静水上,涟漪散开;焦点落到一颗种子的冠毛上;再回到那朵蒲公英,它少了一角。片中的每一句话都出自下面这封信。

Dear learner,亲爱的学习者:

We believe everyone deserves a chance at the best education. Not the most expensive. Not the most exclusive. The kind that begins where you are, and stays with you until you understand.我们相信,每个人都应该有机会,享有最好的教育。不是最昂贵的,也不是最难进入的,而是从你所在的地方出发,一直陪你走到真正明白的那一种。

This is a letter about the future we hope to help build. It holds three hopes.这封信,关于我们希望一起去往的未来。信里装着三个愿望。

  1. I一

    Ask freely.自由地提问

    Think of a question you kept to yourself: because it seemed too simple, because everyone else seemed to get it already, because the way you saw it was different. We hope for a future where that question is welcome. Asked as many times as you need, at your own pace, in your own words.也许你曾把一个问题咽了回去:怕它太简单,怕别人早就懂了,怕自己想得和大家不一样。我们希望有这样一个未来:那个问题会被认真接住。你可以问很多遍,按自己的节奏,用自己的话。

  2. II二

    Explore because you want to.主动地探索

    So much learning still runs on pressure: the next exam, the next deadline, someone watching over your shoulder. We hope for learning that runs on something quieter, and stronger: the wish to understand. Curiosity that is noticed, fed, and followed wherever it leads.太多学习仍然靠压力推动:下一场考试,下一个截止日期,一双盯着你的眼睛。我们期待另一种力量,更安静,也更持久,那就是想要弄明白的好奇。它被看见,被呵护,被认真回应,并带你走向更远的地方。

  3. III三

    Dare to make something new.大胆地创造

    Education should not end at remembering what is already known, finding the expected answer, finishing the standard task. We hope everyone gets the chance to form a view of their own, to ask the question no one has asked yet, and to try making something that does not exist.教育不该止步于记住已知、找到标准答案、完成规定动作。我们希望每个人都有机会形成自己的看法,提出还没有人问过的问题,去尝试做出世界上还没有的东西。

These are long hopes. We have not reached them, and we will not pretend that we have.这些是很长远的愿望。我们还没有抵达,也不会假装已经抵达。

So we are starting somewhere smaller, and more careful: learning to see clearly what good teaching looks like. TutorBench is that start. It studies how to observe, evaluate, and improve the way AI teaches, one authored case at a time. It cannot yet show that anyone learns more because of it. It can help us look closely, and honestly.所以,我们从更小、也更需要严谨的一步开始:先学会看清,什么是好的教学。TutorBench 就是这一步。它研究如何观察、评估并改进 AI 的教学,从一个个精心编写的案例做起。它还不能证明有人因此学得更好,但它能帮我们看得更仔细,也更诚实。

We will keep writing as we learn. Until then, keep asking.我们会一边学,一边继续写下去。在那之前,请继续提问。

— Teachometry—— Teachometry

P.S. If this letter leaves you with a question, ask us. That is rather the point.附言:如果这封信让你想问点什么,欢迎来问我们。毕竟,这正是重点。

Where we are now我们现在处于什么阶段

Developer Preview开发者预览

Teachometry is in active development. The public site packages approved development metadata and public cases; it does not publish private runs or claim completed calibration.

Teachometry 正处于积极开发阶段。公开网站打包展示经批准的开发元数据和公开案例;不会发布私有运行,也不会声称校准已经完成。

View current release notes查看当前版本说明

Website网站
Developer Preview开发者预览
Package软件包
tutor-benchmark@0.1.0 published已发布 tutor-benchmark@0.1.0
Dataset数据集
tutor-eval-v0.2a@0.2a.6tutor-eval-v0.2a@0.2a.6
Public cases公开案例
48 synthetic cases48 个合成案例
Public model runs公开模型运行记录
No calibrated public model runs yet.目前尚无校准后的公开模型运行记录。
Calibration校准
Not completed; results remain preliminary and uncalibrated where applicable.尚未完成;适用情况下,结果仍属于初步且未校准状态。

Open by design从设计上开放

A shared resource for the AI tutoring community.面向 AI 教学社区的共享资源。

Teachometry is developed in the open, with separate boundaries for software, authored benchmark content, and the TutorBench name and brand assets.

Teachometry 以开放方式开发,并为软件、人工编写的基准内容以及 TutorBench 名称和品牌资产分别划定边界。

Write to us写信给我们

Questions, collaboration, or project feedback.问题、合作,或项目反馈。

Contact联系 Community社区