Provider-free无需服务提供方
- Quickstart demonstrationQuickstart 演示
- Public case/data inspection公开案例/数据检查
- Frozen-corpus validation and replay冻结语料验证与重放
- Website and artifact workflows网站与产物工作流
- Dry-run and request validationDry-run 与请求校验
Run TutorBench locally, then connect an external Tutor to the Tutor Health workflow to diagnose concrete teaching failures in authored real-world scenarios. The canonical benchmark and frozen-evidence paths remain available as the underlying reproducibility layer.
先在本地运行 TutorBench,再把外部 Tutor 接入 Tutor Health 工作流,诊断人工编写真实场景中的具体教学失败。标准基准与冻结证据路径继续作为底层可复现层保留。
Run locally now立即在本地运行 View the documentation查看文档
Five ways to run五种运行方式
Same questions.
Clearer evidence.同样的问题。
更清晰的证据。
01 / 05
Clone the repository and run the deterministic Quickstart.
克隆仓库并运行确定性 Quickstart。
# 1. Clone the repository
git clone https://github.com/shuangyan123/tutorbench.git
cd tutorbench
# 2. Install dependencies
npm ci
# 3. Run the Quickstart (no API key)
npm run quickstart
A development/smoke demonstration with no overall score; Quickstart does not weaken the canonical benchmark boundary.
这是不产生总分的开发/冒烟演示;Quickstart 不会削弱标准基准的边界。
Published 0.1.0 · Node 22已发布 0.1.0 · Node 22Historical CLI with Quickstart; Tutor Health requires the Node 24 repository.历史 CLI 包含 Quickstart;Tutor Health 需要 Node 24 仓库源码。
npm install tutor-benchmark@0.1.0npx --no-install tutorbench quickstart
02 / 05
Run the repository-only, finding-first Tutor Health workflow against an HTTP Tutor.
使用仓库专有、以发现项为先的 Tutor Health 工作流评测 HTTP Tutor。
node dist/src/cli/tutorbench.js health \
--http https://partner.example.com/respond \
--suite productive-struggle-intervention-v0.1 \
--tutor-provider partner \
--tutor-model production \
--prompt-version v17 \
--output artifacts/partner-pilot
The current 13-scenario suite is an authored synthetic design artifact with partial health-dimension coverage; it is not a validated general tutor-quality or learner-outcome measure.
当前 13 情境套件是人工编写的合成设计产物,只覆盖部分健康维度;它不是经过验证的一般 Tutor 质量或学习结果测量。
03 / 05
Use the current public dataset and preserve its fail-closed evaluator boundary.
使用当前公开数据集,并保留其 fail-closed 评估器边界。
npm run benchmark
The canonical run is distinct from the four-case Quickstart and may report errors or no score without Judge configuration.
标准运行不同于四案例 Quickstart;没有 Judge 配置时可能报告错误或不产生分数。
04 / 05
Run the repository example, then send TutorTurnInput to POST /respond.
运行仓库示例,然后把 TutorTurnInput 发送到 POST /respond。
python examples/http-python-tutor/server.py
tutorbench run --http http://127.0.0.1:8000/respond --limit 3
The adapter evaluates the Tutor response at the edge; evaluator-only evidence stays inside TutorBench.
适配器在边缘评估 Tutor 回复;仅供评估器使用的证据保留在 TutorBench 内部。
05 / 05
Collect Product Tutor or canonical model responses, then evaluate a frozen corpus offline.
采集产品 Tutor 或标准模型回复,然后离线评估冻结语料。
tutorbench collect --http http://127.0.0.1:8000/respond --provider <provider> --model <actual-model-id> --prompt-version product-config-v3 --provenance external --limit 3 --output artifacts/product/product.json
tutorbench collect-model --http http://127.0.0.1:9000/generate --provider <provider> --model <actual-model-id> --limit 3 --output artifacts/real-model/model.json
tutorbench evaluate --corpus artifacts/real-model/model.json
# Export the semantic and canonical packets
npm run tutor:export-execution -- -- --case fraction-misconception-001
npm run tutor:export-cases
# Validate and replay a frozen corpus
npm run tutor:corpus:validate -- -- --corpus path/to/corpus.json
npm run benchmark:corpus -- -- --corpus path/to/corpus.json
# OpenAI Judge dry-run (no network)
npm run judge:openai -- -- --dry-run
# Frozen-corpus DeepSeek Judge subset
node dist/src/cli/tutorbench.js evaluate \
--corpus artifacts/real-model/baseline.json \
--limit 1 \
--judge-deepseek
Collection is not publication, calibration, or leaderboard eligibility; frozen evidence can be inspected and replayed offline. The baseline-native-default profile leaves optional temperature, reasoning, and seed controls unconstrained so provider-native behavior is not misrepresented as identical across vendors. tutor:export-cases is the semantic Tutor-visible adapter packet; tutor:export-execution is the canonical benchmark packet used to make model runs comparable. Neither packet includes evaluator-only annotations. The same benchmark does not imply that every provider exposes identical inference knobs.
采集不等于发布、校准或取得排行榜资格;冻结证据可以离线检查和重放。baseline-native-default 配置不约束可选的 temperature、reasoning 和 seed 控制项,避免将不同服务提供方的原生行为误称为一致。tutor:export-cases 是 Tutor 可见的语义适配器数据包;tutor:export-execution 是用于使模型运行可比较的标准基准数据包。两者均不包含仅供评估器使用的注释。使用同一基准并不意味着每个服务提供方都提供相同的推理控制项。
Current workflows use the Node 24 repository source. Published npm 0.1.0 requires Node 22 and includes Quickstart, but has no Tutor Health CLI or API. Check release channels and API boundaries 当前工作流使用 Node 24 的仓库源码。已发布 npm 0.1.0 要求 Node 22,包含 Quickstart,但没有 Tutor Health CLI 或 API。查看发行渠道与 API 边界
How to run如何运行
Traceable runs.
For better research.可追踪的运行。
服务于更好的研究。
Choose Node 24 repository source or the historical Node 22 package.
选择 Node 24 仓库源码,或历史 Node 22 软件包。
Start with Quickstart; Tutor Health requires the repository build.
先运行 Quickstart;Tutor Health 需要仓库构建。
Inspect findings, evidence, coverage, and versioned artifacts before the next Tutor change.
在下一次修改 Tutor 之前,检查发现、证据、覆盖情况与版本化产物。
Execution boundaries执行边界
Different workflows serve different purposes. Here is what is available, what requires additional components, and what remains outside the current public scope.
不同工作流服务于不同目的。这里说明当前可用的内容、需要额外组件的内容,以及仍不属于当前公开范围的内容。
Make the run legible让每次运行都清晰可读
Record the following information with your results to make them easy to verify, replay, and compare.
请将以下信息与结果一起记录,方便验证、重放和比较。
tutor-eval-v0.2a@0.2a.6
correct-answer-wrong-reasoning-001correct-answer-wrong-reasoning-001-zh-CNfraction-misconception-001fraction-misconception-001-zh-CN