Office · Layered verification

Rules check exact artifacts. Judge checks evidence-grounded quality.

The same 50 Office tasks are evaluated through two complementary layers: fine-grained deterministic checks and evidence-grounded Judge rubrics.

Figure · verification coverage across 50 Office tasks

shared denominator

50Office workflow tasks

Deterministic rules

Exact files, schemas, values, state, and safety checks.

RULE layer

18,804emitted checks

145median / task

376mean / task

LLM Judge rubrics

Semantic quality judged only from visible evidence.

JUDGE layer

504rubrics

10median / task

10.1mean / task

What rules check

Radial ring segments show task coverage. Hover a family for numbers.

tasks / 50

Rulestask
coverage

What Judge checks

Rose petals show task coverage. Hover a family for numbers.

tasks / 50

Judgetask
coverage

One output surface, two layers

A safe redraw of two output artifacts — a report and one manifest row. Hover a highlight to see the check it triggers.

archive_report.mdreport sections

                  ## Scope / Boundary
                  offline mail export only · archive_out writes only
                  ## Buckets
                  candidate_docs · needs_review · reference_only · missing_refs
                  ## Readback validation
                  manifest parses · source files retained · no overclaim
                

archive_manifest.jsonone manifest row

                  {
                    "source_path": "mail_export/...",
                    "archive_bucket": "needs_review",
                    "relationship_trace": ["message", "attachment", "text sidecar"],
                    "exception_reason_codes": ["weak_link"],
                    "decision_rationale": "weak link; reviewer action kept",
                    "archive_path": "archive_out/..."
                  }
                

What each layer checks

RULE · exactJUDGE · qualitativeEVIDENCE · shared

RULEManifest/report exists, parses, and uses the required schema.

RULESource files are retained and only archive_out is mutated.

RULEBucket, duplicate group, and missing-reference states match deterministic contracts.

EVIDENCEsource_path and relationship_trace bind output rows to visible mail files.

JUDGERationale explains weak links and review handoff without contradicting evidence.

JUDGEReport posture stays useful for the reader and avoids overclaiming live/OCR work.