Office workflow

Verifiable office workflows for agents

A real benchmark task, shown as a concrete story: messy exported mail files become organized deliverables and auditable checks.

Case · invoice-email-archive-manifest

01 / 05

Workspace unfolds

top folders → picked evidence files

input/workspace/

├─ ▾ mail_export/

│ ├─ 2026-04/thread_cedar_booth/ │ │ ├─ message_0412.json │ │ ├─ packet_a_textlayer.pdf │ │ └─ packet_a_supplement.md │ └─ 2026-05/thread_delta_service/ │ └─ service_packet_textlayer.extract.txt

├─ inventory_shards/ ├─ failed_exports/ ├─ stale_outputs/ └─ archive_out/

JSON

messagemessage_0412.json

message_id: msg-20260412-cedar-01thread_id: thread_cedar_boothattachment_refs: packet_a_textlayer.pdf

PDF

packetpacket_a_textlayer.pdf

Packet title: Cedar booth packettext layer: readable invoice pagehint: check correction mail

notepacket_a_supplement.md

supplement: archive policy noteuse: decision contextnot: standalone invoice

TXT

sidecarservice_packet...txt

Supplier: Delta Field ServicesDocument: DFS-0503-CALLAmount: 1,128.00 USD

How the agent builds the archive

reference → workspace file → evidence → archive_out/

① REFERENCED IN MAIL

② WORKSPACE FILE → JUDGMENT

③ WRITTEN TO archive_out/

message_0412.jsonnames files

{ "attachment_refs": ["packet_a.pdf","missing_ref.pdf" ]}

packet_a.pdf · foundopened & read

INVOICE PACKETPDF · p.1

doc typeInvoice packet

packet idPKT-A-0412

amount1,128.00 USD

linked doc · invoice confirmed

archive_out/output package

archive_out/ tree

├─ candidate_docs/
│  └ packet_a.pdf
├─ missing_refs/
│  └ missing_ref.txt
├─ reference_only/
│  └ retry.log
└─ validation/
   ├ archive_manifest.json
   └ archive_report.md

archive_manifest.json

bucketsourcestatuscandidatepacket_afoundmissingmissing_refabsentreferenceretry.loghist

archive_report.md

✓packet_a.pdf copied

!missing_ref.pdf not found

iretry.log kept as reference

missing_ref.pdflookup

?not found in workspace

missing source

failed_exports/retry.logon disk

found on disknot named by any mail

→

retry.log · readopened & read

status: failedexport retry tracenot an invoice doc

history only · not archivable

Input tree becomes output tree

agent writes new files under archive_out/

input/workspace/+6 new files written → archive_out/

├─ mail_export/ unchanged ├─ inventory_shards/ unchanged ├─ failed_exports/ unchanged ├─ stale_outputs/ unchanged └─ archive_out/before: validation/.keep only

├─ candidate_docs/+ new │ └─ packet_a.pdf ├─ missing_refs/+ new │ └─ missing_ref.txt ├─ reference_only/+ new │ └─ retry.log └─ validation/ ├─ archive_manifest.json+ new └─ archive_report.md+ new

PDFcandidate_docs/packet_a.pdf

INVOICE PACKETp.1

doc typeInvoice packet

amount1,128.00 USD

statusfound · verified

JSONvalidation/archive_manifest.json

bucketsourcestatuscandidatepacket_afoundmissingmissing_refabsentreferenceretry.loghistory

MDvalidation/archive_report.md

✓packet_a.pdf copied

!missing_ref.pdf not found

iretry.log kept as reference

Verified output

deliverables · graded on final workspace state

archive_out/├─ candidate_docs/
│  └ packet_a.pdf
├─ missing_refs/
│  └ missing_ref.txt
├─ reference_only/
│  └ retry.log
└─ validation/
   ├ archive_manifest.json
   └ archive_report.md

JSONarchive_manifest.json

bucketitemtracecandidate_docspacket_a.pdffrom message_0412missing_refsmissing_ref.pdfreferenced, not foundreference_onlyretry.loghistory only

MDarchive_report.md

Found:

- packet_a.pdf matched an attachment reference

Needs review:

- missing_ref.pdf was referenced but absent

Reference only:

- retry.log kept as failure-history evidence

TXTmissing_refs/missing_ref.txt

missing_ref.pdf
source: message_0412 attachment_refs
status: not found in workspace

Verifier / Judge

rules + judge check the delivered files

✓required folders created

✓found attachment copied

✓missing ref recorded

✓reference-only log separated

✓manifest & report consistent

✓no hallucinated source files

graded on final workspace statePASS · 6/6

What the task asks

not QA · a workspace job

Turn messy invoice email exports into a clean archive package.

1read exported mail threads and attachment references

2separate invoice candidates, weak links, and missing refs

3write manifest.json and report.md under archive_out/

single real casecopy-not-moveno live emailno OCR/VLM

input/workspace/
├─ mail_export/
├─ inventory_shards/
├─ failed_exports/
├─ stale_outputs/
└─ archive_out/

JSONmail refs

PDFinvoice packet

TXTtext evidence

OUTdeliverables

Truthful: one real WB Bench task, no mixed-case workspace.Safe: redrawn previews, no hidden gold, no GUI recording.