What @awacloud/oconv is measured against: public suites, external validators, wild inputs and published benchmarks.
Both benchmarks below score the pdf → md direction, toMd on PDF files
other projects chose. toMd reads the text layer only: there is no OCR,
so a scanned page gives no text.
pypdf-benchmarks-read compares the extracted text with a ground truth,
character by character, on 14 born-digital PDFs. It measures text
extraction and reading order, not structure.opendataloader-bench averages three scores: reading order (NID),
tables (TEDS) and headings (MHS). Its 200 PDFs are untagged, and on an
untagged PDF the reader emits paragraphs only: it recovers no heading
and no table, so the table and heading scores are 0 and the overall
figure rests on reading order alone. Inferring headings and tables from
the layout of an untagged PDF is not implemented; a tagged PDF's
structure tree is read (see the loss matrix).
Measured coverage
Generated from coverage/scorecard.json (format office-scorecard/v1) in the repository, written by the unpublished tool tools/office-scorecard. Each figure names its suite and version, the UTC day and the commit measured, and the command that replays it; a figure is what was measured on that suite, not a claim of conformance to a standard. Low results are published as measured.
Code coverage
| Suite |
Result |
What is measured |
Measured |
Replay |
bun-coverage-src bun 1.3.13 |
lines 98.24 %, functions 99.15 % (mean of 34 source files) |
the coverage gate's model over the declared basis (src/**) |
2026-10-17, cf58af11f |
bun cli.ts office-scorecard axis-a --pkg oconv |
| Suite |
Result |
What is measured |
Measured |
Replay |
arlington-write-conformance-oconv arlington@c48b363e9 |
324 / 324 |
per key instance of the documents oconv fromMd (target pdf) write (oconv-standard-14, oconv-faces, oconv-images; 58 dictionaries and streams reached from the trailer through the Arlington links): a key present passes when its object lists it, its SinceVersion holds for PDF 2.0, its value has one of the key's types valid in 2.0 and, when that type's possible values are literals, one of those valid in 2.0; every key required for 2.0 and absent (inherited keys looked up the /Parent chain) is a failing case; fn: predicates are evaluated only for versions, TRUE / FALSE, || and && (fn:Extension false) — the others are not evaluated: 19 passing instances carry one, and an absent key required by one is skipped; IndirectReference, SpecialCase, DefaultValue and deprecation are not checked; array positions and map entries are walked, not counted; 11 cases not run, each with its reason in the scorecard |
2026-10-14, 31d88da4c |
bun cli.ts office-scorecard axis-b --suite arlington-write-conformance-oconv |
Output validity, judged by an external validator
| Suite |
Result |
What is measured |
Measured |
Replay |
ooxml-validator-oconv OOXML-Validator@2.1.6 |
25 / 25 |
OOXML-Validator 2.1.6 (Open XML SDK DocumentFormat.OpenXml 3.2.0, FileFormatVersions.Microsoft365) reports 0 errors (schema, semantic, package, or a package that does not open) for the written file; a round trip (read, then write with the same package) counts only when the validator reports 0 errors for its source too — a source with errors, or one the writer refuses with a typed error, is skipped with the reason; documents: 25 (25 written, 0 round trips); 1 case not run, each with its reason in the scorecard |
2026-10-15, 31604fbdb |
bun cli.ts office-scorecard axis-c --suite ooxml-validator-oconv |
odfvalidator-oconv odfvalidator@0.13.0 |
30 / 30 |
odfvalidator 0.13.0 (ODF Toolkit, conformance mode, the schemas of the ODF version each file declares) reports no Error and no Fatal message for the written file (warnings are recorded, not failing); a round trip (read, then write with the same package) counts only when the validator reports none for its source too — a source with errors, or one the writer refuses with a typed error, is skipped with the reason; documents: 30 (30 written, 0 round trips); 1 case not run, each with its reason in the scorecard |
2026-10-15, 31604fbdb |
bun cli.ts office-scorecard axis-c --suite odfvalidator-oconv |
Leaderboards
| Suite |
Result |
What is measured |
Measured |
Replay |
pypdf-benchmarks-read py-pdf/benchmarks@778e6c95a |
0.9287 |
mean over the 14 documents of the normalised Indel similarity (2·LCS / (|a| + |b|), over code points) between the ground truth and the plain text of oconv toMd (no OCR; Markdown parsed with @awacloud/md, its text kept), 0–1 |
2026-10-16, 5a84a2cfc |
bun cli.ts office-scorecard axis-e --suite pypdf-benchmarks-read |
opendataloader-bench opendataloader-bench@7af1d8f4d |
0.5753 |
Overall: mean over the 200 documents of the mean of NID (reading order), TEDS (tables) and MHS (headings) where the ground truth defines them, 0–1; the benchmark's own scorers ported, on the Markdown body of oconv toMd (no OCR; the profile-v1 front matter removed) |
2026-10-16, cff19c437 |
bun cli.ts office-scorecard axis-e --suite opendataloader-bench |
Other projects on the same suites
Figures other projects publish (or that their own test data fixes, marked derived), on the same suite and version as an entry above, with where and when they were read.
| Suite |
Project |
Result |
Source |
Read on |
pypdf-benchmarks-read py-pdf/benchmarks@778e6c95a |
pypdfium2 5.13.0 (the benchmark's run of 2026-10-02, mean of its score cache) |
0.9522 |
source |
2026-10-16 |
pypdf-benchmarks-read py-pdf/benchmarks@778e6c95a |
pypdf 6.19.0 (the benchmark's run of 2026-10-02, mean of its score cache) |
0.9486 |
source |
2026-10-16 |
pypdf-benchmarks-read py-pdf/benchmarks@778e6c95a |
PyMuPDF 1.28.2 (the benchmark's run of 2026-10-02, mean of its score cache) |
0.9423 |
source |
2026-10-16 |
pypdf-benchmarks-read py-pdf/benchmarks@778e6c95a |
Tika 3.3.2 (the benchmark's run of 2026-10-02, mean of its score cache) |
0.9306 |
source |
2026-10-16 |
pypdf-benchmarks-read py-pdf/benchmarks@778e6c95a |
pdftotext 0.86.1 (the benchmark's run of 2026-10-02, mean of its score cache) |
0.9019 |
source |
2026-10-16 |
pypdf-benchmarks-read py-pdf/benchmarks@778e6c95a |
pdfminer.six 20260107 (the benchmark's run of 2026-10-02, mean of its score cache) |
0.8793 |
source |
2026-10-16 |
pypdf-benchmarks-read py-pdf/benchmarks@778e6c95a |
pdfplumber 0.11.10 (the benchmark's run of 2026-10-02, mean of its score cache) |
0.7393 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
opendataloader-hybrid 2.2.1 (the benchmark's run of 2026-04-06) |
0.9066 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
nutrient 1.0.1 (the benchmark's run of 2026-04-30) |
0.8851 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
docling 2.84.0 (the benchmark's run of 2026-04-06) |
0.8817 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
marker 1.10.1 (the benchmark's run of 2026-01-06; scored against the ground truth as it stood before its 2026-01-16 and 2026-02-02 revisions) |
0.8608 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
unstructured-hires 0.17.2 (the benchmark's run of 2026-04-06) |
0.8414 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
edgeparse 0.3.0 (the benchmark's run of 2026-04-06) |
0.8370 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
opendataloader 2.2.1 (the benchmark's run of 2026-04-06) |
0.8312 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
mineru 2.7.0 (the benchmark's run of 2026-01-06; scored against the ground truth as it stood before its 2026-01-16 and 2026-02-02 revisions) |
0.8311 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
pymupdf4llm 0.2.0 (the benchmark's run of 2025-11-27; scored against the first ground truth, before its 2025-12-20, 2026-01-16 and 2026-02-02 revisions) |
0.7316 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
unstructured 0.17.2 (the benchmark's run of 2026-04-06) |
0.6858 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
markitdown 0.1.5 (the benchmark's run of 2026-04-06) |
0.5885 |
source |
2026-10-16 |
opendataloader-bench opendataloader-bench@7af1d8f4d |
liteparse 1.2.1 (the benchmark's run of 2026-04-06) |
0.5756 |
source |
2026-10-16 |