Urdu Nastaliq OCR Computer Vision Data | Human-Verified VLM Evaluation Dataset

Frontier models read historical Urdu Nastaliq with high confidence and frequent mistakes. Urdu has about 246 million speakers (Ethnologue, 2025), but no verified ground truth existed to measure these errors, until now. WHAT WE MEASURED Our public sample is three dense pages of Farhang-i-Asifiyah (1908), 161 lines: - About 70% of lines contained at least one frontier-model mistake. - Many errors were silent, with no uncertainty flagged. - Models dropped over 100 diacritics the scribe wrote. WHAT EACH PAGE INCLUDES - High-resolution scan (400 DPI) - Line-level ground truth typed blind by vetted native experts from the scan, never drafted by a model - Every disagreement adjudicated against the image - Error labels by type: wrong word, missing, invented, diacritic, punctuation, spacing, order WHY IT IS DIFFERENT - Held out: the ground truth is new and unpublished, so it cannot have leaked into public training data. - Evidence first: the method and results are public at the provider.com/human-eval-dataset (PDF report + JSONL, 4,784 records). - Clean provenance: every source work is public domain (published before 1931), traceable to its archive identifier. USE CASES - Evaluating OCR and vision-language models - Regression testing between model releases - Low-resource script benchmarks - Error analysis to guide fine-tuning VOLUME AND DELIVERY The sample is available now. Production runs on demand from 49,421 sourced pages. Delivery is JSONL plus PNG page images, via S3, Google Cloud