GLM-OCR: Compact Document OCR and Information Extraction

GLM-OCR is a multimodal OCR model from Z.ai for complex document understanding, built on the GLM-V encoder-decoder architecture (zai-org/GLM-OCR). It reads page images and returns their text, tables, and formulas, and it can fill a caller-supplied JSON schema with structured fields pulled from a document.

Key Features

GLM-OCR scores 94.62 on OmniDocBench V1.5, and the publisher reports strong results across formula recognition, table recognition, and information extraction benchmarks.

The model is tuned for real business documents rather than clean scans. It keeps its accuracy on complex tables, code-heavy pages, seals and stamps, and other difficult layouts.

Training introduces a Multi-Token Prediction (MTP) loss and stable full-task reinforcement learning, which the publisher credits for better training efficiency, recognition accuracy, and generalization.

The model is small and fast. In the publisher's single-replica, single-concurrency speed test it parsed 1.86 PDF pages per second and 0.67 images per second when exporting Markdown.

Prompting

GLM-OCR works with two kinds of prompts. For document parsing, send the image with one of three task prompts: Text Recognition:, Formula Recognition:, or Table Recognition:. For information extraction, send the image with a JSON template whose keys name the fields to fill (for example ID number, name, date of birth, and address); the model's output follows that schema.

Architecture

GLM-OCR pairs a CogViT visual encoder, pre-trained on large-scale image-text data, with a lightweight cross-modal connector that downsamples visual tokens efficiently and a GLM-0.5B language decoder. For full-page document parsing, the publisher's SDK runs a two-stage pipeline: PP-DocLayout-V3 first detects the page layout, then GLM-OCR recognizes each region in parallel and the SDK assembles structured output. The served model handles the recognition stage and direct information extraction.

Use Cases

• Converting scanned pages and PDFs into Markdown for search, RAG, and LLM training data
• Table extraction with structure preserved
• Formula recognition for scientific and technical documents
• Text extraction from code screenshots
• Structured field extraction from IDs, forms, receipts, and invoices into fixed JSON schemas
• Multilingual OCR across Chinese, English, French, Spanish, Russian, German, Japanese, and Korean