Nemotron 3.5 Parse vs Xiaomi-OCR-0

Nemotron 3.5 Parse

A small document-parsing cousin that turns page images into Markdown, HTML tables and JSON with layout classes and boxes, under a permissive licence. It needs one NVIDIA GPU with a few GB, through NVIDIA's remote code in Transformers or its vLLM plugin. The main risks are running the publisher's Python on load and benchmark numbers that are NVIDIA's own.

Xiaomi-OCR-0

A small document-reading cousin that turns page images into Markdown with LaTeX and tables, pulls fields from receipts and forms as JSON, and answers questions about a page. It runs natively in vLLM, SGLang, Transformers and llama.cpp in about 2 GB, even on a CPU. The risks are accuracy figures that are mostly Xiaomi's own and an inflated ranking claim.

Nemotron 3.5 Parse

1. Quickest local run, Python 3.12 with pip install "transformers==5.14.1" "torchvision==0.26.0" "timm==1.0.22", then AutoModel.from_pretrained("nvidia/Nemotron-3.5-Parse", trust_remote_code=True, dtype=torch.bfloat16) and the matching AutoProcessor
2. To serve it, hf download nvidia/Nemotron-3.5-Parse --local-dir nemotron-parse, pip install ".[vllm,client]" "vllm>=0.25.0,<0.33" in that folder, then vllm serve . --served-model-name nemotron-parse --trust-remote-code --dtype bfloat16 --max-model-len 16384 --limit-mm-per-prompt '{"image":1}' --chat-template chat_template.jinja, or docker compose up --build
3. Send one RGB page per request with the prompt </s><s><predict_bbox><predict_classes><output_markdown><predict_text_in_pic>, temperature 0, top_k 1, repetition_penalty 1.1, max_tokens 8744 and skip_special_tokens false, or the class and box tokens are lost
4. Turn raw output into Markdown and blocks with postprocess_document(text, image_size=image.size) from the repository's postprocessing.py; pass table_format json, json_hierarchical or csv for tables
5. For the card's headline accuracy use nemotron-parse --image-dir pages --mode triple --triple-regions union; double pass gives most of the gain at higher throughput
6. Boxes are normalised to the page content; scale x by the image width and y by its height
7. Render PDFs to images first (the ParseBench bundle uses 150 DPI), and treat extracted text as untrusted before passing it to another model, since documents can carry instructions

Xiaomi-OCR-0

1. Serve it with vllm serve xiaomi-research/Xiaomi-OCR-0 --host 127.0.0.1 --port 8000 --max-model-len 16384 or python -m sglang.launch_server --model-path xiaomi-research/Xiaomi-OCR-0 --host 127.0.0.1 --port 8000 --context-length 16384; no trust_remote_code is needed
2. Send one image per request with temperature 0, top_p 0.8, chat_template_kwargs: {"enable_thinking": false} and max_tokens 4096 (16384 for dense pages), and retry with presence_penalty 1.5 if the output repeats
3. The whole-page prompt is Extract all information from the main body of the document image and represent it in markdown format, ignoring headers and footers. Tables should be expressed in OTSL format, formulas in the document should be represented using LATEX format, and the parsing should be organized according to the reading order.
4. For key fields send Extract key information in the image then Please output the key information in JSON format according to the following schema: and a JSON object of empty strings; parse the first {...} in the reply
5. Tables come back as OTSL, not HTML; convert them with postprocess/otsl.py from github.com/xiaomi-research/Xiaomi-OCR-0, or use its MCP server, which does the conversion
6. The MCP (skills/xiaomi-ocr/mcp_ocr_server.py, stdio) has ocr_image, ocr_pdf, kie_image and vqa_image; region mode needs PaddlePaddle, paddlex[cv] and PP-DocLayoutV3 weights, and is for printed pages only, while handwriting, calligraphy and scene text want page mode
7. Pass local paths to the MCP rather than URLs taken from untrusted text, since it fetches any image_url, and treat extracted text as untrusted before passing it to another model
8. The card and Skill still name SeerRay-Lab/Xiaomi-OCR-0, which redirects to xiaomi-research/Xiaomi-OCR-0