Qwen3.5-4B vs Baidu Unlimited-OCR-2

Qwen3.5-4B

A small vision-language cousin for tool calls, light agent steps and image or video reading on almost any machine, with switchable thinking. Ollama's 3.3 GB build or the 9.3 GB BF16 files run in every major runtime under plain Apache 2.0. The main risks are no safety results, no structured-output guarantee and no answers from Alibaba since March.

Baidu Unlimited-OCR-2

A small MIT document-parsing cousin that reads up to hundreds of pages in one request and returns ordered layout blocks with boxes and HTML tables. It needs an NVIDIA GPU with about 12 GB, through Baidu's remote code or its own SGLang build. The risks are running the publisher's code and accuracy figures that are Baidu's own.

Qwen3.5-4B

1. Smallest start is ollama run qwen3.5:4b (Q4_K_M, about 4 GB with context)
2. Full precision vllm serve Qwen/Qwen3.5-4B --port 8000 --max-model-len 262144 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder
3. Turn thinking off with "chat_template_kwargs": {"enable_thinking": false} in the request body, since it's on by default and /think and /nothink aren't supported
4. There's no generation_config.json, so pass the card's sampling yourself, for example temperature=0.7, top_p=0.8, top_k=20, presence_penalty=1.5 without thinking
5. Keep the context at 128K or more if you rely on thinking, and turn on YaRN only past 262,144 tokens
6. Validate any JSON it returns, since structured output isn't documented

Baidu Unlimited-OCR-2

1. Quickest local run, pip install torch==2.13.0 torchvision pillow transformers==4.52.1 huggingface-hub==0.36.0 tokenizers==0.21.4 addict==2.4.0 einops==0.8.2 easydict==1.13, then AutoModel.from_pretrained("baidu/Unlimited-OCR-2", trust_remote_code=True, use_safetensors=True, torch_dtype=torch.bfloat16).eval().cuda() and model.infer_multi(tokenizer, image_files=pages, output_path="out")
2. To serve it, download the repository, install wheel/sglang-0.5.20.dev10+g8d62f5fb9c-py3-none-any.whl with pymupdf and fastokens in a Python 3.10 venv, and start python -m sglang.launch_server --model-path baidu/Unlimited-OCR-2 --chat-template unlimited-ocr --enable-custom-logit-processor --enable-streaming-session --disable-radix-cache --mem-fraction-static 0.7 --max-prefill-tokens 2048 with the SGLANG_UOCR_ variables from the card
3. Send all pages of a document in one request, images first, then the text <PAGE><image> repeated per page and Multi page parsing., with temperature 0, skip_special_tokens false, max_tokens of about 8,000 per page and the DeepseekOCRNoRepeatNGramLogitProcessor (ngram_size 35, window_size 1024 for several pages)
4. The output is layout blocks, not Markdown; run utils/process_unlimitedOCR2.py from github.com/baidu/Unlimited-OCR, or convert(raw_output) in Python, to get Markdown with HTML tables
5. Render PDFs to images first (the card uses PyMuPDF at 300 DPI), keep save_results off for untrusted documents, and treat extracted text as untrusted before passing it to another model