Ming-3.0-audio-flash vs Voxtral Mini 4B Realtime 2602

Ming-3.0-audio-flash

Ming-3.0-audio-flash is a cousin for Chinese and English transcription, questions about long audio and spoken replies in built-in or cloned voices, under MIT. It needs about 260 GB on a multi-GPU server, with inclusionAI's own runtime branches or Transformers remote code. The main risks are running repository code to load it and no published safety results.

Voxtral Mini 4B Realtime 2602

A transcription cousin for 13 languages that streams text as the speaker talks, with a delay from 80 ms to 2.4 s, under Apache 2.0 and on Mistral's API. It needs a 16 GB GPU at BF16 through vLLM, Transformers or llama.cpp, or under 4 GB at Q4_K. The main gaps are no timestamps, safety results or answers from Mistral.

Ming-3.0-audio-flash

1. Start it with inclusionAI's recipe, hf download inclusionAI/Ming-3.0-audio-flash --local-dir inclusionAI/Ming-3.0-audio-flash && export MODEL_PATH="$PWD/inclusionAI/Ming-3.0-audio-flash" && pip install -r requirements-vllm-omni.txt && MING_OUTPUT_MODE=asr ./scripts/serve_vllm_omni.sh "$MODEL_PATH", run from a clone of github.com/inclusionAI/Ming-3.0-audio-flash
2. The 128K profile uses tensor parallelism over eight 80 GB H800 GPUs, the SGLang profile four and SGLang-Omni's cookbook two. It doesn't fit the 107.4 GB reference machine
3. Use MING_OUTPUT_MODE=asr with the validated prompt and greedy decoding for transcripts, and split the language label from the text at the tab
4. Use MING_OUTPUT_MODE=text for audio questions and the Talker profile with modalities=["text", "audio"] for spoken replies
5. Send audio as a base64 data URL in an audio_url item for vLLM-Omni, or as input_audio for SGLang
6. Pick a voice with audio.voice and an id, or clone one with ref_audio and its exact prompt_text, never both
7. Turn thinking on per request with chat_template_kwargs {"enable_thinking": true}, it's off by default and the reasoning comes back in think tags
8. Tool calls follow the template's <tool_call> with <arg_key> and <arg_value> pairs, and no parser is named, so parse them yourself
9. Review the repository's Python and the talker/ code, .pyc files included, before loading with trust_remote_code

Voxtral Mini 4B Realtime 2602

1. Serve it with VLLM_DISABLE_COMPILE_CACHE=1 vllm serve mistralai/Voxtral-Mini-4B-Realtime-2602 --compilation_config '{"cudagraph_mode": "PIECEWISE"}' and install soxr, librosa and soundfile; mistral_common 1.9.0 or later comes with vLLM
2. Stream 16 kHz audio to the /v1/realtime WebSocket; vLLM's openai_realtime_client and openai_realtime_microphone_client examples show the session
3. Set temperature 0.0. Change the delay with transcription_delay_ms in tekken.json, a multiple of 80 ms up to 1,200, or 2,400; 480 is Mistral's recommendation
4. One text token is 80 ms, so set --max-model-len to at least seconds / 0.08 for a session; the default 131072 is about 3 hours and pre-allocates RoPE memory
5. In Transformers (5.2.0 or later), resample with processor.feature_extractor.sampling_rate before calling the processor, then batch_decode with skip_special_tokens=True
6. For a small machine, ggml-org's Q4_K_M GGUF with its mmproj runs in llama.cpp, and Mistral Experimental's Q4_K runs in audio.cpp
7. Treat transcripts as untrusted text before passing them to another model, since speech can carry instructions