Mistral
A 7B dense instruction model and a 119B multimodal MoE.
Mistral covers both ends of the matrix. Mistral 7B Instruct is the small end of the text side: a plain dense decoder that leaves the most headroom for everything else on the machine. Mistral Small 4 is a 119B multimodal MoE with about 6.5B parameters active per token, added in v0.5.3, and it sits with the other large vision-language entries on the same snapshot.
Mistral 7B Instruct v0.3
- Kind: Language model
- Task: Text generation
- Parameters: 7B
- Architecture: Dense transformer
- Quantization: int4 AWQ, block 128
- Base model: mistralai/Mistral-7B-Instruct-v0.3
- Validation status: Validated
- Measured in v0.5.3: 50.26 tokens/s, 1.20 s to first token, at a 2048-token prompt — all three lengths
Mistral Small 4 119B
Added in v0.5.3. A multimodal MoE that puts Mistral into the same size class as the matrix’s other 100B+ entries.
- Kind: Vision-language model
- Task: Image understanding, text generation
- Parameters: 119B total, 6.5B active
- Architecture: Sparse mixture of experts + vision encoder
- Quantization: int4 RTN text decoder, group 128; fp16 vision encoder
- Base model: mistralai/Mistral-Small-4-119B-2603
- Validation status: New in v0.5.3
- Measured in v0.5.3: 25.63 tokens/s, 6.68 s to first token, at a 2048-token prompt — all three lengths