Mistral

A 7B dense instruction model and a 119B multimodal MoE.

Mistral covers both ends of the matrix. Mistral 7B Instruct is the small end of the text side: a plain dense decoder that leaves the most headroom for everything else on the machine. Mistral Small 4 is a 119B multimodal MoE with about 6.5B parameters active per token, added in v0.5.3, and it sits with the other large vision-language entries on the same snapshot.

Mistral 7B Instruct v0.3

  • Kind: Language model
  • Task: Text generation
  • Parameters: 7B
  • Architecture: Dense transformer
  • Quantization: int4 AWQ, block 128
  • Base model: mistralai/Mistral-7B-Instruct-v0.3
  • Validation status: Validated
  • Measured in v0.5.3: 50.26 tokens/s, 1.20 s to first token, at a 2048-token prompt — all three lengths

Mistral Small 4 119B

Added in v0.5.3. A multimodal MoE that puts Mistral into the same size class as the matrix’s other 100B+ entries.

  • Kind: Vision-language model
  • Task: Image understanding, text generation
  • Parameters: 119B total, 6.5B active
  • Architecture: Sparse mixture of experts + vision encoder
  • Quantization: int4 RTN text decoder, group 128; fp16 vision encoder
  • Base model: mistralai/Mistral-Small-4-119B-2603
  • Validation status: New in v0.5.3
  • Measured in v0.5.3: 25.63 tokens/s, 6.68 s to first token, at a 2048-token prompt — all three lengths