Mistral
A 7B dense instruction model — the smallest text model in the matrix.
Mistral 7B Instruct is the small end of the text side of the matrix. It is a plain dense decoder with nothing unusual in it, which is the point: it is the model that leaves the most headroom for everything else on the machine, and the one to reach for when the question is how the rest of a system behaves rather than how large a model can be.
Mistral 7B Instruct v0.3
- Kind: Language model
- Task: Text generation
- Parameters: 7B
- Architecture: Dense transformer
- Quantization: int4 AWQ, block 128
- Base model: mistralai/Mistral-7B-Instruct-v0.3
- Validation status: Validated
- Measured in v0.5.1: 48.02 tokens/s, 1.45 s to first token, at a 2048-token prompt — all three lengths