tech
A new open AI coding model is closing in on proprietary options
Open-weights Devstral 2 model scores 72% on industry benchmark, nearing proprietary rivals.
TL;DR
- Mistral AI released Devstral 2, a 123 billion parameter open-weights coding model, and Devstral Small 2, a 24 billion parameter version.
- Devstral 2 achieved a 72.2% score on SWE-bench Verified, a benchmark for AI systems solving GitHub issues.
- Mistral also launched Mistral Vibe, a command-line interface (CLI) for interacting with Devstral models.
- Mistral Vibe can scan file structures, maintain project context, make multi-file changes, and execute shell commands autonomously.
- Devstral Small 2 can run locally on consumer hardware without an internet connection.
- Both models support a 256,000 token context window.
- Devstral 2 is free via API with future pricing of $0.40/million input tokens and $2.00/million output tokens.
- Devstral Small 2 is priced at $0.10/million input tokens and $0.30/million output tokens.
- The 'vibe coding' concept, described by Andrej Karpathy, involves developers accepting AI-generated code without close review.
- Mistral aims for Devstral 2 to handle complex tasks like bug fixing and legacy system modernization at repository scale.