Mistral API

Mistral Fine-Tuning API: Here’s What You Need To Know

Mistral API for fine-tuning Mistral 7B and Mistral Small models

Mistral AI has launched a new service and SDK to make fine-tuning their models easier than ever. Here’s what you need to know:

What’s New?

• Fine-Tuning Capabilities

Mistral AI now offers tools to fine-tune their models, allowing users to customize them for specific needs. This fine-tuning improves the performance of large language models (LLMs) for specific applications, making them more efficient and cost-effective. Fine-tuning smaller models can help achieve similar performance to larger ones, cutting down on deployment costs and speeding up applications.

Mistral’s models are great for tasks like text generation, coding assistance, and other natural language processing activities.

• LoRA Training

Mistral uses Low-Rank Adaptation (LoRA) training to make fine-tuning memory-efficient and high-performing. This means you can fine-tune models on various hardware setups without losing performance or efficiency.

• SDK and Infrastructure Support

The mistral-finetune SDK works with multi-GPU setups but can also scale down to just one Nvidia A100 or H100 GPU for smaller models like Mistral 7B. For example, fine-tuning on a dataset like UltraChat (1.4 million dialogs) takes about 30 minutes with eight H100 GPUs.

Mistral LoRA finetuning is more efficient while having similar performance than full Fine-tuning for both Mistral 7B and Mistral Small: The evaluation metric is an normalized internal benchmark very similar to the MTBench evaluation (1 being the reference to full Fine-tuning of Mistral Small).

Three Ways to Fine-Tune

Mistral offers three methods to fine-tune their models:

1. Open-source SDK: Fine-tune models on your own hardware.
2. Serverless Fine-Tuning Services: Use Mistral’s managed services on their platform, la Plateforme, for quick and cost-effective model customization.
3. Custom Training Services: Tailor models using your own data for specialized applications.

How to Get Started

1. Register on la Plateforme: Sign up to access the fine-tuning services.
2. Download the SDK: Get the mistral-finetune SDK from GitHub.
3. Follow the Guide and Tutorial: Use the provided materials to start building your custom fine-tuned models.

Choose the fine-tuning method that best suits your needs and start customizing your AI models with Mistral’s new tools.