Supported Models

Supported LLMs

The following models are currently supported for fine-tuning:

Model	Parameters	Size	Architecture	Context Window
meta-llama/Meta-Llama-3.1-8B	8.03B	16.07 GB	LlamaForCausalLM	32,768 tokens
meta-llama/Meta-Llama-3.1-8B-Instruct	8.03B	16.07 GB	LlamaForCausalLM	32,768 tokens
meta-llama/Meta-Llama-3.1-70B	70.6B	141.1 GB	LlamaForCausalLM	32,768 tokens
meta-llama/Meta-Llama-3.1-70B-Instruct	70.6B	141.1 GB	LlamaForCausalLM	32,768 tokens
meta-llama/Llama-2-7b-hf	6.74B	13.49 GB	LlamaForCausalLM	4,096 tokens
meta-llama/Llama-2-7b-chat-hf	6.74B	13.49 GB	LlamaForCausalLM	4,096 tokens
meta-llama/Llama-2-13b-hf	13B	26.03 GB	LlamaForCausalLM	4,096 tokens
meta-llama/Llama-2-13b-chat-hf	13B	26.03 GB	LlamaForCausalLM	4,096 tokens
meta-llama/Llama-2-70b-hf	69B	137.96 GB	LlamaForCausalLM	4,096 tokens
meta-llama/Llama-2-70b-chat-hf	69B	137.96 GB	LlamaForCausalLM	4,096 tokens
mistralai/Mistral-7B-v0.1	7.24B	14.48 GB	MistralForCausalLM	32,768 tokens
mistralai/Mistral-7B-v0.3	7.25B	14.5 GB	MistralForCausalLM	32,768 tokens
mistralai/Mistral-7B-Instruct-v0.1	7.24B	14.48 GB	MistralForCausalLM	32,768 tokens
mistralai/Mistral-7B-Instruct-v0.2	7.24B	14.48 GB	MistralForCausalLM	32,768 tokens
mistralai/Mistral-7B-Instruct-v0.3	7.25B	14.5 GB	MistralForCausalLM	32,768 tokens
phi-3-mini-4k-instruct	3.82B	7.64GB	Phi3ForCausalLM	4,096 tokens
google/gemma-2b	2.51B	5.01 GB	GemmaForCausalLM	8,192 tokens
google/gemma-1.1-2b-it	2.51B	5.01 GB	GemmaForCausalLM	8,192 tokens
google/gemma-7b	8.54B	17.08 GB	GemmaForCausalLM	8,192 tokens
google/gemma-1.1-7b-it	8.54B	17.08 GB	GemmaForCausalLM	8,192 tokens
google/gemma-2-9b	9.24B	18.48 GB	Gemma2ForCausalLM	8,192 tokens
google/gemma-2-9b-it	9.24B	18.48 GB	Gemma2ForCausalLM	8,192 tokens
google/gemma-2-27b	27.2B	54.45 GB	Gemma2ForCausalLM	8,192 tokens
google/gemma-2-27b-it	27.2B	54.45 GB	Gemma2ForCausalLM	8,192 tokens
Qwen/Qwen2-7B	7.62B	15.23 GB	Qwen2ForCausalLM	32,768 tokens
Qwen/Qwen2-7B-Instruct	7.62B	15.23 GB	Qwen2ForCausalLM	32,768 tokens
Qwen/Qwen2-72B	72.7B	145.41 GB	Qwen2ForCausalLM	8,192 tokens
Qwen/Qwen2-72B-Instruct	72.7B	145.41 GB	Qwen2ForCausalLM	8,192 tokens

LLM Types

As you can see from the table above, there are two types of LLMs available:

1. Base Models: These are foundational models trained on large, diverse datasets without specific task instructions. The main objective of such models is text completion.

2. Instruct Models (usually with base, instruct, chat or it present in their name): These models are fine-tuned from base models to excel in interactive or instruction-based tasks. They are often trained on structured dialogues, enabling them to adopt specific roles or tones (e.g., "helpful assistant") and they are specialized for interpreting and responding to prompts in a conversational, helpful manner.

Currently, only fine-tuning for instruct models is supported. Base model fine-tuning for text completion will be supported in the future.