Search the docs, videos, blog, apps, teams, skills and release notes.
At a glance
What it is
Large language model fine-tuning expertise covering LoRA and QLoRA parameter-efficient methods, full fine-tuning strategies, dataset preparation and curation, instruction tuning, DPO alignment, evaluation frameworks, Hugging Face TRL and PEFT libraries, compute optimization, and deployment of fine-tuned models.
A skill is a written procedure a bot loads when a job calls for it. This one is a single file, SKILL.md, and the whole file is on this page.
The whole file, exactly as it downloads. Read it before you rely on it.
SKILL.md
552 lines · 2,062 words
Fine-tuning large language models adapts pre-trained foundation models to specific domains, tasks, or behaviors. This skill covers the full fine-tuning workflow: deciding when to fine-tune vs prompt engineer, curating high-quality datasets, choosing between full fine-tuning and parameter-efficient methods (LoRA, QLoRA), training with modern tooling, evaluating results rigorously, and deploying fine-tuned models to production.
Large language model fine-tuning expertise covering LoRA and QLoRA parameter-efficient methods, full fine-tuning strategies, dataset preparation and curation, instruction tuning, DPO alignment, evaluation frameworks, Hugging Face TRL and PEFT libraries, compute optimization, and deployment of fine-tuned models.
Nothing. It ships with Murage, which is open source. Your bots run on the AI plan you already have.
Yes. Open it in Settings → Skills to read, edit or duplicate it. Skill Guard scans it again after every edit.
Download the app, connect the AI you already pay for, and tell your Chief of Staff what needs doing. Plan on about ten minutes from install to a working team.
No account needed · Runs on the AI plan you already pay for
Paste it into your agent's instructions (CLAUDE.md, AGENTS.md or a custom GPT), or add it to a bot in Murage.
Ask Ember for the bot, then switch the skill on in its settings.
Step 1
Send your Chief of Staff this message. It proposes the bot and its role first, and creates it when you say go.
Send this to Ember
“Create a bot that uses the LLM Fine-Tuner skill, and show me the plan first.”
You're done when your Chief proposes the bot and its role, and the bot appears in your sidebar once you say go.
Step 2
You're done when you see the bot’s skills and the library.
Step 3
You're done when the skill shows as on for that bot.
Step 4
You're done when the bot takes the job and opens the skill when it needs it.
Prefer to do it yourself? Choose + in the sidebar, then New Bot.Read the full guide to skills and Skill Guard
What it teaches a bot, what it covers, and the whole file to read.
What it covers
Overview
When to Fine-Tune vs Prompt Engineer
Dataset Preparation
LoRA (Low-Rank Adaptation)
QLoRA (Quantized LoRA)
Training with TRL (Transformer Reinforcement Learning)
Evaluation
Deployment
Fine-Tuning Checklist
When to Use
Output Format
Example
Edge Cases
Decision Matrix:
Factor Prompt Engineering Fine-Tuning
------------------------------------------------------------
Latency sensitivity Low (long prompts OK) High (need short prompts)
Training data available < 100 examples 100 - 100K+ examples
Task specificity General tasks Domain-specific behavior
Output format control Moderate High (consistent format)
Cost per inference Higher (long context) Lower (learned behavior)
Iteration speed Minutes Hours to days
Knowledge freshness Retrieval/RAG based Baked into weights
Behavior modification Limited Deep behavior change
Compute budget Zero training cost GPU hours required
Decision: Fine-tune when you need consistent specialized behavior
that cannot be achieved through prompting alone, AND you have
sufficient quality training data.# Standard instruction format (Alpaca-style)
instruction_dataset = [
{
"instruction": "Classify the sentiment of this customer review.",
"input": "The product arrived damaged and customer service was unhelpful.",
"output": "Negative. The review expresses dissatisfaction with both product quality (arrived damaged) and customer service (unhelpful)."
},
{
"instruction": "Extract all medication names from this clinical note.",
"input": "Patient is currently taking metformin 500mg BID, lisinopril 10mg daily, and was prescribed amoxicillin 500mg TID for 7 days.",
"output": "1. Metformin 500mg (twice daily)\n2. Lisinopril 10mg (once daily)\n3. Amoxicillin 500mg (three times daily, 7-day course)"
}
]
# ChatML / conversation format
chat_dataset = [
{
"messages": [
{"role": "system", "content": "You are a medical coding assistant."},
{"role": "user", "content": "Code this diagnosis: Type 2 diabetes with peripheral neuropathy"},
{"role": "assistant", "content": "ICD-10: E11.40 - Type 2 diabetes mellitus with diabetic neuropathy, unspecified"}
]
}
]import hashlib
from datasets import Dataset
class DatasetCurator:
"""Curate high-quality fine-tuning datasets."""
def curate(self, raw_data: list[dict]) -> Dataset:
data = raw_data
initial_count = len(data)
# Step 1: Deduplicate
data = self._deduplicate(data)
print(f"After dedup: {len(data)} (removed {initial_count - len(data)})")
# Step 2: Filter by quality
data = [d for d in data if self._quality_check(d)]
print(f"After quality filter: {len(data)}")
# Step 3: Filter by length
data = [d for d in data if self._length_check(d)]
print(f"After length filter: {len(data)}")
# Step 4: Balance distribution
data = self._balance_distribution(data)
print(f"After balancing: {len(data)}")
return Dataset.from_list(data)
def _deduplicate(self, data):
seen = set()
unique = []
for item in data:
content_hash = hashlib.md5(
(item.get('instruction', '') + item.get('input', '')).encode()
).hexdigest()
if content_hash not in seen:
seen.add(content_hash)
unique.append(item)
return unique
def _quality_check(self, item):
"""Filter low-quality examples."""
output = item.get('output', '')
# Reject empty or very short outputs
if len(output.strip()) < 10:
return False
# Reject outputs that are just the input repeated
if output.strip() == item.get('input', '').strip():
return False
# Reject outputs with obvious errors
if 'as an ai language model' in output.lower():
return False
return True
def _length_check(self, item, max_tokens=2048):
"""Ensure examples fit within context window."""
total_text = item.get('instruction', '') + item.get('input', '') + item.get('output', '')
# Rough token estimate: 1 token per 4 characters
estimated_tokens = len(total_text) / 4
return estimated_tokens <= max_tokens
def _balance_distribution(self, data):
"""Balance task type distribution to prevent overfitting."""
from collections import Counter
if 'task_type' not in data[0]:
return data
type_counts = Counter(d['task_type'] for d in data)
median_count = sorted(type_counts.values())[len(type_counts) // 2]
max_per_type = median_count * 3 # Cap at 3x median
balanced = []
type_tracker = Counter()
for d in data:
if type_tracker[d['task_type']] < max_per_type:
balanced.append(d)
type_tracker[d['task_type']] += 1
return balancedfrom transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from peft import LoraConfig, get_peft_model, TaskType
from trl import SFTTrainer
# Load base model
model_name = "meta-llama/Llama-3.1-8B"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
device_map="auto",
attn_implementation="flash_attention_2",
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token
# LoRA configuration
lora_config = LoraConfig(
r=16, # Rank (4-64, higher = more capacity)
lora_alpha=32, # Scaling factor (typically 2x rank)
target_modules=[ # Which layers to adapt
"q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj",
],
lora_dropout=0.05,
bias="none",
task_type=TaskType.CAUSAL_LM,
modules_to_save=["embed_tokens", "lm_head"], # Optional: train embeddings
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# trainable params: 20M || all params: 8B || trainable%: 0.25%Parameter Recommended Range Notes
--------------------------------------------------------------
r (rank) 8-64 Start with 16. Higher for complex tasks.
8 for simple classification.
32-64 for code/math.
lora_alpha 2x rank Controls learning rate scaling.
alpha/r = effective multiplier.
target_modules All attention + Start with attention only, add MLP
MLP projections if undertrained. More modules = more
capacity but slower training.
lora_dropout 0.0-0.1 0.05 is safe default. Increase if
overfitting (val loss diverges).
learning_rate 1e-4 to 5e-4 Higher than full fine-tuning.
2e-4 is common starting point.
batch_size 4-32 Larger batches stabilize training.
Use gradient accumulation if GPU
memory limited.
epochs 1-5 Start with 3. Watch for overfitting
after epoch 2-3 on small datasets.from transformers import BitsAndBytesConfig
# 4-bit quantization config
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4", # NormalFloat4 quantization
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True, # Nested quantization
)
# Load quantized model
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-3.1-70B",
quantization_config=bnb_config,
device_map="auto",
attn_implementation="flash_attention_2",
)
# Apply LoRA on quantized model
model = get_peft_model(model, lora_config)
# Memory comparison:
# Full fine-tuning 70B: ~280GB VRAM (impossible on consumer GPUs)
# LoRA 70B (bf16): ~140GB VRAM (multi-GPU needed)
# QLoRA 70B (4-bit): ~40GB VRAM (fits on 2x A100 or 1x A100-80GB)from trl import SFTTrainer, SFTConfig
training_args = SFTConfig(
output_dir="./results",
num_train_epochs=3,
per_device_train_batch_size=4,
per_device_eval_batch_size=4,
gradient_accumulation_steps=4, # Effective batch size: 4 * 4 = 16
learning_rate=2e-4,
lr_scheduler_type="cosine",
warmup_ratio=0.1,
weight_decay=0.01,
max_grad_norm=1.0,
# Precision
bf16=True,
# Logging
logging_steps=10,
eval_strategy="steps",
eval_steps=100,
save_strategy="steps",
save_steps=100,
save_total_limit=3,
load_best_model_at_end=True,
metric_for_best_model="eval_loss",
# SFT specific
max_seq_length=2048,
packing=True, # Pack short examples together
dataset_text_field="text",
# Memory optimization
gradient_checkpointing=True,
optim="adamw_torch_fused",
)
trainer = SFTTrainer(
model=model,
train_dataset=train_dataset,
eval_dataset=eval_dataset,
tokenizer=tokenizer,
args=training_args,
)
trainer.train()
trainer.save_model("./fine-tuned-model")from trl import DPOTrainer, DPOConfig
# DPO dataset format: chosen vs rejected completions
dpo_dataset = [
{
"prompt": "Explain quantum entanglement simply.",
"chosen": "Quantum entanglement is like having two coins that always land on opposite sides, no matter how far apart they are. When you flip one and get heads, the other instantly becomes tails. In quantum physics, particles can become 'entangled' so measuring one instantly determines the state of the other.",
"rejected": "Quantum entanglement is a quantum mechanical phenomenon in which the quantum states of two or more objects have to be described with reference to each other, even though the individual objects may be spatially separated. This leads to correlations between observable physical properties of the systems."
}
]
dpo_config = DPOConfig(
output_dir="./dpo-results",
num_train_epochs=1,
per_device_train_batch_size=2,
gradient_accumulation_steps=8,
learning_rate=5e-5, # Lower LR for DPO
beta=0.1, # KL penalty coefficient
max_length=1024,
max_prompt_length=512,
bf16=True,
gradient_checkpointing=True,
logging_steps=10,
)
dpo_trainer = DPOTrainer(
model=model,
ref_model=None, # Uses implicit reference with LoRA
train_dataset=dpo_dataset,
tokenizer=tokenizer,
args=dpo_config,
)
dpo_trainer.train()class FineTuneEvaluator:
"""Evaluate fine-tuned model quality."""
def evaluate(self, model, tokenizer, eval_dataset) -> dict:
results = {}
# 1. Perplexity (lower is better)
results['perplexity'] = self._compute_perplexity(model, tokenizer, eval_dataset)
# 2. Task-specific accuracy
results['task_accuracy'] = self._evaluate_task(model, tokenizer, eval_dataset)
# 3. Regression check against base model
results['regression'] = self._check_regression(model, tokenizer)
# 4. Generation quality (human-evaluate proxy)
results['generation_quality'] = self._evaluate_generation(model, tokenizer, eval_dataset)
return results
def _evaluate_task(self, model, tokenizer, dataset):
"""Evaluate on held-out examples with exact/fuzzy match."""
correct = 0
total = 0
for example in dataset:
prompt = format_prompt(example)
generated = generate(model, tokenizer, prompt, max_new_tokens=256)
expected = example['output']
# Exact match
if generated.strip() == expected.strip():
correct += 1
# Fuzzy match (for free-form text)
elif self._fuzzy_match(generated, expected) > 0.8:
correct += 0.5
total += 1
return correct / total if total > 0 else 0
def _check_regression(self, model, tokenizer):
"""Ensure fine-tuning did not degrade general capabilities."""
regression_prompts = [
"What is 2 + 2?",
"Translate 'hello' to French.",
"Write a haiku about the moon.",
]
# Compare outputs to expected baselines
regressions = []
for prompt in regression_prompts:
output = generate(model, tokenizer, prompt)
if not self._basic_sanity_check(prompt, output):
regressions.append(prompt)
return {"regressions_found": len(regressions), "details": regressions}from peft import PeftModel
# Load base model and LoRA adapter
base_model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-3.1-8B",
torch_dtype=torch.bfloat16,
)
model = PeftModel.from_pretrained(base_model, "./fine-tuned-lora-adapter")
# Merge LoRA weights into base model
merged_model = model.merge_and_unload()
# Save merged model
merged_model.save_pretrained("./merged-model")
tokenizer.save_pretrained("./merged-model")
# Convert to GGUF for llama.cpp / Ollama deployment
# python convert_hf_to_gguf.py ./merged-model --outtype q4_k_mfrom vllm import LLM, SamplingParams
# Deploy merged fine-tuned model
llm = LLM(
model="./merged-model",
dtype="auto",
tensor_parallel_size=1, # Increase for multi-GPU
max_model_len=4096,
gpu_memory_utilization=0.9,
)
sampling_params = SamplingParams(
temperature=0.1, # Low for consistent domain tasks
top_p=0.95,
max_tokens=512,
stop=["<|eot_id|>"],
)
outputs = llm.generate(["Your prompt here"], sampling_params)Planning:
[ ] Confirmed fine-tuning is needed (not achievable via prompting)
[ ] Defined success criteria and evaluation metrics
[ ] Selected base model appropriate for task and budget
[ ] Estimated compute requirements and costs
Data Preparation:
[ ] Collected 500+ high-quality examples (1000+ preferred)
[ ] Deduplicated and quality-filtered dataset
[ ] Split into train/evaluate/test (80/10/10)
[ ] Balanced task distribution
[ ] Verified formatting matches model's expected template
Training:
[ ] Selected method (LoRA for most cases, QLoRA for large models)
[ ] Configured hyperparameters (start with recommended defaults)
[ ] Enabled gradient checkpointing for memory efficiency
[ ] Set up evaluation during training (evaluate every N steps)
[ ] Monitored training loss curve for overfitting
Evaluation:
[ ] Evaluated on held-out test set
[ ] Compared against base model baseline
[ ] Checked for capability regression
[ ] Tested edge cases and adversarial inputs
[ ] Had domain expert review sample outputs
Deployment:
[ ] Merged LoRA weights (or deployed adapter separately)
[ ] Quantized for production (GPTQ, AWQ, or GGUF)
[ ] Set up serving infrastructure (vLLM, TGI, or Ollama)
[ ] Configured monitoring for output quality drift
[ ] Documented model card with training detailsUse this skill when:
Designing or implementing llm fine tuner solutions
Reviewing or improving existing llm fine tuner approaches
Making architectural or implementation decisions about llm fine tuner
Learning llm fine tuner patterns and best practices
Troubleshooting llm fine tuner-related issues
Do NOT use this skill when:
The question is about a fundamentally different technology domain
A more specific sibling skill covers the exact topic needed
The user needs a complete hands-on tutorial rather than expert guidance
# Llm Fine Tuner Analysis
## Context Assessment
[Situation summary and constraints]
## Recommended Approach
[Primary recommendation with rationale]
## Implementation Steps
1. [Step with specific details]
2. [Step with specific details]
3. [Step with specific details]
## Trade-offs and Considerations
- [Key trade-off 1]
- [Key trade-off 2]
## Next Steps
- [Immediate action item]
- [Follow-up action item]Input: "Help me implement llm fine tuner for a medium-scale production application"
Output: A structured analysis covering current state assessment, recommended llm fine tuner approach with specific patterns, implementation roadmap with milestones, and risk mitigation strategies tailored to the application scale and constraints.
Legacy system integration: When llm fine tuner must coexist with legacy approaches, provide a gradual migration path rather than a complete rewrite
Scale mismatch: When the solution complexity exceeds the project scale, recommend a simpler approach and note when to revisit
Team skill gaps: When the team lacks experience with the recommended approach, include learning resources and simpler alternatives
Conflicting requirements: When constraints conflict (e.g., performance vs. maintainability), explicitly state the trade-off and recommend based on stated priorities
SKILL.md
552 lines · 2,062 words
The text of SKILL.md is shown as it downloads, under the Apache-2.0 license, by Ferrox Labs. The license is the one in the file's own frontmatter.
Your new bot
LLM Fine-Tuner bot
Waiting in your sidebar once you say go.
No account needed · Runs on the AI plan you already pay for