llm-training
Use when "training LLM", "finetuning", "RLHF", "distributed training", "DeepSpeed", "Accelerate", "PyTorch Lightning", "Ray Train", "TRL", "Unsloth", "LoRA training", "flash attention", "gradient checkpointing"
How do I install this agent skill?
npx skills add https://github.com/eyadsibai/ltk --skill llm-trainingIs this agent skill safe to install?
- Gen Agent Trust Hubpass
The skill provides informational content and framework comparisons for LLM training and finetuning. It does not contain any executable code, malicious instructions, or security risks.
- Socketpass
No alerts
- Snykpass
Risk: LOW · No issues
- Runlayerwarn
1/1 file flagged
What does this agent skill do?
LLM Training
Frameworks and techniques for training and finetuning large language models.
Framework Comparison
| Framework | Best For | Multi-GPU | Memory Efficient |
|---|---|---|---|
| Accelerate | Simple distributed | Yes | Basic |
| DeepSpeed | Large models, ZeRO | Yes | Excellent |
| PyTorch Lightning | Clean training loops | Yes | Good |
| Ray Train | Scalable, multi-node | Yes | Good |
| TRL | RLHF, reward modeling | Yes | Good |
| Unsloth | Fast LoRA finetuning | Limited | Excellent |
Accelerate (HuggingFace)
Minimal wrapper for distributed training. Run accelerate config for interactive setup.
Key concept: Wrap model, optimizer, dataloader with accelerator.prepare(), use accelerator.backward() for loss.
DeepSpeed (Large Models)
Microsoft's optimization library for training massive models.
ZeRO Stages:
- Stage 1: Optimizer states partitioned across GPUs
- Stage 2: + Gradients partitioned
- Stage 3: + Parameters partitioned (for largest models, 100B+)
Key concept: Configure via JSON, higher stages = more memory savings but more communication overhead.
TRL (RLHF/DPO)
HuggingFace library for reinforcement learning from human feedback.
Training types:
- SFT (Supervised Finetuning): Standard instruction tuning
- DPO (Direct Preference Optimization): Simpler than RLHF, uses preference pairs
- PPO: Classic RLHF with reward model
Key concept: DPO is often preferred over PPO - simpler, no reward model needed, just chosen/rejected response pairs.
Unsloth (Fast LoRA)
Optimized LoRA finetuning - 2x faster, 60% less memory.
Key concept: Drop-in replacement for standard LoRA with automatic optimizations. Best for 7B-13B models.
Memory Optimization Techniques
| Technique | Memory Savings | Trade-off |
|---|---|---|
| Gradient checkpointing | ~30-50% | Slower training |
| Mixed precision (fp16/bf16) | ~50% | Minor precision loss |
| 4-bit quantization (QLoRA) | ~75% | Some quality loss |
| Flash Attention | ~20-40% | Requires compatible GPU |
| Gradient accumulation | Effective batch↑ | No memory cost |
Decision Guide
| Scenario | Recommendation |
|---|---|
| Simple finetuning | Accelerate + PEFT |
| 7B-13B models | Unsloth (fastest) |
| 70B+ models | DeepSpeed ZeRO-3 |
| RLHF/DPO alignment | TRL |
| Multi-node cluster | Ray Train |
| Clean code structure | PyTorch Lightning |
Resources
- Accelerate: https://huggingface.co/docs/accelerate
- DeepSpeed: https://www.deepspeed.ai/
- TRL: https://huggingface.co/docs/trl
- Unsloth: https://github.com/unslothai/unsloth
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/eyadsibai/ltk/llm-training">View llm-training on skillZs</a>