chunking-strategies
Optimize document chunking for RAG performance and retrieval quality. Use this skill when splitting documents, choosing chunk sizes, implementing semantic chunking, or improving RAG retrieval accuracy. Activate when: chunking, split documents, chunk size, text splitting, document processing, RAG performance, semantic chunking, overlap.
How do I install this agent skill?
npx skills add https://github.com/latestaiagents/agent-skills --skill chunking-strategiesIs this agent skill safe to install?
- Gen Agent Trust Hubpass
The skill provides comprehensive documentation and Python code examples for implementing various document chunking strategies in RAG pipelines. It uses standard industry libraries and follows best practices for text processing without any identified security risks.
- Socketpass
No alerts
- Snykwarn
Risk: MEDIUM · 1 issue
- Runlayerwarn
1/1 file flagged
What does this agent skill do?
Chunking Strategies for RAG
Chunk boundaries decide what the retriever can find. Pick a splitter that respects the document's structure, then check it against real queries.
When to Use
- Designing a new RAG pipeline
- Retrieval quality is poor because answers are split across chunk boundaries
- Documents mix content types (code, tables, prose, HTML)
- Balancing context-window limits against retrieval precision
Why Chunking Matters
Poor chunking causes:
- Context fragmentation (answers split across chunks)
- Irrelevant retrieval (too much noise in chunks)
- Lost relationships (parent-child content separated)
- Wasted tokens (chunks too large or too small)
Chunking Methods Comparison
| Method | Best For | Trade-off | Implementation |
|---|---|---|---|
| Fixed-size / recursive | Uniform content, prototyping, general text | Ignores meaning; recursive at least prefers paragraph and sentence breaks | Easy |
| Document-aware (Markdown, HTML, code) | Structured docs | Needs a splitter per format | Easy |
| Semantic | Prose where topics shift without headings | Slower; needs embedding calls at ingest | Medium |
| Parent-child | Hierarchical docs, answers that need surrounding context | Two stores to keep in sync | Medium |
| Late chunking | Chunks that depend on document-wide context | Needs a long-context embedding model that exposes token embeddings | Complex |
Which method retrieves best depends on your documents and queries; compare candidates with the evaluation at the end of this skill.
Pattern 1: Fixed-Size with Overlap
The baseline approach. RecursiveCharacterTextSplitter tries larger separators first (paragraphs, then lines, then sentences) and falls back to smaller ones, so it keeps paragraphs intact when it can. Start here and change only when evaluation shows a problem.
from langchain_text_splitters import RecursiveCharacterTextSplitter
def create_fixed_chunks(
text: str,
chunk_size: int = 512,
chunk_overlap: int = 50
) -> list[str]:
"""
Split text into fixed-size chunks with overlap.
Guidelines:
- chunk_size: 256-1024 tokens is a common starting range
- overlap: 10-20% of chunk_size so sentences at a boundary appear in both chunks
- length_function=len measures characters, not tokens. To size in tokens, use
RecursiveCharacterTextSplitter.from_tiktoken_encoder(...) or pass a tokenizer's
length function.
"""
splitter = RecursiveCharacterTextSplitter(
chunk_size=chunk_size,
chunk_overlap=chunk_overlap,
length_function=len,
separators=["\n\n", "\n", ". ", " ", ""]
)
return splitter.split_text(text)
Avoid when: documents have strong structure (headings, code, tables); use a document-aware splitter (Pattern 5) instead.
Pattern 2: Semantic Chunking
Group by meaning, not arbitrary boundaries:
from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai import OpenAIEmbeddings
def create_semantic_chunks(text: str) -> list[str]:
"""
Split text based on semantic similarity between sentences.
Keeps related content together.
"""
embeddings = OpenAIEmbeddings()
splitter = SemanticChunker(
embeddings=embeddings,
breakpoint_threshold_type="percentile",
breakpoint_threshold_amount=95 # Higher = fewer, larger chunks
)
return splitter.split_text(text)
Trade-off: every sentence is embedded at ingest time, so indexing is slower and costs more than recursive splitting.
Custom Semantic Chunking
import numpy as np
from sentence_transformers import SentenceTransformer
def semantic_chunk(
sentences: list[str],
model_name: str = "all-MiniLM-L6-v2",
threshold: float = 0.5
) -> list[list[str]]:
"""
Group sentences by semantic similarity.
"""
model = SentenceTransformer(model_name)
embeddings = model.encode(sentences)
chunks = []
current_chunk = [sentences[0]]
for i in range(1, len(sentences)):
# Cosine similarity between consecutive sentences
sim = np.dot(embeddings[i-1], embeddings[i]) / (
np.linalg.norm(embeddings[i-1]) * np.linalg.norm(embeddings[i])
)
if sim >= threshold:
current_chunk.append(sentences[i])
else:
chunks.append(current_chunk)
current_chunk = [sentences[i]]
chunks.append(current_chunk)
return chunks
Pattern 3: Parent-Child Chunking
Retrieve small, return with context:
from llama_index.core.node_parser import (
HierarchicalNodeParser,
SentenceSplitter,
get_leaf_nodes
)
from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.core.retrievers import AutoMergingRetriever
def create_hierarchical_index(documents):
"""
Create parent-child chunk hierarchy.
Small chunks for retrieval, auto-merge to parents for context.
"""
# Define chunk sizes for each level
node_parser = HierarchicalNodeParser.from_defaults(
chunk_sizes=[2048, 512, 128] # Parent → Child → Leaf
)
nodes = node_parser.get_nodes_from_documents(documents)
leaf_nodes = get_leaf_nodes(nodes)
# Store all nodes
storage_context = StorageContext.from_defaults()
storage_context.docstore.add_documents(nodes)
# Index only leaf nodes
index = VectorStoreIndex(
leaf_nodes,
storage_context=storage_context
)
# Retriever auto-merges to parents when siblings retrieved
retriever = AutoMergingRetriever(
index.as_retriever(similarity_top_k=12),
storage_context=storage_context,
simple_ratio_thresh=0.3 # Merge if 30%+ siblings retrieved
)
return retriever
LangChain Equivalent: ParentDocumentRetriever
In LangChain 1.x the retriever lives in langchain-classic:
from langchain_classic.retrievers import ParentDocumentRetriever # pip install langchain-classic
from langchain_core.stores import InMemoryStore
from langchain_text_splitters import RecursiveCharacterTextSplitter
retriever = ParentDocumentRetriever(
vectorstore=vectorstore, # indexes the small child chunks
docstore=InMemoryStore(), # holds the parents; use a persistent store in production
child_splitter=RecursiveCharacterTextSplitter(chunk_size=400),
parent_splitter=RecursiveCharacterTextSplitter(chunk_size=2000),
)
retriever.add_documents(docs)
parents = retriever.invoke("query") # matches children, returns their parents
Small chunks give precise matches; returning the parent gives the model enough context to answer.
Pattern 4: Late Chunking
Embed full document first, then chunk - preserves global context:
def late_chunking(
document: str,
model,
chunk_size: int = 512
) -> list[dict]:
"""
PSEUDOCODE: `model.tokenize`, `model.encode_tokens`, and `model.decode` stand in
for your embedding model's API. You need a long-context model that returns
token-level embeddings; see the late chunking paper for a reference implementation.
Late chunking: embed the whole document, then pool token embeddings per chunk,
so each chunk vector reflects document-level context.
"""
# 1. Get token-level embeddings for full document
tokens = model.tokenize(document)
token_embeddings = model.encode_tokens(tokens)
# 2. Split into chunks
chunks = []
for i in range(0, len(tokens), chunk_size):
chunk_tokens = tokens[i:i + chunk_size]
chunk_embeddings = token_embeddings[i:i + chunk_size]
# 3. Pool chunk embeddings (mean pooling)
chunk_vector = np.mean(chunk_embeddings, axis=0)
chunks.append({
"text": model.decode(chunk_tokens),
"embedding": chunk_vector
})
return chunks
Pattern 5: Markdown/Code-Aware Chunking
from langchain_text_splitters import (
MarkdownHeaderTextSplitter,
Language,
RecursiveCharacterTextSplitter
)
def chunk_markdown(text: str) -> list[dict]:
"""Split markdown by headers, preserving structure."""
headers_to_split_on = [
("#", "h1"),
("##", "h2"),
("###", "h3"),
]
splitter = MarkdownHeaderTextSplitter(
headers_to_split_on=headers_to_split_on
)
return splitter.split_text(text)
def chunk_code(code: str, language: str = "python") -> list[str]:
"""Split code respecting language syntax."""
lang_map = {
"python": Language.PYTHON,
"javascript": Language.JS,
"typescript": Language.TS,
}
splitter = RecursiveCharacterTextSplitter.from_language(
language=lang_map.get(language, Language.PYTHON),
chunk_size=1000,
chunk_overlap=100
)
return splitter.split_text(code)
def chunk_html(html: str) -> list:
"""Split HTML by header tags; header text is added to each chunk's metadata."""
from langchain_text_splitters import HTMLHeaderTextSplitter
splitter = HTMLHeaderTextSplitter(
headers_to_split_on=[("h1", "Header 1"), ("h2", "Header 2"), ("h3", "Header 3")]
)
return splitter.split_text(html)
Header-based splitters can produce sections longer than your target size; run a size-based splitter over oversized sections afterwards.
Metadata Enrichment
Attach metadata to every chunk at ingest. It enables filtering, citations, access control, and targeted re-indexing:
import hashlib
from datetime import datetime, timezone
def enrich(chunks: list, source_doc) -> list:
for i, chunk in enumerate(chunks):
chunk.metadata.update({
"source": source_doc.metadata["source"], # URL or file path, for citations
"section": chunk.metadata.get("h2") or chunk.metadata.get("Header 2"),
"chunk_index": i,
"total_chunks": len(chunks),
"content_hash": hashlib.sha256(chunk.page_content.encode()).hexdigest(),
"has_code": "```" in chunk.page_content,
"indexed_at": datetime.now(timezone.utc).isoformat(),
})
return chunks
The content hash lets you skip re-embedding unchanged chunks when a document is updated.
Chunk Size Guidelines
Starting points to tune, not measured optima (sizes in tokens):
| Content Type | Starting Size | Overlap |
|---|---|---|
| Q&A / FAQ | 256-512 | 25-50 |
| Technical docs | 512-1024 | 50-100 |
| Legal documents | 1024-2048 | 100-200 |
| Code | 500-1000 | 50-100 |
| Conversations | 256-512 | 50-100 |
Evaluation: How to Know If Chunking Is Good
def evaluate_chunking(chunks: list[str], test_queries: list[dict]):
"""
Evaluate chunk quality with test queries.
test_queries format:
[{"query": "What is X?", "expected_chunk_contains": "X is..."}]
"""
results = {
"avg_chunk_size": np.mean([len(c) for c in chunks]),
"chunk_size_std": np.std([len(c) for c in chunks]),
"total_chunks": len(chunks),
"retrieval_hits": 0
}
for tq in test_queries:
# Check if expected content is in a single chunk
for chunk in chunks:
if tq["expected_chunk_contains"] in chunk:
results["retrieval_hits"] += 1
break
results["hit_rate"] = results["retrieval_hits"] / len(test_queries)
return results
This checks whether an answer survives chunking intact. To measure whether the retriever then finds it, use the retrieval metrics in rag-evaluation.
Structural Checklist
Sample chunks from each document type and check:
- Chunks don't break mid-sentence
- Code blocks stay intact
- Tables aren't split across chunks (or each piece repeats the header row)
- Headings stay with their content
- Overlap preserves continuity at boundaries
- Metadata is present and supports the filters you need
Best Practices
- Match chunk size to query length - Chunks should be similar size to expected queries
- Preserve meaning boundaries - Never split mid-sentence or mid-paragraph
- Include metadata - Add source, page, section info to each chunk
- Test with real queries - Evaluate on your actual use cases
- Consider the embedding model - Stay within its maximum input length; text beyond it is truncated
- Start with recursive splitting - Move to semantic or parent-child chunking when evaluation shows a gap
Quick Decision Tree
What type of content?
├─ Structured (headers, sections)
│ └─ Use: Markdown/recursive splitter + hierarchy
├─ Unstructured (prose, articles)
│ └─ Use: Semantic chunking
├─ Code
│ └─ Use: Language-aware splitter
└─ Mixed
└─ Use: Parent-child with semantic leaves
References
- LangChain text splitters: https://docs.langchain.com/oss/python/integrations/splitters
- LlamaIndex node parsers (HierarchicalNodeParser) and AutoMergingRetriever: https://developers.llamaindex.ai/python/framework/module_guides/loading/node_parsers/modules/
- Late chunking paper (Günther et al., 2024): https://arxiv.org/abs/2409.04701
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/latestaiagents/agent-skills/chunking-strategies">View chunking-strategies on skillZs</a>