Chunking
Agentic Chunking
Agentic chunking is an intelligent method of splitting documents into smaller chunks by using an LLM to determine natural breakpoints in the text. Rather than splitting text at fixed character counts, it analyzes the content to find semantically meaningful boundaries like paragraph breaks and topic transitions.
Usage
Params
Parameter | Type | Default | Description |
---|---|---|---|
model | Model | OpenAIChat | The model to use for chunking. |
max_chunk_size | int | 5000 | The maximum size of each chunk. |
Was this page helpful?