The S3TextKnowledgeBase reads text files from an S3 bucket, converts them into vector embeddings and loads them to a vector databse.

​
Usage

We are using a local PgVector database for this example. Make sure it’s running
pip install textract

from phi.knowledge.s3.text import S3TextKnowledgeBase
from phi.vectordb.pgvector import PgVector

db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"

knowledge_base = S3TextKnowledgeBase(
    bucket_name="phi-public",
    key="recipes/recipes.docx",
    vector_db=PgVector(table_name="recipes", db_url=db_url),
)
Then use the knowledge_base with an Agent: 
from phi.agent import Agent
from knowledge_base import knowledge_base

agent = Agent(
    knowledge=knowledge_base,
    search_knowledge=True,
)
agent.knowledge.load(recreate=False)

agent.print_response("How to make Hummus?")

​
Params

ParameterTypeDefaultDescription
formatsList[str][".doc", ".docx"]Formats accepted by this knowledge base.
readerS3TextReaderS3TextReader()A S3TextReader that converts the Text files into Documents for the vector database.
vector_dbVectorDb-Vector Database for the Knowledge Base.
num_documentsint5Number of documents to return on search.
optimize_onint-Number of documents to optimize the vector db on.
chunking_strategyChunkingStrategyFixedSizeChunkingThe chunking strategy to use.