Skip to main content
The ArxivKnowledgeBase reads Arxiv articles, converts them into vector embeddings and loads them to a vector databse.

​
Usage

We are using a local PgVector database for this example. Make sure it’s running
pip install arxiv
knowledge_base.py
from phi.knowledge.arxiv import ArxivKnowledgeBase
from phi.vectordb.pgvector import PgVector

knowledge_base = ArxivKnowledgeBase(
    queries=["Generative AI", "Machine Learning"],
    # Table name: ai.arxiv_documents
    vector_db=PgVector(
        table_name="arxiv_documents",
        db_url="postgresql+psycopg://ai:ai@localhost:5532/ai",
    ),
)
Then use the knowledge_base with an Agent:
agent.py
from phi.agent import Agent
from knowledge_base import knowledge_base

agent = Agent(
    knowledge=knowledge_base,
    search_knowledge=True,
)
agent.knowledge.load(recreate=False)

agent.print_response("Ask me about something from the knowledge base")

​
Params

ParameterTypeDefaultDescription
queriesList[str]-Queries to search
readerArxivReaderArxivReader()A ArxivReader that reads the articles and converts them into Documents for the vector database.
vector_dbVectorDb-Vector Database for the Knowledge Base.
num_documentsint5Number of documents to return on search.
optimize_onint-Number of documents to optimize the vector db on.
chunking_strategyChunkingStrategyFixedSizeChunkingThe chunking strategy to use.