Multi-Agent Chatbot for Multiple PDFs using Autogen

The challenge

A knowledge-driven organization aimed to improve document accessibility and information retrieval by implementing an AI-powered conversational assistant capable of processing multiple PDF documents simultaneously. The objective was to build an intelligent chatbot that could retrieve contextual information from diverse document sources and generate highly accurate, reliable responses. The solution was designed to streamline knowledge discovery, reduce manual document searching, and improve decision-making by enabling users to interact with large volumes of information through a conversational interface.

Key challenges

• Difficulty retrieving relevant information from multiple unstructured PDF documents
• Managing response consistency and contextual relevance across large datasets
• Lack of intelligent orchestration between retrieval, ranking, and synthesis systems
• Challenges in handling incomplete or ambiguous user queries
• Ensuring scalability and accuracy for enterprise-level document intelligence

Technology used

• Programming Language: Python
• Framework: Autogen 0.4 for multi-agent orchestration
• LLM Used: GPT-4o-mini for conversational intelligence and response generation
• Retrieval Technologies: BM25, SEERT, and Cross-Encoder for advanced retrieval and reranking
• Vector Database: Pinecone for semantic search and embedding storage
• Frontend: Streamlit Chat UI for interactive real-time communication

What we built

• Multi-agent architecture for filtering, retrieval, reranking, and response synthesis
• SWARM-based coordination for controlled execution and agent collaboration
• Query preprocessing and normalization for better retrieval accuracy
• Intelligent reranking and verification mechanisms for response quality assurance
• Human-in-the-loop functionality for follow-up handling and conversational control

Objectives

• Build a multi-agent chatbot capable of processing and understanding multiple PDF documents
• Improve answer accuracy through intelligent retrieval and reranking mechanisms
• Enable seamless conversational interaction with large document repositories
• Reduce irrelevant or inaccurate responses through verification workflows
• Implement controlled agent coordination for efficient execution and query handling
• Enhance user engagement with real-time conversational assistance

Outcomes

• Improved response accuracy through modular multi-agent coordination and validation
• Faster and more contextual retrieval from multiple PDF sources
• Better query management using SWARM-based routing and execution control
• Enhanced user experience with real-time conversational interactions
• Increased reliability through intelligent reranking and response verification
• Scalable architecture for enterprise knowledge management and document-based AI systems