Building a Router Engine with

LlamaIndex

Building a router engine to efficiently route queries

to the right context.
This article is part of our series on building advanced query systems with LlamaIndex.

Introduction
Imagine having a tool that effortlessly routes your queries to retrieve the most relevant
information instantly. This is what a router engine can achieve. Today, we introduce the concept
of a router engine and its role in enhancing query processing. We will guide you through building
a router engine using LlamaIndex with practical examples.

Our goal is to help you understand and implement a router engine with LlamaIndex, improving
how your queries are processed and answered.

Setting Up the Environment

Setting up a proper development environment is crucial for smooth progress and avoiding
issues down the line. A well-configured environment ensures that all necessary dependencies
are installed correctly and that the tools can interact seamlessly. Let’s walk through the steps
required to set up your environment for building a router engine with LlamaIndex.

Install Required Packages

To begin, ensure you have all necessary packages. These packages include openai for
accessing OpenAI models, nest_asyncio for managing asynchronous tasks in Jupyter
Notebooks, and llama_index for indexing and querying documents.

After installing the necessary packages, the next step is to import the required libraries and set
up your OpenAI API key. The API key is essential for accessing OpenAI's models, which will be
used for processing and understanding queries.

from helper import get_openai_api_key

OPENAI_API_KEY = get_openai_api_key()
 import nest_asyncio
nest_asyncio.apply()

Now that your environment is set up, you're ready to start building the router engine. Next we'll
load and prepare the data we'll be working with.

Loading and Preparing Data

For this example, we use the MetaGPT paper. The quality and structure of your data play a
significant role in how effectively your query engine can process and retrieve information. We'll
walk through the steps to download and prepare the data using LlamaIndex.

To begin, we need a dataset to work with. In this case, we use the MetaGPT paper, a research
document available online. If you already have the PDF file, you can skip the download step.
Otherwise, use a simple command to download the paper.

# Download the paper

#!wget "https://openreview.net/pdf?id=VtmBAGCN7o" -O metagpt.pdf
Next, load the document into your environment using the
SimpleDirectoryReader from LlamaIndex. This utility reads the PDF file and
prepares it for further processing.

Next, load the document into your environment using the SimpleDirectoryReader from
LlamaIndex. This utility reads the PDF file and prepares it for further processing.

from llama_index.core import SimpleDirectoryReader

# Load documents
documents = SimpleDirectoryReader(input_files=["metagpt.pdf"]).load_data()

Now that we have our data loaded and prepared, we can move on to defining the language
model (LLM) and embedding model. These models will help in processing and understanding
queries more effectively.

Defining the LLM and Embedding Model

Setting up the language model (LLM) and embedding model is vital for processing and
understanding queries. These models enable the system to interpret and generate meaningful
responses based on the content of the documents. In this section, we'll configure these models
to work with our data.

Use SentenceSplitter for Document Parsing

Before defining the models, it's important to parse the document into manageable chunks. This
helps in processing the data more efficiently and improves the accuracy of the models.

from llama_index.core.node_parser import SentenceSplitter

splitter = SentenceSplitter(chunk_size=1024)
nodes = splitter.get_nodes_from_documents(documents)
The SentenceSplitter class splits the document into smaller chunks, each
with a specified size (in this case, 1024 characters). This parsing helps
the models process smaller pieces of text more effectively.

Define and Configure the LLM and Embedding Models

Set up the LLM and embedding models using OpenAI. These models will be used to generate
embeddings for the text and to process queries.

from llama_index.core import Settings

from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
Settings.llm = OpenAI(model="gpt-3.5-turbo")
Settings.embed_model = OpenAIEmbedding(model="text-embedding-ada-002")

With the language model and embedding model configured, our system is now capable of
understanding and processing queries based on the content of the documents. These models
are crucial for generating meaningful responses and retrieving relevant information efficiently.

Creating Summary and Vector Indexes

Summary and Vector indexes are crucial for efficient query processing. They help retrieve
relevant information quickly and accurately. In this section, we'll create these indexes over our
data to enable both summarization and detailed context retrieval.

Define Summary and Vector Indexes

Creating these indexes over the same data allows for effective summarization and detailed
context retrieval. The SummaryIndex will be used to generate summaries, while the
VectorStoreIndex will be used for retrieving specific information based on similarity searches.
 from llama_index.core import SummaryIndex, VectorStoreIndex
summary_index = SummaryIndex(nodes)
vector_index = VectorStoreIndex(nodes)

Here, SummaryIndex and VectorStoreIndex are initialized using the parsed document nodes.
These indexes will play a key role in handling different types of queries, whether they require
concise summaries or detailed information retrieval.

With these indexes set up, our system can now efficiently handle summarization and
similarity-based retrieval tasks. Next, we'll set up the query engines and create tools for each
engine with appropriate metadata.

Defining Query Engines and Setting Metadata

Query engines and metadata are essential for handling diverse queries effectively. They enable
the system to route queries to the appropriate tools based on the query type. We'll set up the
query engines and create tools for each engine with appropriate metadata.

Set Up Summary and Vector Query Engines

Define and configure the query engines for summarization and vector retrieval. These engines
will process the queries and return the relevant results.

summary_query_engine = summary_index.as_query_engine(
response_mode="tree_summarize",
use_async=True,
)
vector_query_engine = vector_index.as_query_engine()

Create Tools for Each Query Engine

Define tools with appropriate metadata for each query engine. These tools will be used by the
router engine to process different types of queries.

from llama_index.core.tools import QueryEngineTool

summary_tool = QueryEngineTool.from_defaults(
query_engine=summary_query_engine,
description="Useful for summarizing questions related to MetaGPT",
)
 vector_tool = QueryEngineTool.from_defaults(
query_engine=vector_query_engine,
description="Useful for retrieving specific context from the MetaGPT
paper.",
)

With the query engines and tools set up, our system can now handle various types of queries
efficiently. The metadata descriptions help the router engine understand which tool to use for a
given query type.

Building the Router Query Engine

The Router Query Engine routes queries to the appropriate tools based on the query type,
enhancing efficiency and accuracy. We'll set up the router query engine and integrate it with the
tools created earlier.

Define and Integrate the Router Query Engine

Set up the router query engine and integrate it with the tools created earlier. This engine will
decide which tool to use based on the query type and route the query accordingly.

from llama_index.core.query_engine.router_query_engine import

RouterQueryEngine
from llama_index.core.selectors import LLMSingleSelector

query_engine = RouterQueryEngine(
selector=LLMSingleSelector.from_defaults(),
query_engine_tools=[summary_tool, vector_tool],
verbose=True
)
response = query_engine.query("What is the summary of the document?")
print(str(response))
print(len(response.source_nodes))
response = query_engine.query("How do agents share information with other
agents?")
print(str(response))

Putting Everything Together

We now consolidate all the steps into a cohesive workflow to build a fully functional router
engine. This involves using a helper function to streamline the process.
Use a Helper Function to Streamline the Process
Simplify the process with a helper function. This function will combine all the steps into a single,
cohesive workflow, making it easier to manage and execute.

from utils import get_router_query_engine

query_engine = get_router_query_engine("metagpt.pdf")
response = query_engine.query("Tell me about the ablation study results?")
print(str(response))

This helper function ensures that all components work together seamlessly, providing a
streamlined process for building and using the router engine.

Final Thoughts
We’ve explored the intricate process of building a router engine using LlamaIndex. We started
by setting up the development environment, ensuring all necessary dependencies were installed
correctly. Then, we delved into loading and preparing our data, using the MetaGPT paper as an
example.

Next, we configured our language model (LLM) and embedding model to process and
understand queries effectively. We proceeded to create summary and vector indexes, which are
essential for efficient query processing. With our query engines and tools set up, we built a
flexible Router Query Engine capable of routing different types of queries to the appropriate
tools.

Finally, we put everything together into a cohesive workflow, demonstrating how to use a helper
function to streamline the process. Each step has been crucial in creating an efficient and robust
query system.

Stay tuned for the next article in this series, where we will delve into the exciting world of Tool
Calling with LlamaIndex.