Beginner’s Guide To Conversational Retrieval Chain Using LangChain
In the last article, we created a retrieval chain that can answer only single questions. Let’s now learn about Conversational Retrieval Chain which will allows us to create chatbots that can answer follow up questions. This requires that the LLM has knowledge of the history of the conversation. LangChain provides us with Conversational Retrieval Chain that works not just on the recent input, but the whole chat history.
Ingredients:
Chains: create_history_aware_retriever, create_stuff_documents_chain, create_retrieval_chain
llm — OpenAI
Retriever
Prompts:
prompt_search_query = ChatPromptTemplate.from_messages([
MessagesPlaceholder(variable_name="chat_history"),
("user","{input}"),
("user","Given the above conversation, generate a search query to look up to get information relevant to the conversation")
)]prompt_get_answer = ChatPromptTemplate.from_messages([
("system", "Answer the user's questions based on the below context:\\n\\n{context}"),
MessagesPlaceholder(variable_name="chat_history",
("user","{input}"),
])Creating the Conversational Retrieval Chain:
LLM
First, let us have an llm created as mentioned in the previous articles.
# Import ChatOpenAI and create an llm with the Open AI API key
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(api_key="<Your Open AI API key here")Retriever
We create a retriever with a vector store as below. You may refer to the previous article on Retrieval Chain to know more about the below code.
The below retriever first downloads data from a website https://www.dasa.org into a vector store and then retrieve relevant information from the vector store based on the user input.
from langchain_community.document_loaders import WebBaseLoader
loader = WebBaseLoader("https://www.dasa.org")docs = loader.load()
from langchain_text_splitters import RecursiveCharacterTextSplitter
text_splitter = RecursiveCharacterTextSplitter()
documents = text_splitter.split_documents(docs)
from langchain_openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
from langchain_community.vectorstores import FAISS
vector = FAISS.from_documents(documents, embeddings)
retriever = vector.as_retriever()For the Retrieval chain, we got a retriever to fetch documents from the vector store relevant to the user input. For the Conversational retrieval chain, we have to get the retriever fetch documents relevant not only to the user input but also to the chat history. Therefore, the retriever needs to have a query based not only on the user input but also on the relevant documents from the chat history. In order to do this, we provide the LLM with the chat history and user input and ask it to derive a search query for the retriever to fetch the relevant data from the vector store.
Prompt To Generate Search Query For Retriever
The prompt contains the user input, the chat history, and a message to generate a search query.
We need to first import a special kind of prompt template called the MessagesPlaceholder. The MessagesPlaceholder is a prompt template that assumes that the variable provided to it as input is a list of messages. In our case, the variable is chat_history, which is the list of previous messages.
The MessagesPlaceholder is part of the prompt along with user input and user message.
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.prompts import MessagesPlaceholder
prompt_search_query = ChatPromptTemplate.from_messages([
MessagesPlaceholder(variable_name="chat_history"),
("user","{input}"),
("user","Given the above conversation, generate a search query to look up to get information relevant to the conversation")
])Retriever Chain
We use the create_history_aware_retriever chain to retrieve the relevant data from the vector store.
from langchain.chains import create_history_aware_retrieverThe create_history_aware_retriever creates a chain that does the following steps:
- Sends a prompt to the LLM with the chat_history and user input to generate a search query for the retriever
- The retriever uses the search query to obtain the relevant documents from the vector store.
So, the inputs to the create_history_aware_retriever are the llm, retriever and the prompt.
retriever_chain = create_history_aware_retriever(llm, retriever, prompt_search_query)The retriever_chain will return the retrieved documents from the vector store that are relevant to the user input and chat history.
Prompt To Get Response From LLM Based on Chat History
The next step is to send the retrieved documents from the vector store along with a prompt to the llm to get the response to the user input.
We create a prompt containing the context (retrieved documents from vector store), chat history and the user input.
prompt_get_answer = ChatPromptTemplate.from_messages([
("system", "Answer the user's questions based on the below context:\\n\\n{context}"),
MessagesPlaceholder(variable_name="chat_history",
("user","{input}"),
])Document Chain
Next, we create a chain using create_stuff_documents_chain which will send the prompt to the llm.
from langchain.chains.combine_documents import create_stuff_documents_chain
document_chain=create_stuff_documents_chain(llm,prompt_get_answer)To recap, we now have a retriever_chain that retrieves the relevant data from vector store, and document_chain that sends the chat_history, relevant data and user input to the llm.
Conversational Retrieval Chain
So, in the final step, we combine retriever_chain and document_chain using create_retrieval_chain to create a Conversational retrieval chain.
from langchain.chains import create_retrieval_chain
retrieval_chain = create_retrieval_chain(retriever_chain, document_chain)Invoking the Chain
All that is remaining is to invoke it.
To test it, we create a sample chat_history and then invoke the retrieval_chain.
from langchain_core.messages import HumanMessage, AIMessage
chat_history = [HumanMessage(content="Does DASA allow anybody to be certified?", AIMessage(content="Yes")]
response = retrieval_chain.invoke({
"chat_history":chat_history,
"input":"How?"
})
print (response['answer'])The output is:
DASA offers certification programs for individuals who are interested in enhancing their skills and knowledge in DevOps principles. By enrolling in the certification programs, individuals can undergo training and assessment to become certified in various roles such as DASA DevOps Leader™, DASA DevOps Coach™, DASA DevOps Product Owner™, and more. The certification programs are designed to help individuals improve their capabilities and contribute to high-performance digital organizations.Thus, the output for the user input “How” has taken the chat history into account. You have successfully created a Conversational Retrieval Chatbot.
In the next article, we will look at creating:
AI Agent — Where the LLM decides what step to take
Previous Articles:
Beginner’s Guide To Retrieval Chain From LangChain
If you like my articles, please follow me to read more articles on AI and AI agents.
