Question Answering

LangChain Tutorial 5

Jan Kirenz

Question Answering

Setup

Python

from langchain.prompts import PromptTemplate
from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Chroma
import datetime
from dotenv import load_dotenv, find_dotenv
import os
import openai
# import sys
# sys.path.append('../..')

_ = load_dotenv(find_dotenv())  # read local .env file

openai.api_key = os.environ['OPENAI_API_KEY']

RetrievalQA chain

Vector Database setup

persist_directory = '../docs/chroma/'
embedding = OpenAIEmbeddings()
vectordb = Chroma(persist_directory=persist_directory,
                  embedding_function=embedding)
print(vectordb._collection.count())
  • 209

ChatOpenAI model

llm_name = "gpt-3.5-turbo"

llm = ChatOpenAI(model_name=llm_name, temperature=0)

RetrievalQA chain

qa_chain = RetrievalQA.from_chain_type(
    llm,
    retriever=vectordb.as_retriever()
)

Result

result = qa_chain({"query": question})
result["result"]
  • ‘The major topics for this class are machine learning and its various extensions.’

RetrievalQA chain with Template

Prompt template

template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. Use three sentences maximum. Keep the answer as concise as possible. Always say "thanks for asking!" at the end of the answer. 
{context}
Question: {question}
Helpful Answer:"""

QA_CHAIN_PROMPT = PromptTemplate.from_template(template)

Question answer chain

qa_chain = RetrievalQA.from_chain_type(
    llm,
    retriever=vectordb.as_retriever(),
    return_source_documents=True,
    chain_type_kwargs={"prompt": QA_CHAIN_PROMPT}
)

Question and result

question = "Is probability a class topic?"
result = qa_chain({"query": question})
result["result"]
  • ‘Yes, probability is a class topic. Thanks for asking!’

Source documents

result["source_documents"][0]
  • Document(page_content=“of this class will not be very program ming intensive, although we will do some , mostly in either MATLAB or Octa ve. I’ll say a bit more about that later. also assume familiarity with basic proba bility and statistics. So most undergraduate class, like Stat 116 taught here at Stanford, will be more than enough. I’m gonna all of you know what ra ndom variables are, that all of you know what expectation , what a variance or a random variable is. And in case of some of you, it’s been a while you’ve seen some of this material. At some of the discussion sections, we’ll actually over some of the prerequisites, sort of as a refresher course under prerequisite class. ‘ll say a bit more about that later as well. , I also assume familiarity with basi c linear algebra. And again, most undergraduate algebra courses are more than enough. So if you’ve taken courses like Math 51, , Math 113 or CS205 at Stanford, that would be more than enough. Basically, I’m assume that all of you know what matrix es and vectors are, that you know how to matrices and vectors and multiply matrix and matrices, that you know what a matrix inverse is. If you know what an eigenvect or of a matrix is, that’d be even better. if you don’t quite know or if you’re not qu ite sure, that’s fine, too. We’ll go over it in review sections.”, metadata={’source’: ‘../docs/cs229_lectures/MachineLearning-Lecture01.pdf’, ‘page’: 4})

RetrievalQA Chain Types

Map Reduce

qa_chain_mr = RetrievalQA.from_chain_type(
    llm,
    retriever=vectordb.as_retriever(),
    chain_type="map_reduce"
)

Result

result = qa_chain_mr({"query": question})
result["result"]
  • ‘There is no mention of probability as a class topic in the provided text.’

LangChain plus platform

Basics

  • If you wish to experiment on the LangChain plus platform:
    • Go to langchain plus platform and sign up
    • Create an API key from your account’s settings
    • Use this API key in the code below
    • uncomment the code

Code

# os.environ["LANGCHAIN_TRACING_V2"] = "true"
# os.environ["LANGCHAIN_ENDPOINT"] = "https://api.langchain.plus"
# os.environ["LANGCHAIN_API_KEY"] = "..."  # replace dots with your api key
# qa_chain_mr = RetrievalQA.from_chain_type(
#     llm,
#     retriever=vectordb.as_retriever(),
#     chain_type="map_reduce"
# )
# result = qa_chain_mr({"query": question})
# result["result"]
# qa_chain_mr = RetrievalQA.from_chain_type(
#     llm,
#     retriever=vectordb.as_retriever(),
#     chain_type="refine"
# )
# result = qa_chain_mr({"query": question})
# result["result"]

RetrievalQA Limitations

Conversational history

  • QA fails to preserve conversational history.
qa_chain = RetrievalQA.from_chain_type(
    llm,
    retriever=vectordb.as_retriever()
)
question = "Is probability a class topic?"
result = qa_chain({"query": question})
result["result"]
  • ‘Yes, probability is a topic that will be covered in this class. The instructor assumes familiarity with basic probability and statistics.’

Conversational history

question = "why are those prerequesites needed?"
result = qa_chain({"query": question})
result["result"]
  • ‘The prerequisites are needed because they provide the foundational knowledge and skills necessary to understand and apply machine learning algorithms. knowledge of computer science and computer skills is important because machine learning algorithms often involve programming and working with data. Understanding concepts like big-O notation helps in analyzing the efficiency and scalability of algorithms.with probability and statistics is necessary because machine learning involves working with data and making predictions based on statistical models. Understanding concepts like random variables, expectation, and variance is crucial in understanding and evaluating machine learning algorithms.knowledge of linear algebra is important because many machine learning algorithms involve manipulating matrices and vectors. Understanding concepts like matrix multiplication, matrix inverse, and eigenvectors is essential in understanding and implementing these algorithms., these prerequisites provide the necessary background knowledge and skills to effectively learn and apply machine learning algorithms.’

Limitations

  • Note, The LLM response varies.

  • Some responses do include a reference to probability which might be gleaned from referenced documents.

  • The point is simply that the model does not have access to past questions or answers, this will be covered in the next tutorial (Tutorial 6).

Acknowledgments

What’s next?

Congratulations! You have completed this tutorial 👍

Next, you may want to go back to the lab’s website

Jan Kirenz