from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.document_loaders import PyPDFLoader
from langchain.retrievers import TFIDFRetriever
from langchain.retrievers import SVMRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor
from langchain.retrievers import ContextualCompressionRetriever
from langchain.chains.query_constructor.base import AttributeInfo
from langchain.retrievers.self_query.base import SelfQueryRetriever
from langchain.llms import OpenAI
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from dotenv import load_dotenv, find_dotenv
import os
import openai
# import sys
# sys.path.append('../..')
_ = load_dotenv(find_dotenv()) # read local .env file
openai.api_key = os.environ['OPENAI_API_KEY']Vectorstore Retrieval
LangChain Tutorial 4
Jan Kirenz
Vectorstore Retrieval
Grasp advanced techniques for accessing and indexing data in the vector store, enabling you to retrieve the most relevant information beyond semantic queries.
Setup
Python
Vector Database
Setup
Let’s get our vectorDB from Tutorial 3.
Example
texts = [
"""The Amanita phalloides has a large and imposing epigeous (aboveground) fruiting body (basidiocarp).""",
"""A mushroom with a large fruiting body is the Amanita phalloides. Some varieties are all-white.""",
"""A. phalloides, a.k.a Death Cap, is one of the most poisonous of all known mushrooms.""",
]Result 1
- [Document(page_content=‘A mushroom with a large fruiting body is the Amanita phalloides. Some varieties are all-white.’, metadata={}), Document(page_content=‘The Amanita phalloides has a large and imposing epigeous (aboveground) fruiting body (basidiocarp).’, metadata={})]
Result 2
- [Document(page_content=‘A mushroom with a large fruiting body is the Amanita phalloides. Some varieties are all-white.’, metadata={}), Document(page_content=‘A. phalloides, a.k.a Death Cap, is one of the most poisonous of all known mushrooms.’, metadata={})]
Addressing Diversity
Basics
Addressing Diversity: Maximum marginal relevance (MMR)
In Tutorial 3 we introduced one problem: how to enforce diversity in the search results.
Maximum marginal relevancestrives to achieve both relevance to the query and diversity among the results.
Question about matlab
- Similarity search
Results
- ‘those homeworks will be done in either MATLA B or in Octave, which is sort of — I some people’
MMR
- Note the difference in results with
MMR.
- ‘those homeworks will be done in either MATLA B or in Octave, which is sort of — I some people’
Addressing Specificity: Metadata
Basics
Addressing Specificity: working with metadata
In Tutorial 3, we showed that a question about the third lecture can include results from other lectures as well.
To address this, many vectorstores support operations on
metadata.metadataprovides context for each embedded chunk.
Question about third lecture
- Similarity search
Result
- {‘source’: ‘../docs/cs229_lectures/MachineLearning-Lecture03.pdf’, ‘page’: 0}
- {‘source’: ‘../docs/cs229_lectures/MachineLearning-Lecture03.pdf’, ‘page’: 14}
- {‘source’: ‘../docs/cs229_lectures/MachineLearning-Lecture03.pdf’, ‘page’: 4}
Addressing Specificity: Self-query retriever
Basics
Addressing Specificity: working with metadata using self-query retriever
But we have an interesting challenge: we often want to infer the metadata from the query itself.
To address this, we can use
SelfQueryRetriever, which uses an LLM to extract:
- The
querystring to use for vector search - A metadata filter to pass in as well
- Most vector databases support metadata filters, so this doesn’t require any new databases or indexes.
metadata_field_info
metadata_field_info = [
AttributeInfo(
name="source",
description="The lecture the chunk is from, should be one of `../docs/cs229_lectures/MachineLearning-Lecture01.pdf`, `../docs/cs229_lectures/MachineLearning-Lecture02.pdf`, or `../docs/cs229_lectures/MachineLearning-Lecture03.pdf`",
type="string",
),
AttributeInfo(
name="page",
description="The page from the lecture",
type="integer",
),
]document_content_description
Question about third lecture
- Retriever
Result
{'source': '../docs/cs229_lectures/MachineLearning-Lecture03.pdf', 'page': 14}
{'source': '../docs/cs229_lectures/MachineLearning-Lecture03.pdf', 'page': 0}
{'source': '../docs/cs229_lectures/MachineLearning-Lecture03.pdf', 'page': 10}
{'source': '../docs/cs229_lectures/MachineLearning-Lecture03.pdf', 'page': 10}Additional Tricks: Compression
Basics
Another approach for improving the quality of retrieved docs is compression.
Information most relevant to a query may be buried in a document with a lot of irrelevant text.
Passing that full document through your application can lead to more expensive LLM calls and poorer responses.
Contextual compression is meant to fix this.
Helper function: pretty print
Load LLM
ContextualCompressionRetriever
Question about matlab
- Retriever
Result
Document 1:
"MATLAB is I guess part of the programming language that makes it very easy to write codes using matrices, to write code for numerical routines, to move data around, to plot data. And it's sort of an extremely easy to learn tool to use for implementing a lot of learning algorithms."
----------------------------------------------------------------------------------------------------
Document 2:
"MATLAB is I guess part of the programming language that makes it very easy to write codes using matrices, to write code for numerical routines, to move data around, to plot data. And it's sort of an extremely easy to learn tool to use for implementing a lot of learning algorithms."
----------------------------------------------------------------------------------------------------
Document 3:
"And the student said, "Oh, it was the MATLAB." So for those of you that don't know MATLAB yet, I hope you do learn it. It's not hard, and we'll actually have a short MATLAB tutorial in one of the discussion sections for those of you that don't know it."
----------------------------------------------------------------------------------------------------
Document 4:
"And the student said, "Oh, it was the MATLAB." So for those of you that don't know MATLAB yet, I hope you do learn it. It's not hard, and we'll actually have a short MATLAB tutorial in one of the discussion sections for those of you that don't know it."Combining Various Techniques
ContextualCompressionRetriever with MMR
Question
- Retriever
Result
Document 1:
"MATLAB is I guess part of the programming language that makes it very easy to write codes using matrices, to write code for numerical routines, to move data around, to plot data. And it's sort of an extremely easy to learn tool to use for implementing a lot of learning algorithms."
----------------------------------------------------------------------------------------------------
Document 2:
"And the student said, "Oh, it was the MATLAB." So for those of you that don't know MATLAB yet, I hope you do learn it. It's not hard, and we'll actually have a short MATLAB tutorial in one of the discussion sections for those of you that don't know it."Other Types of Retrieval
Basics
It’s worth noting that vectordb as not the only kind of tool to retrieve documents.
The
LangChainretriever abstraction includes other ways to retrieve documents, such as TF-IDF or SVM.
Load
Split
Retrieve with SVM and TF-IDF
SVM retriever
- Document(page_content=‘don't have a MATLAB license, for the purposes of this class, there's also — [inaudible] that down [inaudible] MATLAB — there' s also a software package called Octave you can download for free off the Internet. And it has somewhat fewer features than MATLAB, but it's free, and for the purposes of this class, it will work for just about . actually I, well, so yeah, just a side comment for those of you that haven't seen before I guess, once a colleague of mine at a different university, not at , actually teaches another machine l earning course. He's taught it for many years. one day, he was in his office, and an old student of his from, lik e, ten years ago came his office and he said, “Oh, professo r, professor, thank you so much for your learning class. I learned so much from it. There's this stuff that I learned in your , and I now use every day. And it's help ed me make lots of money, and here's a of my big house.” my friend was very excited. He said, “W ow. That's great. I'm glad to hear this learning stuff was actually useful. So what was it that you learned? Was it regression? Was it the PCA? Was it the data ne tworks? What was it that you that was so helpful?” And the student said, “Oh, it was the MATLAB.” for those of you that don't know MATLAB yet, I hope you do learn it. It's not hard,’, metadata={})
TFIDF retriever
- Document(page_content=“Saxena and Min Sun here did, wh ich is given an image like this, right? This is actually a taken of the Stanford campus. You can apply that sort of cl ustering algorithm and the picture into regions. Let me actually blow that up so that you can see it more . Okay. So in the middle, you see the lines sort of groupi ng the image together, the image into [inaudible] regions. what Ashutosh and Min did was they then applied the learning algorithm to say can take this clustering and us e it to build a 3D model of the world? And so using the , they then had a lear ning algorithm try to learn what the 3D structure of the looks like so that they could come up with a 3D model that you can sort of fly , okay? Although many people used to th ink it’s not possible to take a single and build a 3D model, but using a lear ning algorithm and that sort of clustering is the first step. They were able to. ’ll just show you one more example. I like this because it’s a picture of Stanford with our Stanford campus. So again, taking th e same sort of clustering algorithms, taking same sort of unsupervised learning algor ithm, you can group the pixels into different . And using that as a pre-processing step, they eventually built this sort of 3D model of Stanford campus in a single picture. You can sort of walk into the ceiling, look”, metadata={})
Acknowledgments
- This tutorial is mainly based on the excellent course “LangChain: Chat with Your DataI” provided by Harrison Chase from LangChain and Andrew Ng from DeepLearning.AI.
What’s next?
Congratulations! You have completed this tutorial 👍
Next, you may want to go back to the lab’s website
Jan Kirenz
Vectorstore Retrieval LangChain Tutorial 4 Jan Kirenz