.ipynb
.pdf
Self-querying
 Contents 
Creating a Pinecone index
Creating our self-querying retriever
Testing it out
Filter k
Self-querying#
In the notebook we’ll demo the SelfQueryRetriever, which, as the name suggests, has the ability to query itself. Specifically, given any natural language query, the retriever uses a query-constructing LLM chain to write a structured query and then applies that structured query to it’s underlying VectorStore. This allows the retriever to not only use the user-input query for semantic similarity comparison with the contents of stored documented, but to also extract filters from the user query on the metadata of stored documents and to execute those filters.
Creating a Pinecone index#
First we’ll want to create a Pinecone VectorStore and seed it with some data. We’ve created a small demo set of documents that contain summaries of movies.
To use Pinecone, you to have pinecone package installed and you must have an API key and an Environment. Here are the installation instructions.
NOTE: The self-query retriever requires you to have lark package installed.
# !pip install lark
#!pip install pinecone-client
import os
import pinecone
pinecone.init(api_key=os.environ["PINECONE_API_KEY"], environment=os.environ["PINECONE_ENV"])
/Users/harrisonchase/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/pinecone/index.py:4: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)
  from tqdm.autonotebook import tqdm
from langchain.schema import Document
from langchain.embeddings.openai import OpenAIEmbeddings
rtdocs_stable/api.python.langchain.com/en/stable/modules/indexes/retrievers/examples/self_query.html