.ipynb
.pdf
Cohere Reranker
 Contents 
Set up the base vector store retriever
Doing reranking with CohereRerank
Cohere Reranker#
Cohere is a Canadian startup that provides natural language processing models that help companies improve human-machine interactions.
This notebook shows how to use Cohere’s rerank endpoint in a retriever. This builds on top of ideas in the ContextualCompressionRetriever.
#!pip install cohere
#!pip install faiss
# OR  (depending on Python version)
#!pip install faiss-cpu
# get a new token: https://dashboard.cohere.ai/
import os
import getpass
os.environ['COHERE_API_KEY'] = getpass.getpass('Cohere API Key:')
os.environ['OPENAI_API_KEY'] = getpass.getpass('OpenAI API Key:')
# Helper function for printing docs
def pretty_print_docs(docs):
    print(f"\n{'-' * 100}\n".join([f"Document {i+1}:\n\n" + d.page_content for i, d in enumerate(docs)]))
Set up the base vector store retriever#
Let’s start by initializing a simple vector store retriever and storing the 2023 State of the Union speech (in chunks). We can set up the retriever to retrieve a high number (20) of docs.
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings import OpenAIEmbeddings
from langchain.document_loaders import TextLoader
from langchain.vectorstores import FAISS
documents = TextLoader('../../../state_of_the_union.txt').load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=100)
texts = text_splitter.split_documents(documents)
rtdocs_stable/api.python.langchain.com/en/stable/modules/indexes/retrievers/examples/cohere-reranker.html