.ipynb
.pdf
LanceDB
LanceDB#
LanceDB is an open-source database for vector-search built with persistent storage, which greatly simplifies retrevial, filtering and management of embeddings. Fully open source.
This notebook shows how to use functionality related to the LanceDB vector database based on the Lance data format.
!pip install lancedb
We want to use OpenAIEmbeddings so we have to get the OpenAI API Key.
import os
import getpass
os.environ['OPENAI_API_KEY'] = getpass.getpass('OpenAI API Key:')
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import LanceDB
from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter
loader = TextLoader('../../../state_of_the_union.txt')
documents = loader.load()
documents = CharacterTextSplitter().split_documents(documents)
embeddings = OpenAIEmbeddings()
import lancedb
db = lancedb.connect('/tmp/lancedb')
table = db.create_table("my_table", data=[
    {"vector": embeddings.embed_query("Hello World"), "text": "Hello World", "id": "1"}
], mode="overwrite")
docsearch = LanceDB.from_documents(documents, embeddings, connection=table)
query = "What did the president say about Ketanji Brown Jackson"
docs = docsearch.similarity_search(query)
print(docs[0].page_content)
They were responding to a 9-1-1 call when a man shot and killed them with a stolen gun. 
Officer Mora was 27 years old. 
Officer Rivera was 22. 
Both Dominican Americans who’d grown up on the same streets they later chose to patrol as police officers.
rtdocs_stable/api.python.langchain.com/en/stable/modules/indexes/vectorstores/examples/lancedb.html