.ipynb
.pdf
SKLearnVectorStore
 Contents 
Basic usage
Load a sample document corpus
Create the SKLearnVectorStore, index the document corpus and run a sample query
Saving and loading a vector store
Clean-up
SKLearnVectorStore#
scikit-learn is an open source collection of machine learning algorithms, including some implementations of the k nearest neighbors. SKLearnVectorStore wraps this implementation and adds the possibility to persist the vector store in json, bson (binary json) or Apache Parquet format.
This notebook shows how to use the SKLearnVectorStore vector database.
%pip install scikit-learn
# # if you plan to use bson serialization, install also:
# %pip install bson
# # if you plan to use parquet serialization, install also:
%pip install pandas pyarrow
To use OpenAI embeddings, you will need an OpenAI key. You can get one at https://platform.openai.com/account/api-keys or feel free to use any other embeddings.
import os
from getpass import getpass
os.environ['OPENAI_API_KEY'] = getpass('Enter your OpenAI key:')
Basic usage#
Load a sample document corpus#
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import SKLearnVectorStore
from langchain.document_loaders import TextLoader
loader = TextLoader('../../../state_of_the_union.txt')
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(documents)
embeddings = OpenAIEmbeddings()
Create the SKLearnVectorStore, index the document corpus and run a sample query#
import tempfile
persist_path = os.path.join(tempfile.gettempdir(), 'union.parquet')
rtdocs_stable/api.python.langchain.com/en/stable/modules/indexes/vectorstores/examples/sklearn.html