.ipynb
.pdf
OpenSearch
 Contents 
Installation
similarity_search using Approximate k-NN
similarity_search using Script Scoring
similarity_search using Painless Scripting
Using a preexisting OpenSearch instance
OpenSearch#
OpenSearch is a scalable, flexible, and extensible open-source software suite for search, analytics, and observability applications licensed under Apache 2.0. OpenSearch is a distributed search and analytics engine based on Apache Lucene.
This notebook shows how to use functionality related to the OpenSearch database.
To run, you should have an OpenSearch instance up and running: see here for an easy Docker installation.
similarity_search by default performs the Approximate k-NN Search which uses one of the several algorithms like lucene, nmslib, faiss recommended for
large datasets. To perform brute force search we have other search methods known as Script Scoring and Painless Scripting.
Check this for more details.
Installation#
Install the Python client.
!pip install opensearch-py
We want to use OpenAIEmbeddings so we have to get the OpenAI API Key.
import os
import getpass
os.environ['OPENAI_API_KEY'] = getpass.getpass('OpenAI API Key:')
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import OpenSearchVectorSearch
from langchain.document_loaders import TextLoader
from langchain.document_loaders import TextLoader
loader = TextLoader('../../../state_of_the_union.txt')
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(documents)
embeddings = OpenAIEmbeddings()
similarity_search using Approximate k-NN#
rtdocs_stable/api.python.langchain.com/en/stable/modules/indexes/vectorstores/examples/opensearch.html