.md
.pdf
Prediction Guard
 Contents 
Installation and Setup
LLM
Example
Basic usage of the controlled or guarded LLM:
Basic LLM Chaining with the Prediction Guard:
Prediction Guard#
Prediction Guard gives a quick and easy access to state-of-the-art open and closed access LLMs, without  needing to spend days and weeks figuring out all of the implementation details, managing a bunch of different API specs, and setting up the infrastructure for model deployments.
Installation and Setup#
Install the Python SDK:
pip install predictionguard
Get an Prediction Guard access token (as described here) and set it as an environment variable (PREDICTIONGUARD_TOKEN)
LLM#
from langchain.llms import PredictionGuard
Example#
You can provide the name of the Prediction Guard model as an argument when initializing the LLM:
pgllm = PredictionGuard(model="MPT-7B-Instruct")
You can also provide your access token directly as an argument:
pgllm = PredictionGuard(model="MPT-7B-Instruct", token="<your access token>")
Also, you can provide an “output” argument that is used to structure/ control the output of the LLM:
pgllm = PredictionGuard(model="MPT-7B-Instruct", output={"type": "boolean"})
Basic usage of the controlled or guarded LLM:#
import os
import predictionguard as pg
from langchain.llms import PredictionGuard
from langchain import PromptTemplate, LLMChain
# Your Prediction Guard API key. Get one at predictionguard.com
os.environ["PREDICTIONGUARD_TOKEN"] = "<your Prediction Guard access token>"
# Define a prompt template
template = """Respond to the following query based on the context.
rtdocs_stable/api.python.langchain.com/en/stable/integrations/predictionguard.html