.ipynb
.pdf
Generic Agent Evaluation
 Contents 
Setup
Testing the Agent
Evaluating the Agent
Generic Agent Evaluation#
Good evaluation is key for quickly iterating on your agent’s prompts and tools. Here we provide an example of how to use the TrajectoryEvalChain to evaluate your agent.
Setup#
Let’s start by defining our agent.
from langchain import Wikipedia
from langchain.chat_models import ChatOpenAI
from langchain.agents import initialize_agent, Tool
from langchain.agents import AgentType
from langchain.agents.react.base import DocstoreExplorer
from langchain.memory import ConversationBufferMemory
from langchain import LLMMathChain
from langchain.llms import OpenAI
from langchain import SerpAPIWrapper
docstore = DocstoreExplorer(Wikipedia())
math_llm = OpenAI(temperature=0)
llm_math_chain = LLMMathChain(llm=math_llm, verbose=True)
search = SerpAPIWrapper()
tools = [
    Tool(
        name="Search",
        func=docstore.search,
        description="useful for when you need to ask with search",
    ),
    Tool(
        name="Lookup",
        func=docstore.lookup,
        description="useful for when you need to ask with lookup",
    ),
    Tool(
        name="Calculator",
        func=llm_math_chain.run,
        description="useful for doing calculations",
    ),
    Tool(
        name="Search the Web (SerpAPI)",
        func=search.run,
        description="useful for when you need to answer questions about current events",
    ),
]
memory = ConversationBufferMemory(
    memory_key="chat_history", return_messages=True, output_key="output"
)
rtdocs_stable/api.python.langchain.com/en/stable/use_cases/evaluation/generic_agent_evaluation.html