)
print("Score from 1 to 5: ", evaluation["score"])
print("Reasoning: ", evaluation["reasoning"])
Score from 1 to 5:  3
Reasoning:  i. Is the final answer helpful?
Yes, the final answer is helpful as it provides an approximate number of Eiffel Towers needed to cover the US from coast to coast.
ii. Does the AI language use a logical sequence of tools to answer the question?
No, the AI language model does not use a logical sequence of tools. It directly uses the Calculator tool without first using the Search or Lookup tools to find the necessary information (length of the Eiffel Tower and distance from coast to coast in the US).
iii. Does the AI language model use the tools in a helpful way?
The AI language model uses the Calculator tool in a helpful way to perform the calculation, but it should have used the Search or Lookup tools first to find the required information.
iv. Does the AI language model use too many steps to answer the question?
No, the AI language model does not use too many steps. However, it repeats the same step twice, which is unnecessary.
v. Are the appropriate tools used to answer the question?
Not entirely. The AI language model should have used the Search or Lookup tools to find the required information before using the Calculator tool.
Given the above evaluation, the AI language model's performance can be scored as follows:
That also sounds about right. In conclusion, the TrajectoryEvalChain allows us to use GPT-4 to score both our agent’s outputs and tool use in addition to giving us the reasoning behind the evaluation.
previous
Data Augmented Question Answering
next
Using Hugging Face Datasets
 Contents
  
Setup
Testing the Agent
Evaluating the Agent
By Harrison Chase
rtdocs_stable/api.python.langchain.com/en/stable/use_cases/evaluation/generic_agent_evaluation.html