The Performance Summary page in your Freddy AI Agent's Analyze tab is a key tool for understanding its engagement metrics and performance outcomes. The page has two sections - Overview and Conversation Sources. This article will explain these sections, what the key metrics mean, and how to use them to optimize your AI Agent's performance.

Overview Section

At the top of the performance summary page, you’ll find a snapshot of your AI Agent’s key performance metrics over the last 7 days:

  • Conversations: The total number of conversations initiated with the AI Agent during this period. This gives insight into how frequently users interact with your AI Agent.

  • Resolution Rate: The percentage of conversations that were resolved by the AI Agent without the need for human intervention. A higher resolution rate typically indicates that the AI Agent successfully handles user queries.

  • Agent Transfer Rate: The percentage of conversations transferred to a live agent after the AI Agent interaction. A high transfer rate may suggest that users' questions need to be simpler for the AI Agent or that there is room for improvement in AI Agent knowledge sources.

To get more insights, go to the detailed analytics report “Freddy AI Agent performance report” in the AI Agent analytics section.

Note: The metrics might take up to 30 minutes to update.

Conversation Sources

Below the overview, you will find a detailed breakdown of the content that the AI Agent is using to respond to conversations. Each row corresponds to a specific knowledge source, such as:

  • Files: Documents like PDFs from which the AI Agent can extract information.

  • URLs: External web links provided to the AI Agent as a knowledge source.

  • FAQs: FAQs or solution articles in your Freshchat knowledge base.

  • Q&As: Predefined question-and-answer pairs that are configured within the AI Agent.

For each content source, the following details are provided:

  • Title: The name or URL of the source.

  • Source Type: Whether the content is derived from a URL, file, or Q&A.

  • Sent: The number of times this content has been sent in a conversation.

  • Helpful: The number of users who marked this response as helpful.

  • Not Helpful: The number of users who marked this response as unhelpful.

Note: You can filter the list by source type and search for a specific source.

Actionable Insights to improve your AI Agent performance

Ensure the content in your Files, URLs, Q&A pairs, and FAQs is clear, concise, and relevant to user queries. Regularly updating and expanding your knowledge base with fresh content is critical to improving resolution rates and reducing agent transfers.

  • Low Helpful Marks: If a piece of content has been sent frequently but has received few or no "Helpful" marks, it may indicate that the content is irrelevant or easy to understand. Review the content to ensure it effectively addresses the users’ needs.

  • High Agent Transfer Rate: If your agent transfer rate is high, investigate whether the AI Agent encounters questions it hasn’t been trained to answer. You may need to expand the AI Agent's knowledge base or refine its training to improve performance.

  • Underutilized Content: If URLs, files, or Q&A pairs are rarely used, ensure the AI Agent is properly configured to pull relevant information. Check whether these resources are needed and whether the AI Agent is correctly referencing them.

  • Focus on Resolution Rate: Aim to increase the resolution rate by providing more comprehensive training for the AI Agent. Incorporating additional content from URLs, files, FAQs, and Q&A can help the AI Agent respond to more user queries independently.