Note: This feature is currently in Beta. To enable it, contact your Customer Success Manager.

The Performance Summary page in your bot's Analyze tab is a key tool for understanding its engagement metrics and performance outcomes. The page has two sections - Overview and Conversation Sources. This article will explain these sections, what the key metrics mean, and how to use them to optimize your bot's performance.

Overview Section

At the top of the performance summary page, you’ll find a snapshot of your bot’s key performance metrics over the last 7 days:

  • Conversations: The total number of conversations initiated with the bot during this period. This gives insight into how frequently users interact with your bot.

  • Resolution Rate: The percentage of conversations that were resolved by the bot without the need for human intervention. A higher resolution rate typically indicates that the bot successfully handles user queries.

  • Agent Transfer Rate: The percentage of conversations transferred to a live agent after the bot interaction. A high transfer rate may suggest that users' questions need to be simpler for the bot or that there is room for improvement in bot knowledge sources.

To get more insights, go to the detailed analytics report “Freddy AI Agent performance report” in the bot analytics section.

Note: The metrics might take up to 30 minutes to update.

Conversation Sources

Below the overview, you will find a detailed breakdown of the content that the bot is using to respond to conversations. Each row corresponds to a specific knowledge source, such as:

  • Files: Documents like PDFs from which the bot can extract information.

  • URLs: External web links provided to the bot as a knowledge source.

  • FAQs: FAQs or solution articles in your Freshchat knowledge base.

  • Q&As: Predefined question-and-answer pairs that are configured within the Bot.

For each content source, the following details are provided:

  • Title: The name or URL of the source.

  • Source Type: Whether the content is derived from a URL, file, or Q&A.

  • Sent: The number of times this content has been sent in a conversation.

  • Helpful: The number of users who marked this response as helpful.

  • Not Helpful: The number of users who marked this response as unhelpful.

Note: You can filter the list by source type and search for a specific source.

Actionable Insights to improve your Bot performance

Ensure the content in your Files, URLs, Q&A pairs, and FAQs is clear, concise, and relevant to user queries. Regularly updating and expanding your knowledge base with fresh content is critical to improving resolution rates and reducing agent transfers.

  • Low Helpful Marks: If a piece of content has been sent frequently but has received few or no "Helpful" marks, it may indicate that the content is irrelevant or easy to understand. Review the content to ensure it effectively addresses the users’ needs.

  • High Agent Transfer Rate: If your agent transfer rate is high, investigate whether the bot encounters questions it hasn’t been trained to answer. You may need to expand the bot's knowledge base or refine its training to improve performance.

  • Underutilized Content: If URLs, files, or Q&A pairs are rarely used, ensure the bot is properly configured to pull relevant information. Check whether these resources are needed and whether the bot is correctly referencing them.

  • Focus on Resolution Rate: Aim to increase the resolution rate by providing more comprehensive training for the bot. Incorporating additional content from URLs, files, FAQs, and Q&A can help the bot respond to more user queries independently.