What is Conversation-based reporting?

What are the Mentions and Conversations?

Customer AI is designed to aggregate and analyze customer feedback across various channels such as support platforms, review sites, surveying tools. Because of that, we created a standard data structure that makes sense for all of these channels.

We have noticed that many forms of customer communication take the form of a thread with multiple child items. A support conversation contains a back-and-forth between a customer and an agent. A survey response often contains multiple answers and so on.

Therefore, in Customer AI we mapped data from all these sources into Conversations and Mentions:

• Mentions are the atomic pieces of customer communication such as individual support ticket comments or reviews. 
• Conversations are structures that contain related Mentions. 

Please see the example below for a visualization:

How are the metrics calculated?

All metrics and reports in Customer AI are calculated in terms of unique conversations. Here are some examples:

• The volume on the Dashboard screen is calculated in terms of how many unique conversations received mentions in a given period
• The volume on the Topic view is calculated in terms of how many unique conversations the topic keywords have appeared in a given period

Mentions are grouped by conversation

When multiple mentions from the same conversation match your topic keywords, they will be grouped and counted as a single conversation. This allows you to see better gauge the context behind each case.

In order to deepen your insights, you may also view metadata associated with Users and Conversations directly from the message preview.

Check out the User Metadata by clicking on the User Icon circled in the picture below:

You’ll be able to see the Conversation Metadata by clicking on the three lines to the right of the User Icon.

Example mappings

We sync customer communication from a variety of third-party systems. Below are some examples of how data from different data sources map to Customer AI's Conversations and Mentions.

Switch to Conversation-based reporting

Up until October 2020, all metrics in Customer AI were counted in terms of individual Mentions. We have since changed our reporting methodology to Conversation-based reporting to improve the accuracy and usability of our metrics. Here is the reasoning behind this change.

Minimize the possibility of over-reporting

Let's say you are looking to track bug reports in your Intercom conversations.

With the old Mention-based reporting, every single mention containing any of the matching topic keywords would be counted towards the topic volume. If the user repeats any of these keywords as they clarify their request, it would be counted as multiple Mentions even though the user could be talking about the same issue.

Conversation-based reporting solves this problem by aggregating any matching mentions in a topic under one Conversation instance. Regardless of how many times the matching keywords appear in a given conversation, it will always be counted as one instance.

Better alignment with support-based metrics

Reporting in many popular support tools is based around Conversations or Tickets, not individual messages. 

With Conversation-based metrics, you will be able to compare "apples for apples" and easily relate your support-based metrics with reporting in Customer AI.

Works seamlessly with single-item sources

You might have single-item feedback in your Customer AI such as reviews or NPS verbatims and wonder how Conversation-based reporting impacts on it. There is no change because any matches will still count as a single instance, just like before.

For example, if you receive a mention of "bug" in an NPS survey and two mentions in the same support conversation, it will count as two Conversations - one for the NPS survey and one for the support conversation.