Report

Configuring Custom Dashboards:

Custom Dashboards are created through saved queries from Data Explorer and used to track specific metrics relevant to your business. The dashboard views, filters, and cards are fully customizable to help automatically monitor your KPIs.

This short video walks through step-by-step how to configure and customize your dashboards.

Using Data Explorer Effectively:

The Data Explorer is an interactive visual tool for exploratory data analysis, feature engineering, and reporting of your bot’s conversational data. These insights can help inform new intents and features for your bot and each query can be saved and used to create a Custom Dashboard

If you have any questions, comments, or suggestions, please feel free to contact us.

Use Cases

The following are a few popular use cases to explain how you might use Data Explorer to answer common questions regarding your data:

Demo video on using the Data Explorer to identify which intents and messages types are being sent by your users

• Use Case 1: View Breakdown of First User Message based on Message Type & Intent
  • View: Messages
  • Filter: First Message In Session
  • Breakdown: Incoming, Message Type, Intent
  • Be able to identify which intents and message types are the first to be sent to your bot by users
• Use Case 2: View Intent and user utterance mapping based on Keywords (e.g. Whenever customers are asking for “help”)
  • View: Messages or Users
  • Filter: Text containing “keyword”
  • Breakdown: Intent, Text
  • Be able to easily identify Mishandled/Unhandled intents that should be Handled correctly
  • Example: Filtering for all messages containing the keyword “Help”
    • Identify which “help” messages are mapping to incorrect and unhandled intents to use as training phrases
• Use Case 3: View Intent Performance by Intent Confidence
  • View:
    • fm.metadata/intentConfidence - avg
  • Filter: Incoming
  • Breakdown: Intent, Users, Sessions
  • Be able to identify which intents have the poorest performing confidence and the respective users and sessions.

Configuration

• View - Data Explorer allows you to select a subset of your conversational data for viewing or analysis. These 5 parameters are different ways of aggregating subsets of your JSON payload to explore your data further.
  • Messages - Count of unique messages
  • Users - Count of unique users across all messages
  • Intents - Count of all unique incoming/outgoing intents
  • General Field - Specific fields associated with parameters related to our platform. You can select to view from any of the following general fields and apply one of the following aggregate functions: count, count_distinct, sum, avg, min, and max
    • platformUserID - the ID for your user on a particular bot platform
    • sessionID - the ID for a distinct conversation session
    • Sentiment - the corresponding sentiment value for a session
    • Datetime - the timestamp for each message
  • JSON Field - Specific fields associated with the JSON payload. This allows for specific/custom fields that you pass through your JSON payload (e.g. intentConfidence). You can select to view any specific category from your JSON payload and apply one of the following aggregate functions: count, count_distinct, sum, and avg.
    • Below is an example of selecting to view the Intent Confidence from the JSON metadata
    • Example - fm.metadata/intentConfidence count as IntentConfidence
• Filter - An initial filter of your bot’s data before viewing the output. A filter selects a subset of your bot’s data determined by specific constraints. You can find the list of possible filter constraints in the Glossary section below.
• Breakdown - A breakdown classifies the resulting data into segments based on the selected category. You can find the list of possible breakdowns in the Glossary section below.

Output

• Date Range - You can select the time range for the conversational history of your bot
• Linear vs Logarithmic - You can select to view the time series graph as either linear or logarithmic
• Output/Graph Type - The following are the different graph and chart views available for Data Explorer:
  • Line
  • Stacked Line
  • Bar
  • Stacked Bar
  • Table
  • Condensed Table
  • Pie Chart
  • KPI

Settings

• Data Explorer let’s you save and download your unique DE configurations.
  • Load - load a previously saved DE configuration
  • Save - saves the current configuration
  • Save As New - saves as a new configuration
  • New - resets the configuration (pretty much a refresh)
  • Download CSV - To download as a CSV file click the ellipses … in the upper right hand corner of the report

Tips for Using Flows:

The Flows Report allows you to visualize the unique behavior or conversational journey of your bot --– the conversation paths users follow. It is represented by a sankey diagram where the width of each individual conversation path is directly proportional to the flow rate.

Use Cases

Some primary use cases are tracking steps that led to a bot failure, tracking initial conversation steps and measuring general conversation trends and patterns.

This short demo video walks through configuring a custom Flow to measure the unique user paths leading to escalation.

• Use Case 1: Measure how users make it from point A to point B in the conversation
  • Use Conversation Paths to identify patterns/trends at a high level, then transition to Flows to be able to answer specific questions identified from the Conversation Paths
• Use Case 2: Tracking all steps that led to an NotHandled Intent
  • 1 - Steps before A
  • Filter Definition: where Intent in NotHandled
  • 1 - Step after A
  • Be able to see which intents directly lead to your users triggering the NotHandled intent
• Use Case 3: Tracking the first steps in the conversation
  • 0 - Steps before A
  • Filter Definition: where Incoming incoming AND First Message In Session is True
  • 2 - Steps after A
  • Be able to track all intents sent from your users for the first 3 steps and the conversation path for each. Identify the transcripts for the NotHandled intent and identify ways to improve bot responses

Configuration

• Steps - the selected stages of your bot’s conversation history. Each Step is essentially applying a specific constraint to your messages to identify a stage of a journey you may want to track.
  • Filter Definition - A filter definition adds a Step where a filter selects a subset of your bot’s data determined by specific constraints. You can find the list of possible step filter definitions in the Glossary section below.
  • Steps Before/After - You can select a certain # of immediate steps before and steps after that step to track. A step before shows all connected events that occur exactly 1 step before the selected filtered step. A step after shows all connected events that occur exactly 1 step after the selected filtered step.
• Filter - An aggregate/global filtering of the message data contained within the “Steps” you defined previously. You can find the list of possible filters in the Glossary section below.

Output

The output is a flow (sankey diagram) showing the specific behavior/journey of your bot & users based on the steps & filters you defined.

• Node - Each node in the graph is an intent, which represents all of the messages matching to that intent for a given step
• Paths - Hovering over a specific path allows to you see the flow rate for the two connected nodes as well as the number of messages that correspond to that given path
• Transcripts - You can access transcripts for an individual conversation path by clicking the ellipses located at each intent.
• Adding Steps - You can add a step by clicking the + sign located near the top corners of the output
• Removing Steps - You can remove a step by clicking on an individual arrow located next to the Step.
• Accessing Transcripts - By clicking on the ellipses (...) to the right of each node and clicking on “Transcripts” you can access the associated transcripts for that given path.

Glossary