Improving the Unblu Conversational Bot’s performance

Once you’ve deployed your prediction model and visitors start interacting with it, you’ll want to check how well the Unblu Conversational Bot (UCB) is performing. You’ll also want to use any insights you gain to improve the UCB’s ability to interact with your visitors.

Reviewing a delivery’s performance

You can configure the UCB to create transcripts of the conversations it takes part in.

Depending on what you use the UCB for, the transcripts may contain personally identifiable information (PII). If that’s the case, you must ensure that only employees cleared to see PII have access to the transcripts.

Conversation transcripts are accessible from the Transcript tab in the Bot Dialog Builder (BDB). By default, the tab lists the transcripts of conversations that took place on the current day.

To display the search options, click the Search bar at the top of the tab. Enter your search criteria and click the Search button to display the conversations that match the criteria you entered.

Expanded Transcript search bar

Matching conversations are displayed on the left beneath the search options. For each transcript, you see the last message sent to the conversation. The conversation whose transcript is currently displayed has a red border around its entry in the list of conversations on the left.

To view the transcript of a conversation, click the conversation. The transcript is displayed on the right. To the right of the transcript, you see metadata about the conversation.

If a conversation hasn’t ended yet, and the last message was sent by the visitor, the conversation sports a green badge labeled live. There’s a brief delay between the visitor sending a message and the message appearing in the transcripts. Also, you must refresh the view of the Transcript tab manually to see changes.

If there are more transcripts of conversations that match your search criteria, you see two buttons labeled Newer transcripts and Older transcripts, respectively, beneath the list of transcripts.

Adding utterances from conversations to intents

When you review conversation transcripts, you’ll no doubt come across utterances that could (and should) be assigned to a specific intent.

For example, suppose you trained the UCB to assign questions about opening a current account to a particular intent. When you review the transcripts, you notice that a significant proportion of visitors inquire about opening a checking account. Since your organization doesn’t offer checking accounts, you think it would be best to direct these enquiries to the same intent as those regarding current accounts.

One way to accomplish this would be to open the intent in question in the Intents tab and write new training phrases that use the term "checking account." An alternative approach is to use the Training tab:

Figure 1. Training tab in the BDB

On the left, you see a list of the utterances that the UCB was unable to assign to an intent. Each entry in the list shows the utterance itself, the date and time it was recorded, and the language of the conversation it was recorded in.

You can filter the list of utterances according to three criteria:

To search for utterances containing the phrase you’re interested in ("checking account" in the example above), enter it in the Containing text field and hit Enter. Note that the search looks for the text you entered as is. If you enter, say, "checking account," it doesn’t return utterances that contain "checking or current account."
You can limit the list of utterances to a particular language with the Language dropdown. If you don’t, the list contains utterances in all languages.
By default, the list displays utterances whose state is Open, that is, they appear in ongoing conversations. To view utterances from conversations that have ended, select Backlog from the State dropdown.

Each utterance has two or three buttons:

The Show conversation button opens the transcript of the conversation in a modal dialog. This lets you check how the utterance was used in the conversation.
The Ignore this utterance button removes the utterance from the search results without taking any further action.
Utterances in the Open state have a third button, Add to backlog , to transfer the utterance to the backlog immediately.

When you find an utterance you want to use as a training phrase for a future delivery, you can add it to the appropriate intent by first clicking the utterance to select it and then clicking the Assign button of the intent you want to assign it to.

You can filter the list of intents on the right of the Transcript tab by domain. You can also search for words and phrases in the name of the intent, in its training phrases, and in the answers it provides. Enter a word or phrase and hit Enter to update the list of intents. The search looks for the text you entered as is. If you enter, say, "open account," it doesn’t return intents that contain "open an account."

When you click the Assign button, the utterance is added to the intent’s training phrases in the language of the conversation, as displayed beneath the utterance. Make sure that this is the language you want to assign the utterance to! Maybe a visitor asked a question about checking accounts in English on the French version of your website. Adding the utterance to your French training phrases, however, is unlikely to be what you intended.