Contact usRequest a demo

Creating deliveries

Once you’ve created and configured your intents, you must train the prediction model to determine which intent the Unblu Conversational Bot (UCB) should use to reply to a visitor.

A trained prediction model is referred to as a delivery, and you train the model on the Delivery tab of the Bot Dialog Builder (BDB).

Training only takes into account productive intents. Before you start training the prediction model, make sure that all the intents you want to include are in the Productive state.

To start the training:

  1. Open the Delivery tab and select the language you want to train the prediction model for. (You must train each language separately.)

  2. Enter a name for the delivery in the field Create new delivery and click Create.

    If an intent has fewer than twenty training phrases, a modal dialog is displayed pointing out that this can have a negative impact on the training. Similarly, a modal dialog appears if an intent is assigned an invalid custom action. To ignore the warning, click Continue in the modal dialog, otherwise click Cancel and rectify the issue.

Once training starts, the delivery appears in the All deliveries list:

*All deliveries* list with training running
Figure 1. All deliveries list with training running

The value in the delivery’s State column is updated as the training session progresses until it displays the state COMPLETED.

Validation deliveries and full deliveries

In the All deliveries list, there’s a column with the heading Type . For each deliviery, this column contains either Validation delivery or Full delivery.

  • Validation deliveries only use a fraction (roughly 70%) of the productive intents' training phrases to train the prediction model. The remaining training phrases are used as test data for the prediction model you trained.

    Any delivery you create as outlined abovce is of the type Validation delivery.

  • Full deliveries are trained using all the intents' training phrases. Only full deliveries are deployable.

Before you create a full delivery, review the validation delivery you created.

Reviewing a delivery

Clicking the name of a delivery opens it details on a page with several tabs.

The first three tabs contain general information about the delivery:

  • The Summary tab provides basic information about the delivery, such as its state and the language the prediction model was trained on.

    You should double-check that you trained the model on the right language.

    The summary also includes the F1 score of the trained prediction model, which gives you a first idea of the precision and recall of the delivery.

  • The Intent changes tab tells you which intents were added or removed compared to the currently deployed delivery. It also shows which intents saw the greatest change in their F1 score.

  • The Training phrase changes tab lists the training phrases that were added or removed compared to the currently deployed delivery.

The final four tabs contain more detailed information on the prediction model’s performance:

  • Correct predictions below threshold lists utterances where the model predicted the correct intent, but its confidence level was below the 85% threshold.

    Intents selected with a confidence level lower than the threshold aren’t used to generate answers in chats.

  • False predictions above threshold lists utterances where the model made a prediction above the 85% threshold, but it predicted the wrong intent.

  • The Intent confusion matrix shows you the confusion matrix of the delivery’s prediction model. It makes it easier to identify intents that are commonly mislabeled as one another.

  • The Intent prediction confidence distribution shows you how many utterances the model assigned to intents correctly or incorrectly, and the confidence level with which it made those assignments.

Deploying a delivery

When you’re satisfied with the performance of your model in a validation delivery, you can prepare it for deployment:

  1. Click the name of the validation delivery to open its details.

  2. Click the Train without split button at the bottom of the Summary tab. This creates a full delivery, where the prediction model is trained on all the training phrases.

  3. Review the results of the full delivery to make sure the training phrases excluded from the validation delivery haven’t had a negative effect on the prediction model’s performance.

  4. If you’re happy with the delivery’s results, click Go live to deploy the model and make it available to your visitors.