The Implementation Handling of Multiple Microsoft Chatbot Modules
The Implementation Handling of Multiple Microsoft Chatbot Modules
2
What’s new for QnA Maker? 43
Rich Text Authoring 43
Role Based Access Control (RBAC) 45
Access control (IAM) 45
Roles 46
Add a role assignment 46
3
About the Author
4
Introduction
Recent trends show that chatbots are essential everywhere, whether it’s an
entertainment, medical, or knowledge-based website. Since this is an emerging
technology, developers all around the world will be wanting to know more
about this, specifically, what is happening in the background of chat bot
applications.
This book teaches you how to handle and implement multiple chatbot modules
in real-world scenarios.
There are important bot modules like QnA and Luis, which are key players in
the chatbot e2e (end-to-end) communication or conversation flow.
Throughout this book, you will learn to build and handle different types of
chatbot modules from beginning to end.
Target Audience
The book is mainly focusing on people who are interested in chatbot module
integration in real-time, such as students, teachers and programmers, with and
without knowledge of NLP and AI. A basic knowledge of programming and
cloud technologies, like Azure, is required. This will help to correlate situation
handling in the real-time applications. The language preference is ASP.NET, C#,
and Node.
Azure Subscription
A valid Azure subscription key is required for the creation of any of the services
in the Azure portal. We can create an Azure subscription in multiple ways,
either a free or paid service. Microsoft provides a free Azure subscription for
the first 12 months for a single credit card. Microsoft credits ₹14,500 INR or
200 USD for the first month in the azure portal and the popular services are
free for one year. For this free service activation, you need to register your
Azure account using your credit/debit card free of cost.
5
We can also activate Azure subscription through Microsoft Visual Studio. If you
have Visual Studio other than community edition then go to Visual Studio
website ( https://my.visualstudio.com/ ) and click on the Benefits menu and
activate Azure free $50 monthly credit in your Azure account. This will be
helpful for most of the paid service access in the Azure portal to develop an
application for testing purposes because we are using very limited credit and
once the free access is over, then all services will stop working. Remember that
every Azure service has been chargeable and the amount will be debited from
the monthly credited amount in azure portal. So, once it’s over then we won’t
be able to access any of the paid services in the particular month and need to
wait for the next month for the new credit.
Important Tips: We can delete unwanted services from the Azure portal to
avoid and save the Azure free credited cost. In this way, we can utilize the
Azure free credited amount for every month. Also try to disable application
insight or any other unwanted options while creating an app service,
resources, virtual machine, etc., in the Azure portal.
The pricing tier will be considered as a major concern when you become an
Azure developer because everything will be chargeable and nothing will be
free in the current infrastructure for commercial use. One of the main
preferences of a customer is cost reductions for the infrastructure setup and
we need to take care of that in a proper and efficient manner.
The Azure pricing calculator will help you to estimate the actual cost charges in
each and every service in the Azure portal in days, hours, and months.
● https://azure.microsoft.com/en-in/pricing/calculator/
6
Note: Select the region based on the customer need or nearest to customer
location; this will reduce the latency.
Cognitive Services
7
Note: Cognitive Services will be available at least 99.9% of the time and No SLA
is provided for the free tier.
What is LUIS?
Why LUIS?
The reason for using LUIS in the application Instead of choosing other platforms
is because it provides more features for artificial intelligence than other
platforms. The following are the important benefits for using LUIS in the
application.
8
● Continuously improve natural language models.
● We can give more accuracy for the utterances.
● Build an intelligent application with less time.
● AI expertise is not required for handling the LUIS application because all
are prebuilt AI.
● Easy to retrain and build an intelligent application.
● More machine learning features are available like ML entities, etc.
As per the latest Microsoft build 2020, Microsoft has announced new features
for Language Understanding (LUIS) in the application.
Step 1:
Go to Azure portal and click on "+" icon -> go to "AI + Machine Learning" ->
Click on "See all".
9
Step 2:
If you are not able to see LUIS in the “Cognitive Services” then click on the
"See more" link on the right side.
Step 3:
Once you are able to see Language Understanding in the “Cognitive Services”
list then go and click on it.
10
Step 4:
The following screen will appear once you click on the "Create" button in
"Language Understanding"
● Subscription: We can select our own Azure subscription for Language
Understanding and if you don’t have the subscription then create a new
one, either free or paid.
● Resource group: Group together all the LUIS related resources into one
place and this will be helpful for filtering and applying the permission level
(RBAC, LOCK) for the entire group in the LUIS application. So, we can
create a new resource group or choose from an existing one (We selected
our existing resource group as "luis-test").
● Authoring location: We can select our location of Authoring location. The
best thing is we can choose a location closest to our customer needs.
● Authoring pricing tier: We can choose the appropriate pricing tier as per
our needs.
● Prediction location: We can select our location of prediction. The best
thing is we can choose a location closest to our customer needs.
● Prediction pricing tier: As of now there are two pricing tiers available,
"FO" & "SO" and obviously "FO" is the free one and we can choose the
appropriate pricing tier as per our needs. When you are planning to do a
performance testing then choose “SO” as the priority one because it will
handle a greater number of hits per second otherwise choose “FO” more
learning purposes.
11
● Click on the "Review + Create" button and wait for the build success.
● Once the build has succeeded, then click on the "Dashboard" and we can
see "luis-cog-testing" is created in the All resources list. LUIS is ready for
use!
12
The uses of LUIS Endpoint Key
The endpoint key will be generated while creation of Authoring resources in
LUIS Cognitive services in Azure portal. These keys are used to access your
Cognitive Service API for doing all types of CRUD operations. Do not share your
keys with the public, rather store them securely using Azure Key Vault or any
secure way.
The “Keys and Endpoint” menu is part of the Resource Management section in
the LUIS Cognitive service in Azure portal. We can consider any of the keys for
accessing the Cognitive Service API and also regenerate the key as per your
needs.
We have already created the LUIS authoring resource in Azure and now we can
easily create LUIS model in Luis account. So, go to your LUIS account and create
a new LUIS App.
Output:
The app is successfully created in LUIS and by default it will contain one "Intent"
called "None". We will have a detailed discussion over the topics: intent,
utterances, entity.
14
What is Intent?
In the real world, we have used more numbers of intents in the Covid-19
pandemic situation because during the lockdown period we have used many
online services for purchase, booking, bill payments, etc. All these activities are
internally performing many kinds of tasks. These tasks are created as an intent
in the LUIS application for various purposes.
The prebuilt domain already has a set of intents and utterances for the
respective functional activities like each task has appropriate actions. The
15
following are the prebuilt domains available in the LUIS applications and all are
used for our real-time activities.
● Calendar
● Communication
● Email
● HomeAutomation
● Note
● Places
● RestaurantReservation
● ToDo
● Utilities
● Weather
● Web
We can add prebuilt domains by clicking on “Add domain” button and once
added the entity will be included in the respective LUIS application.
Note: We can easily remove the prebuilt domain entity from the LUIS application
by clicking on the “Remove domain” button which appears on the already added
prebuilt domain entity.
16
E.g. The following JSON response is the utterance of “What is the current
weather in Bangalore” in the LUIS application that we have created and also this
utterance matching top intent is part of the already-added prebuilt domain.
{
"query": "\"What is the current weather in Bangalore\"",
"prediction": {
"topIntent": "Weather.CheckWeatherValue",
"intents": {
"Weather.CheckWeatherValue": {
"score": 0.406864345
},
"Weather.GetWeatherAdvisory": {
"score": 0.06625718
},
"Weather.CheckWeatherTime": {
"score": 0.0321165435
},
"Weather.QueryWeather": {
"score": 0.009715379
}, "Weather.ChangeTemperatureUnit": {
"score": 0.008676363
},
"None": {
"score": 0.00400842959
}, },
"entities": {
"geographyV2": [
17
{
"value": "Bangalore",
"type": "city"
}
],
"$instance": {
"geographyV2": [
{
"type": "builtin.geographyV2.city",
"text": "Bangalore",
"startIndex": 32,
"length": 9,
"modelTypeId": 2,
"modelType": "Prebuilt Entity Extractor",
"recognitionSources": [
"model"
]
}
]
}
}
}
}
18
What are Utterances?
All the intent requires some kind of explanation, otherwise it will be meaningless
so utterances are the explanation or the detailed description of the intent. LUIS
will contain unique utterances across all the intents. When you try to add
duplicate utterances in another intent or the same intent then LUIS will
automatically remove the duplicate utterance from the application. This will
improve the consistency and intelligence of the NLP engine and give us more
accuracy in the LUIS application.
Note: “To train the LUIS application at least one utterance is required”
19
How to create an Utterance?
The entities are variables or parameters in the LUIS application which will pass
important information from the utterances that we have given. There are many
ways we can create entities for different activities in the LUIS application which
is done to improve the machine learning capabilities.
20
Types of entity creation
There are a few kinds of entity creation in the “Create new entity” option and
the following are the different types of entity creation.
Machine Learned entities are the new way of entity creation updated by
Microsoft on MS Build 2020:
“Machine learned entities are learned from context. Use an ML entity to identify
data that are not always well-formatted but have the same meaning. An ML
entity can be composed of smaller sub entities, each of which can have its own
properties.”
21
Without clicking on “Add structure” checkbox, create an entity in the Machine
Learned type, then it will create a simple entity similar to the older version of
LUIS.
22
The name itself describes it, as it is a part by part information into the object,
using a Machine learned entity to represent an object that has many parts. The
ML entity is made up of entities that form the whole object information. For
example, a ML entity called “MobileBillPayment” in an application and it can be
composed of four child entities that describe attributes of the bill to payment:
Plan, Network, State and Amount.
Note: Once you have created your entity you will not be able to rearrange
subentities.
The created Machine-Learned sub entities are manually mapped into the given
utterance “prepaid mobile 123 bill payment for airtel kerala”. The new graphical
representation is clearly displayed as the ML entity details and will give us all the
information about the utterance.
23
LIST
List entities are represented as a fixed entity and it describes the set of related
words mapped together or along with their synonyms. It clearly says that
synonymous is always representing the same meaningful words and it is
interchangeable. The list entity always returns the response for the exact text
match because a list entity isn't part of a machine-learned entity.
Click on the Entities button in the menu list for the respective LUIS application
and it will open a popup window with type of entity information. Select the List
entity radio button and Click on “Create” button.
24
Once created It will display the add list item option in the already created “State”
LIST entity. So, we can add any number of normalized values in the LIST entity
section and we have added “Kerala” as the normalized value and Synonyms as
“KER”, “KER”, etc. All the list items are part of the parent entity and there’s no
need to map or label LIST entities to utterances because this will create a
response based on the exact text match.
25
Here is the JSON response for LIST entity and it will automatically recognize the
LIST entity information for the given utterance “Where is ker”.
{
"query": "\"Where is ker\"",
"prediction": {
"topIntent": "None",
"intents": {
"None": {
"score": 0.9559727
},
"BillPayment": {
"score": 0.0251173135
},
"GasRefillBooking": {
"score": 0.0124031492
}
},
"entities": {
"State": [
[
"Kerala"
]
],
"$instance": {
"State": [
{
"type": "State",
26
"text": "ker",
"startIndex": 10,
"length": 3,
"modelTypeId": 5,
}
]
}
}
}
}
We have added “Circle” and “Network Circle” as the Role or named subtype of
State List entity and this will easily recognize Kerala as the network circle for the
respective activities.
E.g. For mobile prepaid or postpaid recharge we need to select the network
circle.
27
After successful building and training, go to the utterances for mapping the
“Role”. In the following example we have clicked on the “kerala” and It will open
a palette. Click on the “View in entity pane” list menu and it will open a tree
structure for all the entities in the LUIS application.
The “View in entity pane” will open the entity tree structure and click on the
State List Entity and then go to the bottom of the palette and select the role and
it will display the synonyms of the respective role added in the State List entity.
28
Regex
29
E.g. The Regex “^[0-9]+$” only accepts numbers.
The “BookingId” we can add as a global feature in any of the entities in the LUIS
application. This will be similar to what we have added to the phrase list in the
LUIS app. Click on “Add feature” and add a phrase list or already-created entity
in the respective intent.
30
Here is the json response of “What is the booking id 25678” utterances and it
will return accurate entity information with the help of Regex that we have
added.
{
"query": "\"What is the booking id 25678\"",
"prediction": {
"topIntent": "Weather.GetWeatherAdvisory",
"intents": {
"Weather.GetWeatherAdvisory": {
"score": 0.0530414023
},
"Weather.CheckWeatherTime": {
"score": 0.0437874235
},
"Weather.CheckWeatherValue": {
"score": 0.036190737
},
"Weather.QueryWeather": {
31
"score": 0.0283024088
},
"GasRefillBooking": {
"score": 0.0206228029
},
"None": {
"score": 0.0112512978
},
"BillPayment": {
"score": 0.0102754943
},
"Weather.ChangeTemperatureUnit": {
"score": 0.004960226
}
},
"entities": {
"dimension": [
{
"number": 25678,
"units": "Inch"
}
],
"$instance": {
"dimension": [
{
"type": "builtin.dimension",
"text": "25678\"",
"startIndex": 24,
32
"length": 6,
"modelTypeId": 2,
"modelType": "Prebuilt Entity Extractor",
"recognitionSources": [
"model"
]
}
]
}
}
}
}
Here is the json response of “What is the booking id “X25678” utterance and it
will never return accurate entity information because we have added the regex
pattern which will only accept numbers.
{
"query": "\"What is the booking id X25678\"",
"prediction": {
"topIntent": "Weather.GetWeatherAdvisory",
"intents": {
"Weather.GetWeatherAdvisory": {
"score": 0.0530414023
},
"Weather.CheckWeatherTime": {
"score": 0.0437874235
},
"Weather.CheckWeatherValue": {
"score": 0.036190737
33
},
"Weather.QueryWeather": {
"score": 0.0283024088
},
"GasRefillBooking": {
"score": 0.0206228029
},
"None": {
"score": 0.0112512978
},
"BillPayment": {
"score": 0.0102754943
},
"Weather.ChangeTemperatureUnit": {
"score": 0.004960226
}
},
"entities": {}
}
}
The prebuilt entity is when you add a built-in entity, its predictions will be
available to the entity mapped or labelled utterances. The following are the
currently available prebuilt entities in the LUIS application.
● age
● email
● keyPhrase
34
● money
● number
● ordinal
● ordinalV2
● percentage
● phonenumber
● url
The prebuilt domain entity already has a set of intents and utterances, etc for
the respective functional activities. The prebuilt domain models are part of
35
prebuilt domains. It is recommended to use models from the same domain
when possible since they work better together.
36
What is Pattern?
The LUIS pattern will give more intelligence to the utterances and
conversational flow in chat bot. This will improve the performance of NLP
engine and analysis more deeply for the utterances sub entities information.
All the entities in patterns are surrounded by curly brackets, {} also patterns
can include entities, and entities with roles.
Here is the example for patterns that we have added in the LUIS application. In
this “State” is the LIST entity and added inside curly brackets, {} because
patterns are following some kind of template mode.
Phrase lists are similar to the synonymous or the similar meaning of the word
that we have entered. This will give more intelligence to the utterances and
improve the conversation flow in the LUIS application.
E.g. The real-time example for phrase list is the booking synonym, or similar
meaningful words are reservation, appointment, date, engagement, etc.
37
While adding a phrase list feature for the intent, it will give us a number of
suggestions in the suggestion box with the help of machine learning features.
The phrase list will be part of the machine learning feature in a LUIS application
and we can add other prebuilt features as well. Once enabled, the global feature
will include the global vocabulary of the synonyms.
38
How to Train a LUIS application?
The red circle displayed in the Train menu denotes that the app has untrained
changes or some changes have happened during the interval of time which is
not yet updated in the application. So, we can update the application by clicking
on the Train menu button in the LUIS application and once the Train is successful
then the red color changes into green. Whenever the color of the Train is
negative, the score of the utterances will be negative (–1). Once build is
successful, the score changes to positive (0.5,1.0...)
39
Note : “Click on the Train menu and update the app”
Once the application build is successful or the circle in the Train menu changes
to green, we can test the application through the help of a Test menu in the LUIS
application. The test menu will open a test chat window and it will give you the
brief details about the entered text or utterances in the chat box.
40
The start over link will reset the history of the chat and it will create or begin a
new chat history.
As we all know, the term Publish is used to make the application live. There are
two types of development slots available in the Luis application.
● Staging
● Production
Select the appropriate development slot and publish the LUIS application to the
respective environment;this will be more helpful during the development phase
of an application.
Staging
The Staging environment is used for the purpose of testing only. In this way, we
can improve the quality of the LUIS application or the product.
Production
Once the application is ready for use then publish on the production slot and it
will be available for public use.
41
Note: We can choose any of the development slots for publishing the LUIS
application environment.
One of the important language category APIs in Cognitive Services is QnA Maker.
It is a cloud-based API service provided by Microsoft that lets you create a
conversational question and answer layer over your data. All of the chatbot’s
answers come from an already-configured custom KB (or Knowledge Base) in
QnA Maker or any other medium like LUIS, etc. We all know most industries are
using custom chatbots in their website for customer assistance to reduce
additional work. In the current Covid-19 pandemic assisting large amounts of
customers through any medium is quite difficult. So implementing customer
assisting chatbots is relevant for this current situation, and in the future as well.
There are very simple real time scenarios like leave requests and ticket creation,
which are handled easily through chatbots with the help of QnA maker
knowledge base, or LUIS. Nowadays many people are using QnA Maker for most
chatbot conversational flow.
One of the most familiar examples for QnA Maker implementation is FAQ
(Frequently asked questions) in the chatbot. If the user is asking one or more
42
questions to an intelligent chat bot (robot or non-human agent) through an chat
window that appear in the website or like Facebook messenger, slack, Microsoft
teams, skype, etc. The user will get an appropriate or predefined answer from
the chat bot, so all these FAQ or predefined questions are stored in the QnA
maker Knowledgebase.
We can easily create a QnA Maker model with the help of a
https://www.qnamaker.ai/ account and for this we require "QnA service in
Microsoft Azure." For that, first we need to create a QnA service API in Azure.
As per the latest Microsoft build in 2020, Microsoft has announced new features
for QnA Maker. The following features will be helpful to handle more extra
features in QnA maker applications in real time scenarios:
● Rich Text Authoring
● Role Based Access Control (RBAC)
43
E.g. Clicking on the “</>” icon in the rich text editor will display the detailed
markdown syntax information for the above screenshot.
Important: The video link is required as a thumbnail image option in the editor
section then it will display the video image otherwise it will display as a
corrupted image in the editor view. So, we can add video markdown syntax
included thumbnail as manually in the markdown section and it will work as
expected.
44
Role Based Access Control (RBAC)
Every application has three types of major roles which are reader, editor, and
publisher. Microsoft has configured it this way as well. The role assign option
will be available in QnA maker cognitive service in Azure portal.
Each role has the following capabilities to control the QnA maker user activities
for E.g. Edit, Add, Import, Delete, Publish, etc.
45
Roles
Before going to add a role assignment for user activity in QnA maker we can see
the important roles that we want to use for the further activities.
46
Create QnA Maker Service in Azure
Step 1:
Go to Azure portal and click on "+" icon -> go to "AI + Machine Learning" -> Click
on "See all".
Step 2:
If you are not able to see LUIS in the “Cognitive Services” then go to click on the
"See more" link on the right side.
Step 3:
Once you are able to see QnA Maker in the “Cognitive Services” list then go and
click on it.
47
Step 4:
The following screen will appear once you click on "Create" button in "QnA
Maker"
● Name: We can give one unique name for our QnA Maker service.
● Subscription: We can select our Azure subscription for QnA Maker
service.
● Pricing tier: As of now there are two pricing tiers available, "FO" & "SO",
and obviously "FO" is the free one. We can choose the appropriate pricing
tier as per our needs. When you are planning to do performance testing
then choose “SO” as the priority one because it will handle more number
of hits per second otherwise choose “FO” more for learning purposes.
● Resource group: We can create a new resource group or choose from an
existing one (We created a new resource group as "qna-cog-test").
● Resource group location: We can select our location from the Resource
group location. The best thing is we can choose a location closest to our
customer's needs.
● Azure search pricing tier: Each index will be considered as Knowledge
Base publish permission, so based on the needs select the appropriate
index accordingly.
● Azure search location: The best thing is we can choose a location closest
to our customer needs.
● App name: This will be part of our service url and give proper name.
● Website location: The best thing is we can choose a location closest to
our customer needs.
48
● App insights: If you are planning to track the service and log , etc then
you should enable app insights.
● Click on the "Create" button and wait for the build to succeed.
● Once the build has succeeded, then click on the "Dashboard" and we can
see "qna-cog-test" is created in the all resources list. QnA is ready for use!
49
Create Knowledge Base in QnA Maker
We have already created the QnA service in Microsoft Azure and now we can
easily create a Knowledge Base in our QnA Maker account. So go to QnA Maker
account and create a new Knowledge Base.
Microsoft Azure Directory ID: Select the appropriate directory from the list.
Azure subscription name: We can select our Azure subscription for QnA Maker
service.
Azure QnA service: The QnA service that we have created in azure.
Language: The current culture or language we’re going to use in QnA Maker
application.
Name your KB: The name of your KB.
50
● Create your KB.
Step 4:
The following step is not mandatory to add. If you select any chit-chat options
then automatically generate custom questions and answers in the respective
Knowledge Base.
51
Output:
The Knowledge Base is successfully created in QnA Maker with friendly chit-chat
questions and answers.
52
What is Chit-Chat?
The following are the currently available Chit-Chat options in the knowledge
base and based on the question “When is your birthday?” each chit-chat option
will respond with various types of answers.
Personality Example
Professional Age doesn't really apply to me.
Friendly I don't really have an age.
Witty I'm age-free.
Caring I don't have an age.
Enthusiastic I'm a bot, so I don't have an age.
E.g. The Professional Chit-Chat option will provide all kinds of professional
answers and we can choose this option for the business conversational flow. We
can try a friendly Chit-Chat option for the customer interaction in chatbot
conversation flow.
Note: All chit chat selections are based on our business requirements.
53
● Chinese
● English
● French
● Germany
● Italian
● Japanese
● Korean
● Portuguese
● Spanish
The definition of a Metadata is data about the data and in knowledge base it will
contain important information about the question pair such as store product id
, group name, etc as Metadata in the question pair. Metadata is a text-based
Key & Value pair used in each and every question in the knowledge base. It is
not mandatory to add metadata in the question section and also is case sensitive
in nature and it will store all the Key & Value info into small letters or lowercase.
Metadata information by default hides from the editorial page in the knowledge
base and enables it by clicking on the “Show Metadata” menu in the “View
options” menu list present in the knowledge base. It will list already added or
empty Metadata tags in each and every question pair. The following example
“editorial” is the Key and “chitchat” is the value of Metadata and it will always
store it as a small letter.
54
These are important points to keep in mind while creating a metadata in the
knowledge base.
We can expose all the knowledge base information using cognitive service api.
Once published (click on the PUBLISH menu in KB similar to what we have done
in LUIS application) then it will automatically generate the following information
to call a knowledge base through an HTTP request.
55
Knowledge Base API response
The generated search API credentials can be used for accessing the knowledge
base information. Based on the requirement we can configure additional
information to the API request.
We can include the following parameters in the request body based on the
requirements.
{
"question":"How are you?",
"isTest": false,
"Top": 2
56
Top: This will return the top question pair in the response based on the
threshold score in QnA maker. E.g. Top 2 questions will return from the QnA
maker based on the score.
Note: The question parameter must require in the api request body.
Output:
The following output returns based on the question that we have given in the
question parameter. That is the only parameter we have considered in the
request body and other parameters are not mandatory to add in the request
body section.
{
"answers": [
{
"questions": [
"Hey, how are you?",
57
"Are you doing well?",
"Are you doing OK?",
"How are things going?",
"How's it going?",
"metadata": [
{
"name": "editorial",
"value": "chitchat"
}
],
"context": {
"isContextOnly": false,
"prompts": []
}
}
],
"debugInfo": null,
"activeLearningEnabled": false
}
58
If the question that we have asked is in any of the knowledge bases in QnA maker
then it will return the detailed information about that particular question or qna
pair. We can return multiple qna or question pairs for giving the “Top”
parameter in the request body.
Summary of Dispatch
As we know, all of the applications that we are using in the real world are
growing day by day. So, handling all of this single handedly is very difficult in the
current situation since it will take too much time to develop and maintain the
application even if you are using a chatbot application in a real-time scenario.
As a Microsoft chatbot developer we are familiar with QnA maker knowledge
base and Luis models for the chatbot e2e communication. For E.g. all the content
that we have entered into the chatbot is considered as utterances in the Luis
model and questions in the knowledge base. When we are going to use multiple
models in our application at a time, we need to handle, train and publish the
model and we also need to hit multiple api calls for the relevant response for
each activity. So, we can reduce all this activity using the dispatch command tool
in the npm package. The dispatch has the capability to handle multiple bot
modules for the e2e communication. This will be very simple and we can set this
up as a web job or manual command execution in our local system, virtual
machine, CI / CD and Azure app service as well.
The name itself says it is dispatching something, yes this will be helpful when we
are using multiple models in our application. "The dispatch tool to create and
evaluate LUIS models is used to dispatch intent across multiple bot modules such
as LUIS models, QnA knowledge bases and others (added to dispatch as a file
type)".
In the above description it is clearly mentioned that you can add both LUIS and
QnA Maker models together in dispatch LUIS App or create any of the models.
The dispatch App has the capability to handle multiple bot modules together
59
without impacting the performance of the application. The dispatch LUIS App
will find out the intent based on the given utterances across all the modules that
we have integrated in the dispatch version of the LUIS application.
Note: The main purpose of the dispatch app is to identify and route intent for
multiple bot modules so it is concerned only with intent classification. We
cannot directly transfer entities in the dispatch application so we can choose a
pattern for the entity transfer.
60
The command using in Dispatch tool
The following are the important commands that are used in the dispatch model
creation in the LUIS application. All the descriptions are based on the document
provided by dispatch in GitHub account.
Option Description
-t or –type luis , qna , file
-I or –id Required only if type is luis/qna , LUIS
app id or QnA kb id - from application
settings page
--luisAuthoringKey (optional) LUIS authoring key
--luisAuthoringRegion (optional) LUIS authoring region
-s, --secret (optional) .bot file secret
-c, --culture (optional) Used to set LUIS app
culture for dispatch. Required if none
of dispatch source(s) is LUIS app.
--hierarchical (optional) Default to true. If false,
existing intents from source LUIS
model(s) will be available as the
dispatch intents.
-n or –name LUIS app name or QnA name (from
application settings page) or
module/file name for file type
--includePrompts (optional for QnA only) Default to
false. If set to true, QnA KB prompt
questions will be included in the
training set
--includeMetadata (optional for QnA only) Default to
false. If set to true, QnA KB metadata
will be included in the training set
-f or –filePath Required only if type is file, Path to
tsv file containing tab delimited
intent and utterance fields or .txt file
with an utterance on each line
--dispatch (optional) Path to .dispatch file
--dataFolder (optional) Dispatch working directory
61
-h or –help Output usage information
-v, --version (Required only if type is luis) LUIS app
version
-i, --id (required only if type is luis/qna) LUIS
app id or QnA kb id from application
settings page
-c, --culture (optional) Used to set LUIS app
culture for dispatch. Required if none
of dispatch source(s) is LUIS app
-s, --secret (optional) Secret used to
encrypt/decrypt .bot file
--gov (optional) Set to true to target Azure
goverment
--remote (optional) Set to true if invoking tool
remotely
First of all we need to install the “node js” (requires Node.js version 8.5 or above)
package in the target machine otherwise we cannot install the “botdispatch”
package because it will work on top of npm. In the prerequisite we have already
installed the node.js and this command will verify the version details of node in
the target machine.
node -v or node --version
After the installation of node js run this command to Install the botdispatch
package in target machine.
npm install -g botdispatch
62
Initialization Dispatch model
The command will initialize the dispatch LUIS application and that is going to
create a LUIS application with contains of all types chatbot models like qna
maker, knowledge base, and LUIS models.
Command
dispatch init -n <filename-to-create> --luisAuthoringKey "<your-luis-authoring-
key>" --luisAuthoringRegion <your-region>
We can copy the region and luis authoring key from the settings section in LUIS
application https://www.luis.ai/user/settings
63
Tips: Create the dispatch file in a specific folder or directory in your target
machine because it will generate multiple files and folders for the respective
intent in the dispatch luis application.
The initialization command will create the “.dispatch” file in the target machine.
The command will contain all the relevant information about the Luis application
that we have created and follow the existing steps for more clarity.
Output: The initialization command will create the “.dispatch” file with fewer
details because this will grow up while adding multiple chatbot models in the
dispatch application. Now this file will contain the app id of the LUIS application.
"authoringRegion": "westus",
"hierarchical": true,
"useAllTrainingData": false,
"dontReviseUtterance": false,
"copyLuisData": true,
"normalizeDiacritics": true,
"services": [],
"serviceIds": [],
"authoringKey": "8f34728f544e401d8e5e7b89636fd5f9",
"version": "Dispatch",
"region": "westus",
64
"type": "dispatch",
"name": "luis-tst"
}
Once after initialization of the dispatch model then we can add multiple models
in the Dispatch LUIS application. The command will be used for adding the QnA
knowledge base models into the dispatch application using the dispatch tool.
Command for qna
dispatch add -t qna -i "<knowledge-base-id>" -n "<knowledge-base-name>" -k
"<azure-qna-service-key1>" --intentName <intent-name>
The detailed information of the parameters is used in the QnA adding command
in the dispatch tool.
knowledge-base-id : copy from respective qna
65
intent-name: As we mentioned, the dispatch will find the intent based on the
utterances that we are searching in the dispatch-created LUIS application. The
intent name should be required while adding any models in the LUIS application.
This command will update an already created “.dispatch” file with qna maker
information and also remember all this information now added in the target
machine “.dispatch” file and not in the LUIS application.
dispatch add -t qna -i "9b334bc4-af33-4cab-a7ff-e252b9cd5754" -n "qna-cog-
test" -k " 32c3d00a996d4082b0ecff2b98ccb255" --intentName q_test-qna
Finally the qna information is added in the “.dispatch” file in the target machine
and here is the updated “.dispatch” file.
{
"authoringRegion": "westus",
"hierarchical": true,
"useAllTrainingData": false,
"dontReviseUtterance": false,
"copyLuisData": true,
"normalizeDiacritics": true,
"services": [
66
{
"intentName": "q_test-qna",
"includePrompts": true,
"kbId": "9b334bc4-af33-4cab-a7ff-e252b9cd5754",
"subscriptionKey": "0f4399fce1c24c7197a294a88b28dcb4",
"type": "qna",
"name": "qna-cog-test",
"id": "1"
}
],
"serviceIds": [
"1"
],
"authoringKey": "8f34728f544e401d8e5e7b89636fd5f9",
"version": "Dispatch",
"region": "westus",
"type": "dispatch",
"name": "luis-tst"
}
Note: The model will be growing day by day and in the multiple model
identification we can add a prefix in each intent in the dispatch model LUIS
application. Here we have added the prefix “q” for QnA maker and “l” for LUIS
(manually created luis model).
67
Add LUIS models in Dispatch
We can add “LUIS Application” into Luis dispatch application (Auto generated
application through the dispatch command tool and the version will be denoted
as “Dispatch” instead of “1.0”). Here Luis dispatch application acts as a parent
app because while running “dispatch refresh” command the LUIS app will push
all the latest updates to Luis dispatch application (Parent). So, this will be easy
for maintenance of multiple models in LUIS applications.
dispatch add -t luis -i "<app-id-for-luis-test-app>" -n "<name-of-luis-test-app>" -
v <app-version-number> -k "<your-luis-authoring-key>" --intentName <intent-
name>
We can get all the required command parameters for the LUIS app creation in
dispatcher from the LUIS application that we have created. For this, click on the
already created LUIS app and Go to “Settings” tab under the Manage menu and
it will provide all the relevant information.
68
name-of-luis-test-app: Any name we can give as an application name but for the
future maintenance better to give as the same application name.
Intent-name: As we mention the dispatch will find the intent based on the
utterances that we are searching in the dispatch created luis application. The
intent name should be required for adding any models in the LUIS application.
This command will update already created “. dispatch” file with LUIS model
information and also remember all this information now added in the target
machine “.dispatch” file and not in the LUIS application.
dispatch add -t luis -i "4068d4ee-f3b3-4f7c-9220-baeb983a6bd4" -n
"LuisTestingApp" -v 0.1 -k "8f34728f544e401d8e5e7b89636fd5f9" --
intentName l_cog_luis_test
The finally the luis application information added in the “.dispatch” file in the
target machine and here is the updated “.dispatch” file.
{
"authoringRegion": "westus",
"hierarchical": true,
"useAllTrainingData": false,
"dontReviseUtterance": false,
"copyLuisData": true,
69
"normalizeDiacritics": true,
"services": [
{
"intentName": "q_test-qna",
"includePrompts": true,
"kbId": "9b334bc4-af33-4cab-a7ff-e252b9cd5754",
"subscriptionKey": "0f4399fce1c24c7197a294a88b28dcb4",
"type": "qna",
"name": "qna-cog-test",
"id": "1"
},
{
"intentName": "l_cog_luis_test",
"appId": "4068d4ee-f3b3-4f7c-9220-baeb983a6bd4",
"authoringKey": "8f34728f544e401d8e5e7b89636fd5f9",
"version": "0.1",
"region": "westus",
"type": "luis",
"name": "LuisTestingApp",
"id": "2"
}
],
"serviceIds": [
"1",
"2"
],
"authoringKey": "8f34728f544e401d8e5e7b89636fd5f9",
"version": "Dispatch",
70
"region": "westus",
"type": "dispatch",
"name": "luis-tst"
}
We can add multiple sources to dispatch models into .dispatch files other than
Cognitive Services QnA and LUIS models.
dispatch add -t file -n TsvModule -f c:\src\tsvmodule.tsv
dispatch add -t file -n TextModule -f c:\src\textmodule.txt
dispatch add -t file -n JsonModule -f c:\src\jsonmodule.json
dispatch add -t file -f c:\src\rajeesh\filepathmodule.tsv --intentName
l_FilePathModule
The following are the supported file types.
● .tsv - Lines of tab delimited fields of intent and utterance (in that order).
● .txt - Lines of utterances with intent as file name.
● .json - Exported LUIS or QnA Maker json file.
There are a few situations which may occur in the future to remove any chatbot
models from the dispatch luis model application, either QnA or LUIS, based on
the requirement. The following command will remove any of the models from
the dispatch application.
dispatch remove -t luis -i xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
dispatch remove -t qna -i xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
dispatch remove -t file -f c:\src\billmodule.json
71
Dispatch Luis App Creation Command
The following command will create, train and publish the Luis Dispatch
Application with consists of multiple models that we have added in the dispatch
application. This will be a one-time process and it will generate multiple files
including the “.dispatch” file with all the relevant application information.
dispatch create
Once we’ve created the dispatch file then the folder structure will be created in
the target machine for the respective directory and it will look like below:
The data folder will contain each intent name and the utterances details as text
file information.
The summary html file will contain all the detailed information about the intents
that we have used in dispatch Luis application.
72
The dispatch file created in the LUIS application has all the model's information
like qna and luis application and it will contain the utterances, phrase list,
entities, etc. Also, the version of the dispatch Luis application will be “VDispatch”
instead of “1.0” because the “1.0” version denotes manually created luis
application.
73
We can cross verify the dispatch version of the Luis application name and key in
the “.dispatch” file in the target machine. Here is the response of the dispatch
luis application and it contains the dispatch luis application app id and name.
{
"authoringRegion": "westus",
"hierarchical": true,
"useAllTrainingData": false,
"dontReviseUtterance": false,
"copyLuisData": true,
"normalizeDiacritics": true,
"services": [
{
"intentName": "q_test-qna",
"includePrompts": true,
"kbId": "9b334bc4-af33-4cab-a7ff-e252b9cd5754",
"subscriptionKey": "0f4399fce1c24c7197a294a88b28dcb4",
"type": "qna",
74
"name": "qna-cog-test",
"id": "1"
},
{
"intentName": "l_cog_luis_test",
"appId": "4068d4ee-f3b3-4f7c-9220-baeb983a6bd4",
"authoringKey": "8f34728f544e401d8e5e7b89636fd5f9",
"version": "0.1",
"region": "westus",
"type": "luis",
"name": "LuisTestingApp",
"id": "2"
}
],
"serviceIds": [
"1",
"2"
],
"appId": "f55e62e0-7034-417f-a9e2-792471142bc6",
"authoringKey": "8f34728f544e401d8e5e7b89636fd5f9",
"version": "Dispatch",
"region": "westus",
"type": "dispatch",
"name": "luis-tst"
}
75
Testing your dispatch model
The following command will execute evaluation on the dispatch model to verify
the cross validation and generate a summary of the evaluation:
dispatch eval [options]
The “dispatch refresh” command will be published for the new changes in the
already created dispatch application.
In the local machine we can use the following command.
dispatch refresh
In the remote machine we can use the following command.
dispatch refresh --remote true
The Command line interface for the dispatch tool creates the model for dispatch
version of dispatch including other models LUIS or QnA Maker app.
Open a command prompt or terminal window, and change directories to the
CognitiveModels directory or any other folder that we are going to maintain for
76
the Dispatch model because the command will be auto generated depending on
the file for dispatch model.
Step 1:
The npm version should be 8.5 or above required for Dispatch tool installation
and make sure you have the current version of npm and the Dispatch tool.
npm i -g npm
npm i -g botdispatch
Step 2:
The .dispatch file contains all the model information including authorization key.
Use dispatch init to initialize and create a .dispatch file for your dispatch model.
dispatch init -n <filename-to-create> --luisAuthoringKey "<your-luis-authoring-
key>" --luisAuthoringRegion <your-region>
Step 3:
Add LUIS apps and QnA Maker knowledge bases to the .dispatch file using add
command for respective models.
dispatch add -t qna -i "<knowledge-base-id>" -n "<knowledge-base-name>" -k
"<azure-qna-service-key1>" --intentName q_test-qna
dispatch add -t luis -i "<app-id-for-luis-test-app>" -n "<name-of-luis-test-app>" -
v <app-version-number> -k "<your-luis-authoring-key>" --intentName l_luis-test
dispatch add -t file -n TsvModule -f c:\src\tsvmodule.tsv
dispatch add -t file -n TextModule -f c:\src\textmodule.txt
dispatch add -t file -n JsonModule -f c:\src\jsonmodule.json
dispatch add -t file -f c:\src\rajeesh\filepathmodule.tsv --intentName
l_FilePathModule
Note: A maximum of 500 dispatch sources could be added to a Dispatch model.
Step 4:
77
Use dispatch create command to generate a dispatch model from the .dispatch
file.
dispatch create
Publish the dispatch LUIS app you just created.
78
References
● https://docs.microsoft.com/en-us/azure/cognitive-services/luis/
● https://docs.microsoft.com/en-us/azure/cognitive-services/qnamaker/
● https://docs.microsoft.com/en-us/azure/guides/developer/azure-
developer-guide
● https://docs.microsoft.com/en-us/azure/cognitive-services/what-are-
cognitive-services
● https://docs.microsoft.com/en-us/azure/bot-service/bot-builder-
tutorial-dispatch?view=azure-bot-service-4.0&tabs=cs
79