How are AI chatbots trained?

By Neha M

Hello geeks! If you have come to this page, you are up to something really cool! Don’t worry, you won’t be disappointed as you will realise training chatbots is much cooler than you expected, as you dig in.   In my previous blog, Are chatbots really intelligent? I explained how chatbots are capable of learning to speak just like a human. Please go ahead and refer to the blog if you haven’t yet, as it gives out several basic concepts that you require in case you are a newbie. This blog will focus mainly on technical aspects of training chatbots

Know the purpose

Before jumping into building a chatbot, we need to first know what we are building it for?

1. Open-domain vs Closed-domain

Open-domain chatbots are required to answer anything in the world that is asked. It should be able to answer questions like “How many Grammy awards has Eminem won?” to “Could it rain tomorrow?”. On the other hand, Closed domain chatbots are designed to answer or act on a specific topic or domain. The bot should have enough domain knowledge to answer anything about that particular domain.

2. Chit-chat  vs Task-specific

This category is based on the type of conversation the bot can handle. Chit-chat bots, as the name says, are meant for general chit-chatting that happens between humans, like, “how’s your day going?”, “I feel like sleeping”, etc. This is closest to a human a chatbot can be.  Task-specific bots are designed to carry out predefined sets of actions. Examples for this category could be, “play music for me”, “place an order of 5 eggs”, etc. 

Identify the languages you want your chatbot to handle

A conversation with a human means conversing in natural language. Once you know what languages you want the bot to be able to handle, you need to 

  • Make sure the bot can identify and understand the languages
  • Make sure the bot can retrieve answers or replies in the same language

How do ai chatbots online work?

A chatbot, irrespective of what language it can handle or what category it belongs to, it always needs to do the following :

 • Know the context:

The bot needs to save the state of the conversation so far, in order to act upon upcoming input. 

 In the above two conversations, though both the text following says – “52”,  they mean different things because of the context in which they are being mentioned.  

 • Identify the intent of the user :

Consider the two inputs above – Both the queries might seem very similar, but intent wise, the bot should identify that the user is asking a question about Bot/website feature in the former example, whereas, requesting to place an order in latter one. 

When you have a predefined set of intents which you would want your bot to learn, any text classification methods – multiclass or/and multilabel,  can be used to train models to identify the respective intent(s) for each query. 

 • Identify entities mentioned in the query :

While the above two steps are required to decide upon a certain action, this particular step helps us to retrieve the relevant results.  Both the above inputs have the same intent – the intent of seeing products – which means, the action should be – show products.

Entity Recognition comes into picture while deciding what products to show :

  1. Show ‘Dresses’
  2. Show ‘red’ coloured dresses in the first case and ‘yellow’ coloured dresses in the second.

Go ahead and look out for Entity Recognition techniques out there. While some Rule-Based Techniques are still going strong, AI techniques have definitely overtaken most of these. I have tried quite a few techniques, but Spacy has been my personal favourite among the AI-driven Entity Recognition tools.   

 • Decide upon the action :

Based on the context and intents identified, decide the actions which are needed to be executed. We already saw an example of this in the previous bullet, where we decided the action should be to ‘show the products’ since the intent was to ‘see the products’.

Retrieve the results:

Once you decide upon how to respond, the bot should now decide what response to give. As I mentioned earlier, Entity Recognition helps us to do this task, as the entities act as keywords for us.  So what techniques to use to get these responses? This is a huge technology space by itself – ‘Information Retrieval’.

General steps involved : 

  1. Query Expansion: Entities recognised in the user’s input, sometimes might not be enough to match correctly with our resources. Hence, we use something called query expansion which helps us to get more relevant keywords. A knowledge graph/ontology is usually created for this purpose. It is nothing but entities linked with each other over some relation. These knowledge graphs are queried using our existing entities, to get similar relevant keywords.
  2. Match with the relevant resources: This process could again be either rule-based or AI-driven. I prefer AI-driven search/answer generation, not because it’s “AI” or something, but due to its flexibility in the sense, it has the ability to do semantic matching, it can handle spelling mistakes, it could be language agnostic, etc.

That’s it! Cool enough? Looking for a Full Stack Conversational AI solution for your retail businesses? Get in touch with us at or visit our website to start a conversation.

Are AI chatbots really intelligent?

By Neha M

The first thought that we get, as we come across a chatbot is that, if they are really “intelligent”.  And if they are,...Continue

NLP with Python

By Neha M

Hi. When you are building some model/application, it’s always important to know what you have in hand. I know I spend a lot of...Continue

Multilingual Conversational AI Chatbot

By Neha M

Greetings! When we go to a physical store and talk to attendants, we tend to use our regional language rather than an official...Continue