AI without data
Artificial intelligence or AI refers to the simulation of human intelligence in machines. (Refer the article Are AI and machine learning the same if you wish to understand more on what is AI) Artificial intelligence-based system learns from data (also referred as “training data”). These systems find relationships hidden in the training data, develop an understanding of the data, make decisions and then evaluate their performance from the data. And better the training data, better the system performs. Or in other words, the training data (both the quality and quantity) dictates how well the AI system will perform. In this article I will cover some hacks one can use if there is not enough training data available. So read on.
But first let’s understand how the training process works in the AI world.
How does the training process work?
Let’s say you want to make an AI-based system capable to do visual search i.e. it should be able to recognize a particular object in a given image (example: is the person in the photo wearing a jacket or not?). The process of training this AI is as follows (refer the picture below)
- A computer program is fed through a large number, usually thousands, of labelled images of “person wearing jacket” (positive label) and “not wearing jackets” (negative label). This is the training data for the AI.
- The program learns on this data and produces a model to detect jackets.
- This learning is repeated in fixed iterative cycles till the performance of the model improves and then the model is able to detect the jackets with good precision in new unseen images. This iterative process is called Training.
Businesses often struggle to find enough relevant training data to train their AI and this is because most often the data they need resides in multiple sources, are usually available in an unstructured format and therefore not useful for machines to learn from. So in such situations, how can you still make sure that AI works for you?
What can you do when you don’t have much data?
Here are 5 possible hacks you can try out and it is very likely that one or more will work in your situation.
Top 1 hack: Transfer learning
Transfer learning is about using the knowledge gained while solving one problem for which a lot of labeled data is available and applying it to a different but related problem for which only little labeled data is available.
For example, the knowledge gained while learning to recognize a person in an image could apply to recognize the types of clothes that the person is wearing and finding dominant colors in his clothing.
Top 2 hack: Data augmentation
Data augmentation is a process that increases the diversity of data available for training models, without actually collecting new or more data. As an example, for computer vision-related tasks we have the following options:
- Horizontal/Vertical Flip: Flip the image horizontally or vertically to add more diverse data.
- Change in scale: Crop and/or zoom images in various scales
- Lighting conditions: Change the brightness/sharpness/contrast of the images available.
- Rotation: Rotate the image by some different degrees.
For text (NLP) related task you can try the following options:
- Synonym Replacement: Randomly choose some words from the sentence and replace each of these words with one of its synonyms chosen at random. This creates more utterances of as existing sentence.
- Back Translation: English is one of the languages which is having lots of training data for most of the NLP(Natural Language Processing) tasks. So, if there is less data in a language for a particular task then more training data can be created by translating English data into that language.
Top 3 hack: Problem Reduction
Problem reduction approach refers to modifying the new data or unknown problem to a known problem such that it can be easily solved using existing techniques. Here is an example:
- Suppose we have a small labeled dataset of “audio clips”, where the sound in each clip belongs to one of the 10 categories like a car engine, piano note, drilling machine, and a dog barking. Now say, we want an AI-based system capable of recognizing the source of audio in these clips.
- If you try using any existing audio-based algorithms to solve this problem, it might work out. However, the small data set can be a deal-breaker.
- One alternate method is to try converting the “audio clips” into “images” thereby reducing the audio problem to an image classification problem. Now we can use any suitable computer vision-based algorithm to train on this “new image” dataset and create a system capable of classifying these “audio” images into one of the 10 categories.
- Surprisingly results have shown that the performance of these image-based systems is far better than the audio-based systems.
Top 4 hack: One-shot learning
Neural networks are good at learning from high dimensional data like images or spoken language, but only when they have huge amounts of labeled examples to train on. Humans, on the other hand, are capable of one-shot learning – if you take a human who has never seen a fork in his entire life and show him one, next time if you ask him to bring one from that messy drawer of your kitchen, he might be able to fetch that with high precision. For this type of approach, there are advanced algorithms that we will try to cover in later blogs.
Top 5 hack: Open source datasets and APIs
Various data sets are available in the open-source community or on the internet, which can be easily downloaded and used for training purposes. Some examples like the amazon dataset and google dataset are also accessible easily. Apart from these, search engines like google and bing also provide APIs to download datasets and pre-trained models, which can be used in many ways to improve the performance of an AI-based system.
While availability of relevant training data decides whether an AI system will perform well or not, it is certainly not a hopeless case if one does not have enough data. There are multiple techniques and hacks that one can still try out and I have listed the top5 above.