Data labelling, also known as data annotation, is the process of assigning tags or labels to an object or data. There are several ways to annotate data, and the ideal method depends on the nature of data being annotated.
While each type of annotation is difficult in its own right, video labelling is perhaps the most difficult to implement. If you wish to use video labelling for your projects, you must at least be aware of the basics of the process. With that in mind, here are five basic details you must know about video labelling.
5 Basics Of Video Labelling
1. The video format is crucial in video annotations
Video formats vary in quality. A low-quality video would often be much more difficult to label as each frame would be less recognizable. The opposite applies to high-quality videos.
In other words, the video format is crucial in video annotation. A common practice among machine learning practitioners is to choose a format with no video compression. This is because, despite the reduction in file size, compression often leads to a loss in image quality.
Of course, some may argue that the difference in quality between compressed and uncompressed video data is negligible. That might be the case for humans, but the difference is significant for an AI. For that reason, it’s best to look for video formats that use encoding instead of compression. Video encoding shrinks file size, without compromising the quality.
That’s why if you’re looking for a video annotation tool platform, you must always look into their supported video formats. MP4 is one example of a format that uses video encoding.
2. You don’t have to label each object in a video
It’s a common misunderstanding among machine learning (ML) practitioners that data annotation must take into account every object in the data. That’s not always the case.
In a still image, you only label the objects considered as the image’s focal part while ignoring the background. Similarly, in a video, there’s no need to label each object in every frame. Only the objects that change the scene will be labelled. For instance, in a 10-hour video of a stationary apple, you don’t have to label the apple for every frame it’s in.
However, if the scene changes, perhaps because you placed an orange halfway through the video, then you must label the orange. Data labelling may be more straightforward than some think, but it’s not a mere ‘one-click’ process. That’s especially true with videos featuring multiple scenes.
3. Video labelling is the most laborious but insightful form of data annotation
Data annotation is not a mechanically difficult task, but it’s tedious simply because it takes so much time. Assuming it takes five seconds to label an object manually, an image with just six objects will already take half a minute to label completely.
Going further, a single second of a video consists of around 24 to 30 images or frames. Therefore, a video will take even more time to fully annotate. Nevertheless, video data is more insightful as it gives much more information about the surroundings and context.
Suppose you have a still image of a person with their hand stretched towards a product in a grocery store. It’s difficult to say whether that person has just returned the product to its shelf or if they intend to take it. However, a video can answer that question. Simply put, video data gives you insights into actions or scenarios where the context changes over time. That’s precisely why video labelling works best in the automotive industry, specifically for self-driving cars.
4. The automotive industry benefits significantly from video labelling
Video annotation has several real-world applications, but it has the most impact in the automotive industry. Self-driving vehicles utilize AI to detect pedestrians, distinguish and recognize road signs, and perceive road boundaries. The AI was unpolished for a while, but with the help of data from video annotation projects, the AI was able to improve the accuracy of recognition. But of course, video annotation has other uses outside the automotive industry.
You may also find video labelling AI in the retail industry, specifically in self-service retail stores. Stores use video annotation to detect products and calculate their prices accordingly.
Apart from images, video data also contain audio. This is one thing that ML practitioners often forget when performing video annotations. It may not be as insightful as the frames, but audio can also help comprehend a scene’s context. Suppose you have a video of a customer talking to a cashier. The AI can perceive the customer’s emotional state using their tone and pitch. It may be difficult, but video labelling can make this task easier by assigning tags to certain audio cues and allowing the AI to draw from this compiled data.
Wrapping up
It’s no secret that video labelling differs from image and audio labelling. However, ML/AI practitioners often underestimate just how much more difficult it is compared to other forms of data annotation. While it should take a while to familiarize yourself with its techniques, this guide should at least give you a basic understanding of video labelling and help prevent you from getting overwhelmed with the challenges that come with this task.
References:
- “7 Things We Looked for in a Video Labelling Tool”, Source: https://towardsdatascience.com/7-things-we-looked-for-in-a-video-labelling-tool-705968e39d19
- “The Complete Guide to Video Annotation”, Source: https://dataloop.ai/blog/video-annotation-guid0e0/
