In my first blog, ‘Bird Recognition App using Microsoft Custom Vision AI and Power BI’, we looked at the intriguing behaviors and attributes of birds using Power BI. This inspired me to create an ‘AI for birds’ web app’ using Azure Custom Vision along with a phone app using Power Apps and an iPhone / Android platform that could identify a bird in real-time. I created this app to raise awareness of the heart-breaking reality which most birds face around the world.
In this blog, let’s go behind the scenes and take a look at the journey of how this was created.
What is Azure Custom Vision?
Azure Custom Vision is an image recognition AI service part of Azure Cognitive Services that enables you to build, deploy, and improve your own image identifiers. An image identifier applies labels (which represent classes or objects) to images, according to their visual characteristics. It allows you to specify the labels and train custom models to detect them.
What does Azure Custom Vision do?
The Custom Vision service uses a Machine Learning algorithm to analyze images. You can submit groups of images that feature and lack the characteristics in question. You label the images yourself at the time of the submission. Then, the algorithm trains to this data and calculates its accuracy by testing itself on those same images.
Once the algorithm is trained, you can run a test, retrain, and eventually use it in your image recognition app to classify new images. You can also export the model itself for offline use.
How does it work?
1. Upload images – Bring your own labelled images or use Custom Vision to quickly add tags to any unlabelled images.
2. Train the model – Use your labelled images to teach Custom Vision the concepts you care about.
3. Evaluate the result – Use simple REST API calls to quickly tag images with your new custom computer vision model.
In my first blog, ‘Bird Recognition App using Microsoft Custom Vision AI and Power BI’, we looked at the intriguing behaviors and attributes of birds using Power BI. This inspired me to create an ‘AI for birds’ web app’ using Azure Custom Vision along with a phone app using Power Apps and an iPhone / Android platform that could identify a bird in real-time. I created this app to raise awareness of the heart-breaking reality which most birds face around the world.
In this blog, let’s go behind the scenes and take a look at the journey of how this was created.
What is Azure Custom Vision?
Azure Custom Vision is an image recognition AI service part of Azure Cognitive Services that enables you to build, deploy, and improve your own image identifiers. An image identifier applies labels (which represent classes or objects) to images, according to their visual characteristics. It allows you to specify the labels and train custom models to detect them.
What does Azure Custom Vision do?
The Custom Vision service uses a Machine Learning algorithm to analyze images. You can submit groups of images that feature and lack the characteristics in question. You label the images yourself at the time of the submission. Then, the algorithm trains to this data and calculates its accuracy by testing itself on those same images.
Once the algorithm is trained, you can run a test, retrain, and eventually use it in your image recognition app to classify new images. You can also export the model itself for offline use.
How does it work?
1. Upload images – Bring your own labelled images or use Custom Vision to quickly add tags to any unlabelled images.
2. Train the model – Use your labelled images to teach Custom Vision the concepts you care about.
3. Evaluate the result – Use simple REST API calls to quickly tag images with your new custom computer vision model.
How to experience the Custom Vision API in Power Apps, Mobile Application, & the Website:
1. Power Apps:
The Custom Vision API can be linked to the Power Apps by the “Custom Vision” connector. By providing a few details to the custom vision connector such as “Prediction Key” as well as “Site URL”, you can seamlessly use Custom Vision API in your Power App.
2. Mobile Application (Android and iOS):
In the Flutter Application, we called the Custom Vision API by using HTTP requests as well as Dio Packages. For Power BI Reports part of the mobile app, we embedded the Power BI report iframes into the flutter app by using WebView.
3. Website:
The Custom Vision API is connected to the website via Ajax & HTML tags. On the website, we published the Power BI Report through the HTML iframe. The generated Power BI Embedded iframe is effortlessly compatible with all the browsers.
The possibilities of Cognitive Services and Machine Learning are limitless!
If you have not tried the AI for Birds Mobile app yet, there is no better time! Both (Android & iOS) apps are available to download.
To download this app, please search “AI for Birds” in the Google Play Store, or the Apple’s App Store.
How to improve your Classifier:
Let’s talk about the ways to improve the quality of your Custom Vision Service Classifier. The quality of your classifier depends on the amount, quality, and variety of the labelled data that you provide and how balanced the overall dataset is.
A good classifier has a balanced training dataset that represents the submitted classifier. The process of building such a classifier is iterative and it’s common to implement a few rounds of training to reach expected results.
The following is a general pattern to help you build a more accurate classifier:
- First-round training.
- Add more images and balance data, then retrain it.
- Add Images with varying background, lighting, object size, camera angle, and style; retrain.
- Use the new Image(s) to test the prediction.
- Modify existing training data according to predicted results.
References:
- https://www.customvision.ai/
- https://docs.microsoft.com/en-us/azure/cognitive-services/Custom-Vision-Service/overview
- https://azure.microsoft.com/en-us/services/cognitive-services/custom-vision-service/
- https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/getting-started-build-a-classifier
Power BI:
What is Power BI and what does it do?
Power BI is a business analytics service by Microsoft. It aims to provide interactive visualizations and business intelligence capabilities with an interface simple enough for end-users to create their reports and dashboards.
Power BI is a business suite that includes several technologies that work together to deliver outstanding data visualizations and business intelligence solutions.