ML Kit Vision
Installation and getting started with ML Kit Vision.
This module requires that the module is already setup and installed. To install the "app" module, view the Getting Started documentation.
If you're using an older version of React Native without autolinking support, or wish to integrate into an existing project, you can follow the manual installation steps for iOS and Android.
What does it do
ML Kit Vision makes use of Firebase's Machine Learning Kit's Text Recognition, Face Detection, Barcode Scanning, Image Labeling & Landmark Recognition features.
Depending on the service, it is possible to perform Machine Learning on both the local device or cloud.
The table below outlines the current module support for each available service, and whether they are available on local device, cloud or both.
To get started, you can find the documentation for the individual ML Kit Vision services below:
To be able to use the on-device Machine Learning models you'll need to enable them. This is possible by setting the below noted properties on the file at the root of your project directory.
The models are disabled by default to help control app size.
Since only models enabled here will be compiled into the application, any changes to this file require a rebuild.
firebase / firebase-android-sdk Public
[READ] Step 1: Are you in the right place?
Issues filed here should be about bugs in the code in this repository.
If you have a general question, need help debugging, or fall into some
other category use one of these other channels:
- For general technical questions, post a question on StackOverflow
with the firebase tag.
- For general Firebase discussion, use the firebase-talk
- For help troubleshooting your application that does not fall under one
of the above categories, reach out to the personalized
Firebase support channel.
[REQUIRED] Step 2: Describe your environment
- Android Studio version:4.0.1
- Firebase Component: ML(Vision)
- Component version:24.1.0
[REQUIRED] Step 3: Describe the problem
Steps to reproduce:
When I use the latest version of firebase-ml-vision which is 24.1.0, the google service plugin (com.google.gms:google-services:4.3.3) isn't compatible with it. It display some error in such as
The error stacktrace is below:
Even if did the to ignore the version check, the project still won't be compiled.
How to perform text recognition using Firebase ML Kit in Flutter
Connect with 3,000+ mobile app devs on Slack Join Codemagic Community
Written by Souvik Biswas
It is a myth among app developers that integrating machine learning (ML) into any application is a tough job. With recent innovations in ML tools, it is much easier to implement ML into your apps, even without having any expertise in the field of machine learning.
In this article, I will be showing you how to use Firebase ML Kit to recognize texts in your Flutter app.
Here is a brief synopsis of the topics we are going to cover:
- Setting up a new Firebase project
- Accessing the device camera from the app
- Using Firebase ML Kit to recognize texts
- Identifying email addresses from images
- Using the CustomPaint widget to mark the detected texts
Let’s get started.
Create a new Flutter project
Use the following command to create a new Flutter project:
Now, open the project using your favorite IDE.
To open with VS Code, you can use the following command:
Set up Firebase for the project
In order to use Firebase ML Kit in your app, you have to first complete the Firebase setup for both the Android and iOS platforms.
Create a new project
To create a new Firebase project, head over to this link.
Click Add project to create a new project.
Then enter the project name and continue.
The Firebase Console will create a new project and take you to its Dashboard.
From the project Dashboard, click on the Android icon to add Firebase to your Android app.
Enter the package name, app nickname, and SHA-1, then click Register app.
Next, you have to download the file and place it in the appropriate folder: project directory → android → app. Then click Next.
Add the Firebase SDK as per the instructions.
Continue to the console.
Also, make sure that you have your Support email added in the Project Settings.
From the project Dashboard, click on the iOS icon for adding Firebase to your iOS app.
Enter the iOS bundle ID and App nickname, then click Register app:
Next, download the file.
Now, open the folder of your Flutter project using Xcode, then just drag and drop the file in the appropriate location.
Skip Steps 3 and 4.
Then, click Continue to console.
The Firebase setup is now complete, and you can move on to start building the app.
The Flutter app that we are going to build will mainly consist of two screens:
CameraScreen: This screen will only consist of the camera view and a button to take a picture.
DetailScreen: This screen will show the image details by identifying the texts from the picture.
The final app will look like this:
Accessing device camera
To use the device’s camera in your Flutter app, you will need a plugin called camera.
Add the plugin to your file:
For the latest version of the plugin, refer to pub.dev
Remove the demo counter app code present in the file. Replace it with the following code:
In the above code, I have used the method to retrieve the list of device cameras.
Now you have to define the , which will show the camera preview and a button for clicking pictures.
Create a object:
Initialize the inside the method:
requires two parameters:
CameraDescription: Here you have to pass which device camera you want to access.
- 1 is for the front camera
- 0 is for the back camera
ResolutionPreset: Here you have to pass the resolution quality of the camera image.
In order to prevent any memory leaks, dispose the :
Now, let’s define a method called for taking a picture and saving it to the file system. The method will return the path of the saved image file.
Before defining the method, add two new plugins to the file:
- path_provider: For retrieving a path from the file system
- intl: Helps in formatting date and time
I have used the current date and time with each image name in order to easily separate the images from each other. If you try to save an image with the same name, then it will produce an error.
It’s time to build the UI of the . The UI will consist of a Stack with the camera preview, and on top of it, there will be a button for capturing pictures. Upon successful capture of the picture, it will navigate to another screen called .
So, you have completed adding the camera to your app. Now, you can analyze the captured images and recognize the texts in them.
Adding Firebase ML Kit
Import the plugin called firebase_ml_vision in your file:
You have to pass the path of the image file to the . The basic structure of the is defined below:
Here, you will have to define two methods:
- _getImageSize(): For retrieving the captured image size
- _initializeVision(): For recognizing texts
Retrieving image size
Inside the method, you have to first fetch the image with the help of its path and then retrieve the size from it.
Recognizing email addresses
Inside the method, you have to perform the whole operation of recognizing the image and getting the required data from it. Here, I will show you how to retrieve email addresses from the recognized texts.
Retrieve the image file from the path, and call the method:
Create a object and a object:
Retrieve the object by processing the visionImage:
Now, we have to retrieve the texts from the and then separate out the email addresses from it. The texts are present in blocks -> lines -> text.
Store the retrieved text in the variable.
Building the UI
Now, with all the methods defined, you can build the UI of the . The UI will consist of a Stack with two widgets, one for displaying the image and the other for showing the email addresses.
When the variable is null, it will display a .
The app will look like this:
Marking the detected texts
You can use the widget to mark the email addresses using rectangular boxes.
First of all, we have to make a modification to the method in order to retrieve the line elements.
Wrap the containing the image with the widget.
Now, you have to define the class, which will extend .
Inside the method, retrieve the size of the image display area:
Define a method called , which will be helpful for drawing rectangular boxes around the detected texts.
Define a object:
Use to draw the rectangular markings:
After adding the markings, the app will look like this:
Running the app
Before you run the app on your device, make sure that your project is properly configured.
Go to project directory -> android -> app -> build.gradle and set the minSdkVersion to 21:
Add the following in :
Go to .
Uncomment this line:
Add the following at the end:
Now, you are ready to run the app on your device.
You can use Firebase ML Kit to add many other functionalities as well, like detecting faces, identifying landmarks, scanning barcodes, labeling images, etc.
The GitHub repo of the project is available here.
Souvik Biswas is a passionate Mobile App Developer (Android and Flutter). He has worked on a number of mobile apps throughout his journey. Loves open source contribution on GitHub. He is currently pursuing a B.Tech degree in Computer Science and Engineering from Indian Institute of Information Technology Kalyani. He also writes Flutter articles on Medium - Flutter Community.
Ml vision firebase
.ML Kit: Machine Learning for Mobile with Firebase (Google I/O'19)
You will also like:
- Remote driveway alarms
- Fusion 360 grbl
- Ancestry® world archives project
- Sinusitis arrhythmia infection
- Refurbished galaxy s10 unlocked
- Skar rp 800
- Nemo voices