Mobile Vision Searching cameraSource detections for items in a list

418 Views Asked by At

I am currently trying to write an android app in which a user can blacklist any food ingredients he or she wants to avoid. The user should then be able to scan a label and instantly be told whether or not any blacklisted ingredients are found via text recognition.

I am using a cameraSource to detect the text in real time which appears to somewhat work, but only when very few words are present on screen. When there are too many words on screen, it cannot find anything.

What is going wrong when larger amounts of words are present?

        private SurfaceView cameraView;
        private TextView textView;
        private CameraSource cameraSource;
        private const int RequestCameraPermissionID = 1001;
        public JavaList<string> userIngredients;
        public ISharedPreferences pref;
        public ISharedPreferencesEditor edit;
        public Bitmap imageBitmap;


        protected override void OnCreate(Bundle bundle)
        {
            base.OnCreate(bundle);

            // Set our view from the "main" layout resource
            SetContentView(Resource.Layout.ScanLayout);

            cameraView = FindViewById<SurfaceView>(Resource.Id.surface_view);
            textView = FindViewById<TextView>(Resource.Id.text_view);
            pref = Application.Context.GetSharedPreferences("UserPrefs", FileCreationMode.Private);
            edit = pref.Edit();
            var preferences = pref.GetStringSet("UserPrefs", new JavaList<string>());
            userIngredients = new JavaList<string>(preferences);
            var bitmapOptions = new BitmapFactory.Options();

            TextRecognizer textRecognizer = new TextRecognizer.Builder(ApplicationContext).Build();
            if (!textRecognizer.IsOperational)
            {
                Log.Error("Main Activity", "Detector dependancies are not yet available");
            }
            else
            {
                cameraSource = new CameraSource.Builder(ApplicationContext, textRecognizer)
                    .SetFacing(CameraFacing.Back)
                    .SetRequestedFps(2.0f)
                    .SetAutoFocusEnabled(true)
                    .Build();

                cameraView.Holder.AddCallback(this);
                textRecognizer.SetProcessor(this);
            }
        }



        public void SurfaceCreated(ISurfaceHolder holder)
        {
            if (ActivityCompat.CheckSelfPermission(ApplicationContext, Manifest.Permission.Camera) != Android.Content.PM.Permission.Granted)
            {
                //Request Permission
                ActivityCompat.RequestPermissions(this, new string[] {
                    Android.Manifest.Permission.Camera
                }, RequestCameraPermissionID);
                return;
            }
            cameraSource.Start(cameraView.Holder);
        }

        public void SurfaceDestroyed(ISurfaceHolder holder)
        {
            cameraSource.Stop();
        }

        public void ReceiveDetections(Detections detections)
        {
            bool blackListedFound = false;
            SparseArray items = detections.DetectedItems;
            if (items.Size() != 0)
            {
                textView.Post(() =>
                {
                    for (int i = 0; i < items.Size(); ++i)
                    {
                        for (int j = 0; j < userIngredients.Size(); j++)
                        {
                            if (((TextBlock)items.ValueAt(i)).Value.Equals(userIngredients[j]))
                            {
                                blackListedFound = true;
                                textView.Text = "Not reccomended\nIngredient Found: " + userIngredients[j];
                            }
                        }
                    }
                });
            }
            else if (blackListedFound == false)
            textView.Post(() =>
            {
                textView.Text = "No Ingredients found";
            });
        }


    }
}

Here are some example images of my current problem; My settings menu with water added to blacklist

Here is an example of the app failing to find a blacklisted ingredient (Water); enter image description here

0

There are 0 best solutions below