Object Detection in Deveco Studio

Question

Hello World!

I'm trying to develop Object Detection in DevEco Studio. I'm new to DevEco Studio development. I have built the program with Android Studio (Android version) with TFLite and also tried with YOLOv5 PyTorch and it worked. So, I also want to develop with DevEco Studio (Huawei version).

Some of the problems I'm facing in DevEco Studio are as follows:

1. The library from TFLite and PyTorch cannot be used in DevEco Studio, even though it is installed in the dependencies. The reason is because the Class available on TFLite and PyTorch is a Class that is used only for Android. The naming of variables, functions and so on in DevEco Studio is different from Android Studio. For example, RectF in Android Studio is written as RectFloat in DevEco Studio.

2. I tried using DJL(Deep Java Library) PyTorch this time. Even though PyTorch is installed, it can't read the engine and I get an error like this.

[tablet] SEVERE ai.djl.engine.EngineException: No deep learning engine found.

This is the code I use.

private void init(){
        ImageClassificationTranslator translator = ImageClassificationTranslator.builder()
                .addTransform(new RandomResizedCrop(INPUT_SIZE, INPUT_SIZE, 0.6, 1,
                        3. / 4, 4. / 3))
                .addTransform(new ToTensor())
                .addTransform(new Normalize(
                        new float[] {0.5f,0.5f,0.5f},
                        new float[] {0.5f,0.5f,0.5f}))
                .optApplySoftmax(true)
                .optSynset(herbsNames)
                .optTopK(5)
                .build();
        HiLog.debug(LOG_LABEL, "debug process:v---------alue:%{private}s", translator);
        //Engine engine = Engine.getEngine(PtEngine.ENGINE_NAME);
        Model model = Model.newInstance("model", Device.cpu()); //Problem is here
        HiLog.debug(LOG_LABEL, "debug process:v---------alue:%{private}s", model);
        try{
            InputStream inputStream = HerbUtil.class.getClassLoader().getResourceAsStream("yolov5s.pt");
            HiLog.debug(LOG_LABEL, "debug process:v---------alue:%{private}s", inputStream);
            if (inputStream == null){
                throw new RuntimeException("ERROR");
            }
            model.load(inputStream);

            predictor = model.newPredictor(translator);
        } catch (MalformedModelException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

3. I am using Android Studio Dolphin version in this development. And HMS Toolkit seems to be no longer usable in Dolphin versions and above. So I have to do it in DevEco Studio.

I don't want to use an internet connection while this app is running on the device, so I don't use Docker or any other cloud service. Online object detection will reduce the detection speed. Since I'm doing Live Object Detection, I really need speed and accuracy.

So, my simple question is:

1. Is it possible to convert code from Android Studio to DevEco Studio? Even though they both use Java, they are very different in their implementation.

2. Is there a tool for object detection that can be used in DevEco Studio?

3. Is there a way to run my Android program on DevEco Studio or Huawei device?

Really need input from all of you. I've tried many ways and articles about object detection tools (PyTorch, TFLite, .etc) for DevEco Studio are almost non-existent. Thank you in advance.

Answer 1

DJL should work for your case.

The exception is usually because you missed some DJL's dependency in your application. To load PyTorch model in DJL, you must at least include: ai.djl.pytorch:pytorch-engine in your project.

<dependency>
    <groupId>ai.djl.pytorch</groupId>
    <artifactId>pytorch-engine</artifactId>
    <version>0.21.0</version>
    <scope>runtime</scope>
</dependency>

Another reason that can cause this issue if your applicate uses a customized ClassLoader. DJL uses ContextClassLoader to load resources, and your ContextClassLoader failed to locate DJL jar files. You can manually set a proper ContextClassLoader before loading your model in this case.