In Flutter, how do I display CVPixelBuffer from iOS in a Widget?

Question

I have a Flutter app that calls an object detector in CoreML via the iOS platform channel. I would like to ensure the model is getting the image in the correct format by viewing it in a Flutter widget.

I have access to the CVPixelBuffer data. How do I send it back to Flutter and display it in a widget?

Answer 1

Platform channels allow for two way communication. You're already done with half the task.

1. Flutter -> Native

You must have already done this using something like

await platform.invokeMethod('sendDataToIOS',{'data':data});

2. Native -> Flutter

    platform.setMethodCallHandler((call) async {
     if (call.method == 'receiveDataFromiOS') {
      final Map<String, dynamic> arguments = call.arguments;
      final String data = arguments['data'];
      receiveDataFromiOS(data);
    }});
   

Answer 2

import 'dart:ui' as ui;
import 'package:flutter_vision/flutter_vision.dart';

Future<ui.Image> convertPixelBufferToImage(CVPixelBuffer pixelBuffer) async {
  // Convert the CVPixelBuffer to a Flutter image object.
  ui.Image image = await FlutterVision.imageFromPixelBuffer(pixelBuffer);
  return image;
}