QML app: How to capture camera image in-memory

Question

I have a QML-based app where I need to capture images from the camera in order to do QR-code recognition/decoding (using qzxing).

Following the example for the CameraCapture QML class, I can capture images, however they will always be saved to a local file. I don't want to save to a file since I don't want to stress the underlying storage (flash memory, sd card) in order to do QR code recognition. Thus, I want to grab camera images in-memory. Googling for this turned out it seems to be impossible with QML-only.

So I was looking for a C++ based solution, however I cannot seem to come up with a working solution.

Currently I have code which tries to capture the image in C++ while providing a visible preview of the camera for the user in QML:

QML

   ImageReader {
      id: reader
   }

   Rectangle {
      width: 400
      height: 400
      x: 30; y: 90

      Camera {
         id: camera
      }

      VideoOutput {
         source: camera
         focus: visible
         anchors.fill: parent
      }
   }

   Timer {
      id: scanTimer
      interval: 3000
      running: true
      repeat: false

      onTriggered: {
         console.log("capture image")
         reader.startCapture()
      }
   }

C++

ImageReader::ImageReader(QObject *parent) : QObject(parent)
{
   doCapture = true;
   camera = new QCamera();
   capture = new QCameraImageCapture(camera);
   capture->setCaptureDestination(QCameraImageCapture::CaptureToBuffer);

   QObject::connect(capture, &QCameraImageCapture::imageCaptured, [=] (int id, QImage img) {
      qDebug() << "Captured image";
      camera->stop();
   });

   QObject::connect(capture, &QCameraImageCapture::readyForCaptureChanged, [=] (bool state) {
      qDebug() << "Ready for capture changed: " << state;

      if (!doCapture)
         return;

      if(state == true) {
         qDebug() << "is ready for capture";
         camera->searchAndLock();
         capture->capture();
         camera->unlock();

         doCapture = false;
      }
   });
}

Q_INVOKABLE void ImageReader::startCapture()
{
   camera->start();
}

There are several issues with this approach:

1. imageCaptured is never triggered, and the lambda is never called, so no image is captured
2. readyForCaptureChange is called, however, the preview (QML code) goes all white and never goes back to a live-video from the camera
3. in the console I get several warnings/the following output:

** (myapp:24700): WARNING **: 19:26:09.403: capsfilter2: transform_caps returned caps which are not a real subset of the filter caps

** (myapp:24700): WARNING **: 19:26:09.406: capsfilter2: transform_caps returned caps which are not a real subset of the filter caps

** (myapp:24700): WARNING **: 19:26:09.412: imagebin-capsfilter: transform_caps returned caps which are not a real subset of the filter caps

** (myapp:24700): WARNING **: 19:26:09.426: viewfinderbin-capsfilter: transform_caps returned caps which are not a real subset of the filter caps
[D] onTriggered:65 - capture image
[W] unknown:0 - Starting camera without viewfinder available

** (myapp:24700): WARNING **: 19:26:12.463: capsfilter8: transform_caps returned caps which are not a real subset of the filter caps

** (myapp:24700): WARNING **: 19:26:12.469: capsfilter8: transform_caps returned caps which are not a real subset of the filter caps

** (myapp:24700): WARNING **: 19:26:12.483: imagebin-capsfilter: transform_caps returned caps which are not a real subset of the filter caps

** (myapp:24700): WARNING **: 19:26:12.495: viewfinderbin-capsfilter: transform_caps returned caps which are not a real subset of the filter caps
[D] ImageReader::ImageReader(QObject*)::<lambda:26 - Ready for capture changed:  true
[D] ImageReader::ImageReader(QObject*)::<lambda:32 - is ready for capture
[D] ImageReader::ImageReader(QObject*)::<lambda:26 - Ready for capture changed:  false
[D] ImageReader::ImageReader(QObject*)::<lambda:26 - Ready for capture changed:  true

Can anyone guide me to a working solution to achieve the following:

• show a preview (video) of the camera to the user
• capture a picture from the camera so I can trigger QR-code decoding
• must work with existing QML interface implementation (but can contain C++ code, no problem)

I am running this on a XPeria 10 ii phone with SailfishOS 4.1.0.24 installed. Thus, QT version should be 5.2.

Answer 1

You can access the QCamera using the mediaObject property of the Camera item and then use the QCameraImageCapture where the QCameraImageCapture::CaptureToBuffer mode is set and use the imageCaptured signal to get the QImage.

#include <QCamera>
#include <QCameraImageCapture>
#include <QGuiApplication>
#include <QQmlApplicationEngine>
#include <QQmlContext>

class CameraHelper: public QObject{
    Q_OBJECT
    Q_PROPERTY(QObject* qcamera READ qcamera WRITE setQcamera NOTIFY qcameraChanged)
public:
    using QObject::QObject;
    QObject *qcamera() const{
        return q_qcamera;
    }
    void setQcamera(QObject *qcamera)
    {
        if (q_qcamera == qcamera)
            return;
        if(m_capture){
            m_capture->deleteLater();
        }
        q_qcamera = qcamera;
        if(q_qcamera){
            if(QCamera *camera = qvariant_cast<QCamera *>(q_qcamera->property("mediaObject"))){
                m_capture = new QCameraImageCapture(camera, this);
                m_capture->setCaptureDestination(QCameraImageCapture::CaptureToBuffer);
                connect(m_capture, &QCameraImageCapture::imageCaptured, this, &CameraHelper::handleImageCaptured);
            }
            else
                m_capture = nullptr;
        }
        emit qcameraChanged();
    }
public Q_SLOTS:
    void capture(){
        if(m_capture)
            m_capture->capture();
    }
Q_SIGNALS:
    void qcameraChanged();
    void imageChanged(const QImage & image);
private:
    void handleImageCaptured(int , const QImage &preview){
        Q_EMIT imageChanged(preview);
    }
    QPointer<QObject> q_qcamera;
    QCameraImageCapture *m_capture = nullptr;
};

int main(int argc, char *argv[])
{
#if QT_VERSION < QT_VERSION_CHECK(6, 0, 0)
    QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
#endif
    QGuiApplication app(argc, argv);
    CameraHelper camera_helper;
    QObject::connect(&camera_helper, &CameraHelper::imageChanged, [](const QImage & qimage){
        qDebug() << qimage;
    });
    QQmlApplicationEngine engine;
    engine.rootContext()->setContextProperty("cameraHelper", &camera_helper);
    const QUrl url(QStringLiteral("qrc:/main.qml"));
    QObject::connect(&engine, &QQmlApplicationEngine::objectCreated,
                     &app, [url](QObject *obj, const QUrl &objUrl) {
        if (!obj && url == objUrl)
            QCoreApplication::exit(-1);
    }, Qt::QueuedConnection);
    engine.load(url);
    return app.exec();
}

#include "main.moc"

import QtQuick 2.12
import QtQuick.Window 2.12
import QtMultimedia 5.15

Window {
    width: 640
    height: 480
    visible: true
    Camera {
        id: camera
        Component.onCompleted: cameraHelper.qcamera = camera
    }
    VideoOutput {
        source: camera
        focus : visible // to receive focus and capture key events when visible
        anchors.fill: parent

        MouseArea {
            anchors.fill: parent;
            onClicked: cameraHelper.capture();
        }
    }
}