Use async c++/qt functions in exported python module

Question

I have a dll that is relying extensively on Qt and that exports certain async functions returning QFuture, e.g.

QFuture<QString> loadUserAsync();

I want to export such functions (through pybind11) so that customers can write scripts in python. I don't want to leak the async Qt interface into python though, hence I am writing a wrapping API in C++, something like that:

class API_EXPORT API {
public:
   std::string loadUsername();
   //...
};

std::string API::loadUsername() {
   Future<QString> future = _core->loadUserAsync();
   return future.result().toStdString(); 
}

which then gets exported through pybind11:

py::class_<API>(m, "api")
  .def(py::init<>())
  .def("loadUsername", &API::loadUsername);

Well, this has multiple issues and I am struggling how to approach this correctly.

First, I most certainly need to instantiate a QCoreApplication so that signal/slot and events within the library are working correctly. This seems to work but I am really not sure if this is considered best practise and if I have to call the exec function (I cannot call exec on the calling thread, else it will block):

API::API() {
    if (!QCoreApplication::instance()) {
      int argc = 1;
      char* argv[] = {"api"};
      _qt = std::make_shared<QCoreApplication>(argc, argv);
    }
}

Second, future.result().toStdString(); deadlocks. I could "fix" this instantiating my own QEventLoop but I am not sure if this is the way to go:

QFutureWatcher<void> watcher;
QEventLoop loop;

watcher.connect(&watcher, SIGNAL(finished()), &loop, SLOT(quit()), Qt::QueuedConnection);
watcher.setFuture(future);

loop.exec();

Third, somewhere within the dll a QTimer is instantiated so that I am getting nasty warnings printed in python and I am puzzled what to do about it:

QObject::startTimer: Timers can only be used with threads started with QThread

Answer 1

Please let me preface this by stating I avoid Python like the plague.

Did you get any of this to work in C++ land sans any Python? I'm asking because this:

std::string API::loadUsername() {
   Future<QString> future = _core->loadUserAsync();
   return future.result().toStdString(); 
}

cannot possibly work.

Yes, I left a comment earlier. One cannot "sprinkle in a dash of Qt" because Qt is an application framework. It is also an application framework that suffers greatly from single-threadiness in that almost nothing is tested from outside the main event loop. They will also bemoan that running multiple event loops is an anti-pattern. (It's application reality, but that is a different argument.)

The entire point of QFuture is to use a thread from the thread pool and emit a signal when that function/task ends either successfully or tragically.

If your return statement actually returns the value you want then you don't need QFuture at all because you have blocking I/O happening.

Btw, this:

API::API() {
    if (!QCoreApplication::instance()) {
      int argc = 1;
      char* argv[] = {"api"};
      _qt = std::make_shared<QCoreApplication>(argc, argv);
    }
}

is unbelievably dangerous given all of then ancient x86 and lower processor lore about the first argument being the full path to the executable. There is stuff that gets setup in QCoreApplication like applicationFilePath() that kinda-sorta relies on that.

Here are some posts where I used QFuture in a lottery tracker application because the database I/O could potentially be long.

https://www.logikalsolutions.com/wordpress/information-technology/how-far-weve-come-pt-12/ https://www.logikalsolutions.com/wordpress/information-technology/how-far-weve-come-pt-13/ https://www.logikalsolutions.com/wordpress/information-technology/how-far-weve-come-pt-14/

https://www.logikalsolutions.com/wordpress/information-technology/how-far-weve-come-pt-16/

Part 16 has the code I'm going to paste below, but you need to read 12, 13, and 14 to understand the objective.

void DataBaseIO::top12Report()
{
    QFuture future = QtConcurrent::run(this, &DataBaseIO::detachedTop12);
}

void DataBaseIO::detachedTop12()
{
    QString msgTxt;
    QTextStream rpt(&msgTxt);

    QSqlQuery q(db);

    rpt << "Number   hit_count" << endl
        << "--------- ---------" << endl;

    rpt.setFieldWidth(9);
    rpt.setFieldAlignment(QTextStream::AlignRight);

    q.exec("select elm_no, count(*) as hit_count from drawings group by elm_no order by hit_count desc limit 12;");

    while (q.next())
    {
        QSqlRecord rec = q.record();
        int no = rec.field("elm_no").value().toInt();
        int hits = rec.field("hit_count").value().toInt();
        rpt << no << hits << endl;
    }

    rpt.flush();

    emit displayReport( "Top 12 Report", msgTxt);
}

The simplest way to use QFuture is via run(). No watcher required. At the end of your task emit a signal. Admittedly you have no way of knowing if this fell over because you have no watcher.

Btw, your timer is coming from QFutureWatcher because it has a timerEvent(). https://doc.qt.io/qt-5/qfuturewatcher-members.html

Once you get that working in C++ only land, you need to read up on C++ signals and Python.

https://www.tutorialspoint.com/pyqt5/pyqt5_signals_and_slots.htm

https://www.mfitzp.com/tutorials/pyqt-signals-slots-events/

There is no way around the application having to have something like this.

import sys
from PyQt5.QtWidgets import QApplication

app = QApplication(sys.argv)

app.exec()

Syntax may not be perfect because I avoid Python.

You can't just toss in a dash of Qt. Whoever uses what you are developing will have to use the entire framework.

There is a limited subset of Qt that does not require a main event loop. I could not find a list of the things. The project might no longer publish it. The reality is you cannot do much without the main event loop. When you emit a signal it has to have an event loop so it can be placed on the event queue (assuming it doesn't direct connect). With C++ emit is really a direct function call much of the time. I don't know about the world of Python. I would assume it has to be a queued event.

As a rule of thumb, if you are using any class that has signals, you have to have a main event loop running. If you are allowing Python into the mix, then Python has to start the main event loop. If it doesn't it has no ability to communicate with Qt.

Here is a very detailed tutorial on how to use C/C++ from Python. https://realpython.com/python-bindings-overview/

The bottom line is that unless you go all-in on Qt, you cannot do what you want.

Most likely you could do what you are trying to do using pure C++ depending on your ability to rewrite loadUserAsync() to be a direct I/O blocking function sans any Qt.