writeTextElement() of QXmlStreamWriter crashed when writting the content of a big file

Question

Here is the qt code that crash:

Writing the content of a 130MB binary file will crash, but writing a 2MB binary file will not.

Is there any solution for writeTextElement to write big file content? Thanks

env: qt opensource 4.8.7 Visual Studio 2010 Windows 10

big file: https://1drv.ms/u/s!Ap_EAuwC9QkXijOmXqxp9DEav3gm?e=iha0uI

test project: https://1drv.ms/u/s!Ap_EAuwC9QkXijWzOlpaWmtzOdtz?e=fDpo93

#include <QtCore/QCoreApplication>
#include <QFile>
#include <QXmlStreamWriter>


int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);

    QByteArray mContentBuffer;

    QFile file("C:\\Work\\bigfile.xar");
   
    if(!file.open(QFile::ReadOnly))
    {
        return -1;
    }

    mContentBuffer = file.readAll();
    file.close();

    QFile profile("C:\\Work\\profile.xml");

    if(!profile.open(QFile::WriteOnly|QFile::Truncate))
    {
        return -1;
    }

    QXmlStreamWriter stream(&profile);    

    stream.setAutoFormatting(true);
    stream.writeStartDocument();
    stream.writeStartElement("Profile");

    stream.writeTextElement("Content", mContentBuffer.toBase64());

    stream.writeEndElement(); // Profile
    stream.writeEndDocument();

    return a.exec();
}

Answer 1

The minimal reproducible example was a life saver.

Several things seem to be required to solve your issue in a clean way.

1. Mainly, the file must be read in chunks.
   To be precise and as mentioned in that other question, in chunks whose size is a multiple of 3 bytes (I have hardcoded 9000 below).
   You will need to look for the value that gives you the best performance while guaranteeing it never fails yourself.
2. Instead of writing in the XML in 1 go, I made it so the XML file is being written each chunk at a time.
   To achieve that, the code creates the node first, then uses the writeCharacters method to write text in it.
3. Note the if (mContentBuffer.isEmpty()) right before writing. It is indeed specified in Qt Assistant that the read function has no way of reporting errors; returning an empty QByteArray can mean either that no data was currently available for reading, or that an error occurred. Considering the while loop, it can only be an error, so we abort everything.
   BTW, I think but have not tested that you can just end the document, and the XML stream writer will close everything it needs to have a valid XML.
4. Just to make it clean, I reordered the lines of codes so that the XML file is opened before the xar file. I also added QtCore/ on every include. Not sure that last point is going to be useful on your real code.

This code implements all the above changes.
It should not crash and while you will not be able to test the output on the 130MB file, you should be able to test your code vs mine on the 2MB file you has successfully read from before.

#include <QtCore/QCoreApplication>
#include <QtCore/QFile>
#include <QtCore/QXmlStreamWriter>

int main(int argc, char* argv[])
{
    QCoreApplication a(argc, argv);

    QFile profile("C:\\Work\\profile.xml");
    if (!profile.open(QFile::WriteOnly | QFile::Truncate))
        return -1;

    QXmlStreamWriter stream(&profile);
    stream.setAutoFormatting(true);
    stream.writeStartDocument();
    stream.writeStartElement("Profile");
    stream.writeStartElement("Content");

    QFile file("C:\\Work\\bigfile.xar");
    if (!file.open(QFile::ReadOnly))
        return -1;

    while (!file.atEnd()) {
        QByteArray mContentBuffer = file.read(9000);
        if (mContentBuffer.isEmpty()) {
            stream.writeEndDocument();
            file.close();
            profile.close();
            return -1;
        }
        stream.writeCharacters(mContentBuffer.toBase64());
    }
    file.close();

    stream.writeEndElement(); // </Content>
    stream.writeEndElement(); // </Profile>
    stream.writeEndDocument();

    return a.exec();
}