Reading continuously incoming data (usb4java)

Question

I have spent a considerable amount of time trying to figure out the best way of reading continuously a large amount of data using the Low-level functions of usb4java (Libusb).

The amount of data I need to read is 640kbyte/s in a Full Speed device, theoretically is possible, and I should be using less than half of the bandwidth available. The problem I am having is that the data I am reading has glitches that could be coming from either missing or corrupt data.

I have tried both Sync and Async with similar results. Here I post the code I am using to do this in Async mode, any help is appreciated.

public Void doInBackground() {

    loop = true;
    handle = comm_device_async.gethandle();

    buffer = BufferUtils.allocateByteBuffer(PacketSize).order(ByteOrder.LITTLE_ENDIAN);
    Transfer transfer = LibUsb.allocTransfer();

    LibUsb.fillBulkTransfer(transfer, handle, IN_ENDPOINT, buffer, read_callback, null, TIMEOUT);
    int result = LibUsb.submitTransfer(transfer);
    if (result != LibUsb.SUCCESS) {
        throw new LibUsbException("Unable to submit transfer", result);
    }

    while (loop) {
        synchronized (synchObj) {
            while (!transfercompleted) {
                try {
                    synchObj.wait();
                } catch (InterruptedException ex) {
                    Logger.getLogger(GraphPanel_JChart2D.class.getName()).log(Level.SEVERE, null, ex);
                }
            }
        }
        transfercompleted = false;

        multipledatashort[readcyclecount] = read_callback_data;
        readcyclecount++;
        if (readcyclecount == readcycles) {
            synchronized (dataListShort) {
                dataListShort.add(multipledatashort);
                dataListShort.notify();
            }
            readcyclecount = 0;
        }

    }
    return null;
}

TransferCallback read_callback = new TransferCallback() {

    ByteBuffer buffer;
    long startTime = 0;

    @Override
    public void processTransfer(Transfer transfer) {
        System.out.println("ReadCallback loop time " + (System.nanoTime() / 1000 - startTime));
        startTime = System.nanoTime() / 1000;
        read_callback_data = new short[transfer.buffer().capacity() / 2];
        for (int i = 0; i < read_callback_data.length; i++)
            read_callback_data[i] = transfer.buffer().getShort();


        synchronized (synchObj) {
            transfercompleted = true;
            synchObj.notify();
        }

        buffer = BufferUtils.allocateByteBuffer(PacketSize).order(
                ByteOrder.LITTLE_ENDIAN);
        LibUsb.fillBulkTransfer(transfer, collectWorker_usb4java_async_fast.handle, IN_ENDPOINT, buffer,
                read_callback, null, TIMEOUT);
        int result = LibUsb.submitTransfer(transfer);
        if (result != LibUsb.SUCCESS) {
            throw new LibUsbException("Unable to submit transfer", result);
        }

    }
};

Answer 1

I found a thread that describes pretty a similar issue

http://libusb.6.n5.nabble.com/Fwd-FT2232H-asynchronous-maximum-data-rates-td4519549.html

There are two main points

1. He is saying submitting a single request asynchronously (and resubmitting it) turns out to work worse than submitting synchronous requests. This is expected, doing more busywork takes more time.

2. the goal is to keep the list of URB's at the kernel constantly filled. Anytime that list empties, there's a chance it won't be refilled in time to request data and the FT2232H buffer will fill up. So the right way to do the asynchronous mode is to make a list of transfers. Each time one of them returns, another is requested. There's overhead involved in converting this request into the corresponding list of URBs, but as long as the list starts out big enough, we can amortize this overhead over the time the other transfers take to complete

So basically for large amount of data we need to have a buffer of transactions to be submitted, so there is always one pending when the last one ends.

I am going to be working on that, but I haven't found any good examples of how this should be done using usb4java. Again, any help is appreciated