debugging USB read

Question

I am debugging some code that uses a 3rd party 64-bit DLL to access a custom USB device. My environment is Microsoft Visual Studio 2012 on Windows 8.1 x64.

According to an incomplete and unreliable document, the DLL is supposed to issue a USBDEVFS_CONTROL ioctl to read 1 byte from a connected USB device. The definition involves

ctrl.bRequestType = bmRequestType;
ctrl.bRequest     = bRequest;
ctrl.wValue       = wValue;
ctrl.wIndex       = wIndex;
ctrl.data         = ByteArray;
ctrl.wLength      = 64;
ctrl.timeout      = 1000;

Here bmRequestType, bRequest, wValue, and wIndex are constants provided by the device manufacturer, and ByteArray is a uint8_t[64] buffer that contains the specific command.

The DLL accepts application-specific parameters, packs them into the ByteArray, and calls ksproxy.ax->Kernelbase.dll->ntdll.dll. The last disassembly I can see in user mode, is

mov     r10,rcx
mov     eax,47h
syscall
ret

With step-by-step debugger, I can easily see that the ByteArray is constructed exactly as it is supposed to be, according to the document. But I cannot find the usbdevfs_ctrltransfer structure, or its Windows equivalent.

Specifically, we suspect that the value of wIndex, specified in the document, applies to an older version of hardware, and that the Windows DLL actually uses 0x0400 instead of 0x0402.

Any hint (including hardware or software USB sniffers, emulators, etc.) how we can try to verify this unsigned short will be greatly appreciated.

Update

Reading https://reverseengineering.stackexchange.com/questions/2416/how-to-reverse-engineer-simple-usb-device-windows-linux and https://reverseengineering.stackexchange.com/questions/1786/usb-dongle-traffic-monitoring. It looks like these tools are not compatible with Windows 8.1 x64.

Answer 1

While working on the Xbox OS and peripherals, we always used the CATC Chief USB capture hardware, which works as a man-in-the-middle device (it looks like it has been superseded by the Teledyne LeCroy protocol analyzers).

The traffic capture capabilities were indispensable in diagnosing hardware and software errors (bulk, HID, isoch).

Sample capture view (from the manual):