I have a ft232h (sync FIFO), and a Rasberry Pi connected to it via USB 2.0. I found that there is always a delay equal to the value of the latency timer, even if the requested amount of data is ready to be transmitted. So, my read speed is only 60Mbps, but my write speed is quite normal and is about 320Mbps. As I undrerstand it happens because I read one frame per 1ms (latency timeout), when USB 2.0 allows me to read one frame per 125 us.
I tested on several Latency values from 1 to 255 ms, but the number of bytes received is always the same. I tried changing flow control but no positive results even though I disabled flow control. I tried to use libftdi as well, that is told to work better on Linux. It does work better, but there is the same delay.