Jaybird SocketInputStream.socketread0 thread blocking

Question

I am using JDBC connection to firebird.

<groupId>org.firebirdsql.jdbc</groupId>
<artifactId>jaybird-jdk18</artifactId>
<version>3.0.5</version>

When I am reading data with:

resultSet.next(); //resultSet is instance of FBResultSet

sometimes thread will block here:

 private native int socketRead0(FileDescriptor fd,
                                   byte b[], int off, int len,
                                   int timeout)
        throws IOException;

And the thread will never continue. I need to restart whole app. :(

How can I fix this issue? I have tried some parameters from here: https://dzone.com/articles/threads-stuck-in-javanetsocketinputstreamsocketrea

-Dsun.net.client.defaultConnectTimeout=10000
-Dsun.net.client.defaultReadTimeout=10000

But without positive result. I dont even know, what the timeout value is. I cant debug it in that place. (java.net.SocketInputStream)

The connection to the database is connected to the cloud server, and I think that is the problem.

Is it possible to somehow fix this issue? Or setup the timeout to firebird jdbc?

Thanks for any answer.

Edit: here is the screen of jammed thread:

Answer 1

The root cause in your case is a bug in Jaybird 3.0.4 to 3.0.7 (and 4.0.0-beta-1) with Firebird 3 (and 4) when wire encryption is enabled in firebird.conf (setting WireCrypt = Enabled).

I had previously only seen this problem with Jaybird 4 when the tests of Jaybird were run in a specific order, and as that version is not yet released, I hadn't given it priority to find the root cause.

With your help (thanks again!) I was able to identify the problem. This bug is now fixed in Jaybird 3.0.8, which is available from Firebird: JDBC Driver.

Explanation of the bug

Certain types of buffers (eg column data) in the Firebird wire protocol are padded with bytes to multiples of four. The implementation in Jaybird relied on InputStream.skip(long) to skip this padding.

Specifically, it did:

public int skipFully(int n) throws IOException {
    int total = 0;
    int cur;
    while (total < n && (cur = (int) in.skip(n - total)) > 0) {
        total += cur;
    }
    return total;
}

This worked fine without wire encryption, because the combination of the skip implementations in BufferedInputStream and SocketInputStream would at least skip 1 byte for each call to skip unless the socket was closed.

When CipherInputStream was added, this expectation no longer held: if the BufferedInputStream has no bytes in its buffer, it will call skip(long) on the CipherInputStream, and if that is at the end of its buffer, it will skip 0 bytes (which is allowed for skip).

As a result, Jaybird could skip 1, 2 or 3 bytes too few, which caused subsequent reads to read wrong data. Eventually this would either cause a buffer size to be read incorrectly, causing the read to block waiting for more data, or it would result in Jaybird reading a wrong operation code, which will cause an exception with a message starting with "Unsupported or unexpected operation code".

Workarounds or solutions for other problems with similar behaviour

As a workaround to hanging connections, you can use the connection property soTimeout to set the socket read timeout (see Jaybird Manual, appendix A.2 Other properties).

The problem could also be that Firebird is 'just' waiting indefinitely on a lock. To rule out Firebird waiting on a lock, you could also change the transaction wait mode or timeout. The default for Jaybird is to use an indefinite wait.

For example, to change the wait mode of read committed to no-wait, you can set connection property TRANSACTION_READ_COMMITTED with value read_committed,rec_version,write,nowait.

To use wait, but with a timeout, you can use value read_committed,rec_version,write,wait,lock_timeout=5. This will apply a 5 second lock timeout.

If your problem is due to Firebird waiting on a lock held by another transaction, then this should result in an exception with the text containing lock conflict on no wait transaction (for the nowait option) or lock time-out on wait transaction (for the lock_timeout option).

See also Jaybird Manual, appendix A.3 Transaction isolation levels and Jaybird Manual, section 6.4. Transaction Isolation Levels (although I now see that I will need to add more information in there).