Insufficient data written when inserting rows

Question

I am facing this error when running my unit test to insert some rows into my bigquery table today :

Caused by: java.io.IOException: insufficient data written
at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.close(HttpURLConnection.java:3213)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:81)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:960)
at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequest(MediaHttpUploader.java:482)
at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequestWithBackOffAndGZip(MediaHttpUploader.java:504)
at com.google.api.client.googleapis.media.MediaHttpUploader.executeUploadInitiation(MediaHttpUploader.java:456)
at com.google.api.client.googleapis.media.MediaHttpUploader.upload(MediaHttpUploader.java:348)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:418)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)

I thought it was due to the new version of google-http-client (1.16.0.rc) because i updated just before running the test. But rollbacking to 1.15.0-rc has no effect.

Any idea ?

Answer 1

These errors usually happen when there is a communication failure, specially for large files. The way to avoid it is to use resumable upload.

Answer 2

Me too. Also, it seems like a sign that Bigquery just stops receiving any data. Because if you query you table by count(*) after this exception, result won't change anymore. If I keep my program running for a while, it will give me errors such as:

javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1343)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1371)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1355)

waiting for answers...