JedisConnectionException Read timed out intermittently

5.5k Views Asked by At

My application is running on ECS cluster and Redis is ap part of docker on ECS . The application runs fine for a week or more but all of a sudden it started throwing Time out exception .

The issue reported in below block of query

api.query("MATCH (ag:dGrp{v:" + rec.DocGroupId + "}),(pg:resUGrp{v:" + rec.userGroupUID + "}) CREATE (pg)-[:dgE{ppv:" + ppv + ","+IdentifierFlag+":1}]->(ag)");

Full stack Trace

redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
    at redis.clients.jedis.util.RedisInputStream.ensureFill(RedisInputStream.java:205)
    at redis.clients.jedis.util.RedisInputStream.readByte(RedisInputStream.java:43)
    at redis.clients.jedis.Protocol.process(Protocol.java:155)
    at redis.clients.jedis.Protocol.read(Protocol.java:220)
    at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:283)
    at redis.clients.jedis.Connection.getOne(Connection.java:261)
    at redis.clients.jedis.Jedis.sendCommand(Jedis.java:4119)
    at com.redislabs.redisgraph.impl.api.ContextedRedisGraph.sendQuery(ContextedRedisGraph.java:52)
    at com.redislabs.redisgraph.impl.api.RedisGraph.sendQuery(RedisGraph.java:68)
    at com.redislabs.redisgraph.impl.api.AbstractRedisGraph.query(AbstractRedisGraph.java:46)

and this

Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
    at java.net.SocketInputStream.read(SocketInputStream.java:171)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at java.net.SocketInputStream.read(SocketInputStream.java:127)
    at redis.clients.jedis.util.RedisInputStream.ensureFill(RedisInputStream.java:199)

when we restart our ECS task the problem disappear and comes back after a week . We have increased max connection to 160 .

When we try to re produce this issue in Lower env by putting heavy request and in bulk for a week also but this issue we were not able to re produce . We are using Radis 3.5.1 version .

jedis redis -client.jar(version 3.5.1)

Redis Time out Configuration set in redis.conf file:

timeout 0
bind 127.0.0.1
tcp-backlog 511
tcp-keepalive 300
lua-time-limit 5000
loadmodule /usr/lib64/redis/modules/redisgraph.so

Socket Time out in my Code

socket.setSoTimeout(60000);//keep connection for max 60s when idle

this time out is from import java.net.Socket;

RedisGraph Query

And at Reis Graph Query Level we use this Method that does not Time out parameter so i assume it might be taking default one .

/**
* Execute a Cypher query.
* @param graphId a graph to perform the query on
* @param query Cypher query
* @return a result set
*/
public ResultSet query(String graphId, String query) {
return sendQuery(graphId, query);
}

But it has one method that has Timeout parameter also

/**
* Execute a Cypher query with timeout.
* @param graphId a graph to perform the query on
* @param timeout
* @param query Cypher query
* @return a result set
*/
@Override
public ResultSet query(String graphId, String query, long timeout) {
return sendQuery(graphId, query, timeout);
}
1

There are 1 best solutions below

7
sazzad On

I see you are doing graph query which could take more time than general Redis commands. So, for this purpose you can try increasing socket timeout.