krakend rate limit working for 1st time but second time onwards its behaving weiredly

Question

{
"version": 3,
"name": "My lovely gateway",
"port": 8084,
"timeout": "3s",
"extra_config": {
  
},
"endpoints": [
{
  "endpoint": "/rate",
  "method": "GET",
  "output_encoding": "json",
  "extra_config": {
    "qos/ratelimit/router": {
      "max_rate": 5,
      "every": "1m",
      "capacity": 5
      }
  },
  "backend": [
    {
      "url_pattern": "/rate",
      "method": "GET",
     "host": [
          "http://127.0.0.1:8091/"
      ]
    }
  ],
  "input_query_strings":[
     "*"
  ],
  
  "input_headers": [
    "*"
  ]
}
]
}

This is my krakend ratelimit configuration.I tried to achieve max 5 request in 1 minute .It's working fine for 1st time when I hit consecuively 5 apis from Postman.After 5 request it is giving 503 service unavailable.

Now After 2/3 minutes when I again hit consecutively,it's taking 7 request before sending 503 response code. Again after waiting max 30sec,if I hit the server ,it's taking 2/3 requests.

Also After restarting the server If we wait for 1/2 minutes and started hitting the server,it's taking 12/13 requests. Can anyone tell me the reason behind this behavior.I'm using version 2.4.3.

Answer 1

Probably you are expecting that within an absolute minute, you can receive 5 requests in each, and 5 more in the next minute. But this is not how the algorithm used in the rate limiting works. KrakenD uses an implementation called Token Bucket (read their documentation) that is very efficient an popular in many systems.

Not trying to reproduce the documentation here, but the idea is that the rate you write, is how many requests you will allow in an amount of time. If you have set 5 every 1 minute, it means that 60s/5req=12seconds is a new request, but it will never surpass the capacity.

Postman is not the best way to test the rate limiting, since you are not using a clock to make them, but as you can see if your test with postman lasted more than one minute, for every 12 seconds extra you can do a new request.

The important thing here is that a user can consume the 5 requests in a microsecond, but it takes one minute to be able to do 5 more.