Why does a Redis Gears Funcion fail with "Operation not permitted" when it writes data to another Shard?

51 Views Asked by At

(I posted this issue to RedisGears GitHub repo too.)

I created a very simple Redis Gears function as follows.

def failed_function(x):
    execute('SET', 'A', 'SOMEVALUE')
    
GearsBuilder('KeysReader').foreach(failed_function).run('a')

I created an entry using command: SET a somevalue in a Redis cluster with three shards, and then I ran the function. It reported 'Operation not permitted.'

$ cat debug.py | redis-cli -c -p 7000 -x rg.pyexe
cute
1) (empty array)
2) 1) "['Traceback (most recent call last):\\n', '  File \"<string>\", line 2, in failed_function\\n', 'gears.error: Operation not permitted\\n']"

But If I just change one byte of the code: 'A'->'a', then run it in the same environment. It works fine.

def ok_function(x):
    execute('SET', 'a', 'SOMEVALUE')
    
GearsBuilder('KeysReader').foreach(failed_function).run('a')

Obviously, the issue is about the sharding: Key 'A' and 'a' are in different shards. It turns out that if I write the data in another shard, this operation is not permitted.

127.0.0.1:7000> get 'a'
-> Redirected to slot [15495] located at 127.0.0.1:7002
"SOMEVALUE"
127.0.0.1:7002> get 'A'
-> Redirected to slot [6373] located at 127.0.0.1:7001

Does Redis Gears enforce that a function can only access local data? Or did I miss anything? If it is the former, how does Redis handle a complicated application where the stages of a pipeline are hosted in different nodes/shards?

More information:

  • I'm using docker image redislabs/redisgears:latest, which is about 2-years old (called 'latest' though).
  • I planned to try image redislabs/redisgears:2.0.19-m20. But I haven't found the API document of RedisGear2. So I haven't tested it yet.
  • cluster information:
127.0.0.1:7001> RG.INFOCLUSTER
1) "MyId"
2) "4cd5dbe8debbddc23764e65e35dade5f46774d37"
3) "MyRunId"
4) "afcb8bbc6bbc4ada6a36ca3a6f936a8216051d4f"
5) 1)  1) "id"
       2) "949dc5f01a31d64ada4e8eb5c2ca024b1f408512"
       3) "ip"
       4) "127.0.0.1"
       5) "port"
       6) (integer) 7002
       7) "unixSocket"
       8) "None"
       9) "runid"
      10) "994f068609a1a75220b945590ff64b8668f8ae3f"
      11) "minHslot"
      12) (integer) 10923
      13) "maxHslot"
      14) (integer) 16383
      15) "pendingMessages"
      16) (integer) 0
   2)  1) "id"
       2) "4cd5dbe8debbddc23764e65e35dade5f46774d37"
       3) "ip"
       4) "127.0.0.1"
       5) "port"
       6) (integer) 7001
       7) "unixSocket"
       8) "None"
       9) "runid"
      10) "afcb8bbc6bbc4ada6a36ca3a6f936a8216051d4f"
      11) "minHslot"
      12) (integer) 5461
      13) "maxHslot"
      14) (integer) 10922
      15) "pendingMessages"
      16) (integer) 0
   3)  1) "id"
       2) "ec6b13c723a2ee1b1e1ee682e2d070b9a0a1f1c4"
       3) "ip"
       4) "127.0.0.1"
       5) "port"
       6) (integer) 7000
       7) "unixSocket"
       8) "None"
       9) "runid"
      10) "bea227c3cf4edf80adebec764e41d93cd4f69cc8"
      11) "minHslot"
      12) (integer) 0
      13) "maxHslot"
      14) (integer) 5460
      15) "pendingMessages"
      16) (integer) 0
0

There are 0 best solutions below