Previous discussion is on Memoization in J which doesn't answer the issue. The problem is that using memoization M. somehow makes the program much slower.
I encountered the same problem. Consider this example:
collatz =: (%&2)`(1+3&*)@.(2&|)
countchain =: 3 : 0 M.
echo 'xx ' , ": y
if. (1=y) do. (1) else. (1 + countchain collatz y) end.
)
The countchain above will tell you how many collatz steps it took for the number to get to 1. The echo part will show you if the memoization works. And, indeed, it appears to be working:
countchain"0 (1 2 3 4 5)
xx 1
xx 2
xx 3
xx 10
xx 5
xx 16
xx 8
xx 4
1 2 8 3 6
As you can see, the countchain method is not repeated for the same number. However, when I tried it on longer sequence of numbers (e.g. (1+i.1000), and pasting the output to a text file, I observed that there are lots of repetitions. Which mean the program was executed on values it has seen before. I also noticed that the repetition appeared exactly after 32 values have been executed, so the problem appear to be from memory. The documentation also mentioned that "The size of the table used to record arguments and results is not under user control." Is there really no way to alleviate this problem?