I asked this question over at the rakulang sub reddit and was suggested to post here:
I keep falling back to Perl 5 for a lot of my work so I can "get it done" simply because I am so much more familiar with Perl 5.
However, I have the need to build something that will subscribe to multiple MQTT topics (similar conceptually to a websocket subscription) and process data, keeping a lot of this data as internal state. A concurrency project. So I see this as a great opportunity to immerse myself in some Raku :)
So far I understand I need to create a supply / given / when setup, but I'm not totally sure how I will deal with each stream of data's state received over each topic. And a reply from my reddit post suggested Cro, which I think will fit the bill very nicely. But there are still some implementation details I am unclear on.
For example, a message payload arrives on topic foo, I want to add data from that payload to an existing array (my internal state).
But this subscription to topics will be happening for an 'undetermined' number topics, and will adjust at runtime. So it will not be possible to have a hard coded array to store and manage this data in like @foo
In a non concurrent world, I could create a hash (associative array) with a key that matches my topic name, %data<foo> for example, and store the array there.
However in the world of concurrency, I would need an answer to the mutex problem. If each member of the hash is having it's data modified concurrently by different threads, then I would think the entire hash would require a lock.
This has potential to result in a deadlock or poor performance at the very least (I am expecting some several hundred messages per second, across multiple topic subscriptions).
Perhaps I can create a variable 'dynamically' (or better yet, object) based the topic name, so there is a separate memory address for each array of data. However, I'm not sure how to do that, or indeed if that is the 'best' approach in this scenario.
In summary, Question 1: Is creating an object or variable dynamically for this purpose a sound pattern? Question 2: Is there a design approach I am simply not aware of that would be more suitable?
So, any specific advice would be greatly appreciated. I feel like this is a case of "I don't know what I don't know" type of problem!
Thanks!