I am working on an idea where every user (~10,000 Concurrent Users) are streaming their geo-location. I want to aggregate users and display on a map with one single pointer at different zoom levels after that If the user zoom in or zoom out then that pointer also distributed and aggregates users according to that particular zoom level of the map using S2 Cells.
At the beginning any user will see globe just like google earth after that If user zoom in or zoom out at certain position then we need to aggregate users according to that zoom level of the user(current).
In the beginning, I have planned to use some streaming tool like Kafka and for stream processing tools like flink and from front end I will stream user data e.g latitude, longitude, and boundaries longitude and latitude from current zoom level and In the backend, I just check into stream data If users lat, long are falls in boundary lat, long then just return that user with his lat, long and just pinpoint on the map, but this is not that I want to do, I want to aggregate all user at the certain region and display only one pointer after that If I zoom Inside that pointer then It will distribute the pointer and show some others pointers. If I just zoom in at the max level of the map then users show individual.
I want to work on < 30 seconds old data aggregation.
At the frontend, I am using Mapbox to show users on a map. I want to know, how can I use S2 Library Cells to aggregate data and I am also looking for how to make the balance of Mapbox Zoom Levels and S2 Zoom Levels.
The output something like that points
This has nothing to do with Kafka or Streaming other than ingestion. Various Javascript frameworks (start with D3) already provide such a feature, and you'd query the data out of a database/API with query filters around the box of display coordinates, not directly from a stream