We are trying to build in-house infrastructure for RUM metrics in-house with Prometheus
Every metric has 10 labels and some labels can have 40 or even more values like pageId or country. That makes cardinality value very high. And that leads to very low performance of Prometheus
Did anyone succeed with building in-house RUM metrics? if yes please share your design principles.
In a large scale context, we are doing in-house RUM with prometheus.
Our cluster handles scrapes of 75k samples without flinching. We are using Thanos on top of prometheus with a pretty standard architecture. You should have a look at it if you need to scale your Prometheus cluster