Best practice for unregistering/redefining/customizing already-registered Prometheus metrics in Java

Question

Say I'm using a library (Java) which defines / registers metrics in Prometheus. I'm wondering how I might be able to unregister/redefine/customize these metrics which have already been defined/registered by said library? Specifically, if I had a Histogram with certain buckets that were already registered, how could I update the same Histogram with different buckets?

For example, if my-java-library defines a Histogram with name my_histogram, I was thinking I might be able to use the unregister method on CollectorRegistry like so:

in my-java-library:

{
...

this.myHistogram = Histogram.build()
    .name("my_histogram")
    .register();

...
}

public Histogram getMyHistogram() {
    return this.myHistogram;
}

in my-repo that uses my-java-library:

import my-java-library; // ignore the poor naming convention :^)

...

{
...

    Histogram myHistogram = myJavaLibrary.getMyHistogram();
    CollectorRegistry.defaultRegistry.unregister(myHistogram);

...
}

The above example seems to be problematic for a few reasons:

1. The unregister method is apparently only supposed to be used in rare cases for unit tests and not (apparently) for the above purpose ([1], [2], [3])
2. It seems the client_java repo for Prometheus is registering background, inaccessible metrics behind the scenes which, since these Collector instances aren't accessible (at least not to my knowledge), cannot be unregistered using the aforementioned model. This means even if I tried and succeeded at unregistering my_histogram, I will still run into a runtime exception (IllegalArgumentException) since these behind-the-scene metrics are being caught by the assertNoDuplicateNames method.

I'm at a bit of a loss. I will probably create an issue in the client_java repo as well but wanted to see if anyone on SO has run into and solved this problem for themselves.

Answer 1

I guess the problem is that my-java-library will use the this.myHistogram instance. So unregistering this.myHistogram does not help because the library will just continue using the unregistered this.myHistogram instance, but you would never see these metrics as the histogram is no longer registered.

What you could try is to replace this.myHistogram with your own instance. If the library has a setter myJavaLibrary.setMyHistogram(Histogram) that should work, if it doesn't you will likely need to use reflection to replace the this.myHistogram instance in the library.

This is not directly related to unregistering the original myHistogram instance. You can just register your own instance with a different name and keep the original instance registered. You can use CollectorRegistry.defaultRegistry.unregister(myHistogram) for unregistering the original instance, but this is just for cleaning up and should not affect functionality, because after setting this.myHistogram the library will not be able to access the original instance and the original instance will never be used.

Does that help?