Invalid Push metric issue while using Prometheus push gateway simpleclient in spring boot

Question

I'm using prometheus simple client for collecting metrics in my system via a spring boot application

void insertBatchJob(String request , String[] labelNames, String[] labelValues,String counter) throws Exception {
            CollectorRegistry registry = new CollectorRegistry();
            Gauge inprogressRequests ;
            if( !gaugeRegiseryMap.containsKey(request) ) {
                inprogressRequests = Gauge.build()
                        .name(request).labelNames(labelNames).help(request).register();
                gaugeRegiseryMap.put(request,inprogressRequests);
            }else{
                inprogressRequests = gaugeRegiseryMap.get(request);
            }
            try {
                inprogressRequests.labels(labelValues[labelValues.length-4],
                        labelValues[labelValues.length-2],
                        labelValues[labelValues.length-3],
                        labelValues[labelValues.length-1]).set(Double.parseDouble(counter));
                registry.register(inprogressRequests);
            } finally {
                PushGateway pg = new PushGateway("127.0.0.1:9091");
                pg.pushAdd(registry, request);
            }
        }

the following code is working , but when i change inprogressRequests.labels() when i pass the array containing label values i.e.

` 

    void insertBatchJob(String request , String[] labelNames, String[] labelValues,String counter) throws Exception {
            CollectorRegistry registry = new CollectorRegistry();
            Gauge inprogressRequests ;
            if( !gaugeRegiseryMap.containsKey(request) ) {
                inprogressRequests = Gauge.build()
                        .name(request).labelNames(labelNames).help(request).register();
                gaugeRegiseryMap.put(request,inprogressRequests);
            }else{
                inprogressRequests = gaugeRegiseryMap.get(request);
            }
            try {
                inprogressRequests..labels(labelValues).set(Double.parseDouble(counter));
                registry.register(inprogressRequests);
            } finally {
                PushGateway pg = new PushGateway("127.0.0.1:9091");
                pg.pushAdd(registry, request);
            }
        }` 

the same is giving me errror -

Response code from http://127.0.0.1:9091/metrics/job/loop6_2 was 400, response body: pushed metrics are invalid or inconsistent with existing metrics: collected metric .

This is my first time using prometheus , it will be good if someone can share light on this.

Answer 1

Try pushing an empty metric, this issue happens when you try pushing the same metric twice in one push.FOr example adding more that one sample with the same labes to the metric or if the pushgateway was restarted you wont be able to send the same metric againg, you must delete it first.

see : https://github.com/prometheus/pushgateway/blob/master/README.md

on status code 400