Prometheus tries to look for localhost:9090 instead of 192.168.2.5:9090 and that's why I think it gives an error [help]

Question

it’s my 1st time in the forum and I’ve never had problems before but now I do and I’d appreciate it if you could help me. mount my new NAS server with OMV5 in a dell optiplex and it works very well for me, but I wanted to install prometheus with node-xporter and grafana and it is giving me many errors that I will describe in detail with images of all at the end

1st I ran node-exporter:

root@OPENMEDIAVAULT:~# docker run -d --net="host" --pid="host" -v "/:host:ro,rslave" quay.io/prometheus/node-exporter:latest --path.rootfs=host

enter in http://192.168.2.5:9100/metrics its all good

https://i.stack.imgur.com/cYLO2.jpg

2nd create the configuration folder and paste the .yml from my old server:

root@OPENMEDIAVAULT:/var/lib/docker/config-docker/**Prometheus**# nano prometheus.yml
global:
  scrape_interval: 5s
  external_labels:
    monitor: 'node'
scrape_configs:
  - job_name: 'prometheus' 
    static_configs: 
      - targets: ['192.168.2.5:9090'] 
  - job_name: 'node-exporter' 
    static_configs: 
      - targets: ['192.168.2.5:9100'] 
  - job_name: 'cAdvisor'
    static_configs:
      - targets: ['192.168.2.5:8080'] 
  - job_name: 'raspi-openwrt-router1'
    static_configs:
      - targets: ['192.168.2.1:9100']
  - job_name: 'snmp-ddwrt-r9000'
    metrics_path: /snmp
    params:
      module: [ddwrt]
    static_configs:
      - targets: ['192.168.2.2'] 
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: 192.168.2.2:9116  # The SNMP exporter's real hostname:port.

good 3rd I ran prometheus docker with:

root@OPENMEDIAVAULT:/var/lib/docker/config-docker/Prometheus# docker run -d --name prometheus -p 9090:9090 -v /var/lib/docker/config-docker/Prometheus/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus

i open prometheus at http://192.168.2.5:9090/ and enter service discovery and the 1st error

https://i.stack.imgur.com/KbaLb.jpg

I enter targets and the 2nd error

https://i.stack.imgur.com/1aOZA.jpg

ERROR Get "http://localhost:9090/metrics": dial tcp: lookup localhost on [::1]:53: dial udp [::1]:53: connect: cannot assign requested address
and the 3rd error I install grafana
root@OPENMEDIAVAULT:~# docker run -d --name grafana -p 3456:3000 grafana/grafana
[Imgur](https://i.imgur.com/4SNVCWl.jpg)

and does not run
and log with errors is long but I will place
Error relocating /run.sh: rl_set_keymap: symbol not found Error relocating /run.sh: rl_variable_dumper: symbol not found Error relocating /run.sh: rl_bind_keyseq_in_map: symbol not found Error relocating /run.sh: write_history: symbol not found Error relocating /run.sh: rl_on_new_line: symbol not found Error relocating /run.sh: rl_clear_visible_line: symbol not found Error relocating /run.sh: rl_completion_matches: symbol not found Error relocating /run.sh: rl_completion_mode: symbol not found Error relocating /run.sh: rl_add_undo: symbol not found Error relocating /run.sh: rl_cleanup_after_signal: symbol not found Error relocating /run.sh: rl_list_funmap_names: symbol not found Error relocating /run.sh: using_history: symbol not found 

This is an installation that I have done many times without any problem but now I can’t find a way to make it work, and I think it’s because the prometheus is somehow looking for the address of localhost:9090 ainsted of 92.168.2.5:9090 and that’s why it doesn’t find node-exporter either and grafana can’t find any of them…

I wish you could help me or give me a clue since I’ve been trying to install it for several days and I can’t

Answer 1

try this docker-compose file

version: "3.3"

networks:
  net:
    driver: overlay
    attachable: true

services:
    cadvisor:
        image: google/cadvisor
        networks:
          - net
        command: -logtostderr -docker_only
        privileged: true
        volumes:
          #- /etc/kmsg:/etc/kmsg:ro
          - /var/run/docker.sock:/var/run/docker.sock:ro
          - /:/rootfs:ro
          - /var/run:/var/run
          - /sys:/sys:ro
          - /var/lib/docker/:/var/lib/docker:ro
    node-exporter:
        image: 'prom/node-exporter:v1.3.1'
        networks:
          - net
        volumes:
          - '/:/host:ro,rslave'
        command:
          - '--path.rootfs=/host'
        network_mode: host
        pid: host
        restart: unless-stopped  
    prometheus:
        image: 'prom/prometheus:v2.35.0'
        networks:
          - default
          - net
        command:
          - '--config.file=/etc/prometheus/prometheus.yml'
          - '--storage.tsdb.path=/prometheus'
          - '--storage.tsdb.retention=${PROMETHEUS_RETENTION:-24h}'
        volumes:
          - /var/lib/docker/config-docker/Prometheus:/prometheus
          - /var/lib/docker/config-docker/Prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
        ports:
            - "9090:9090"

and this prometheus.yml file

global:
  scrape_interval: 5s
  external_labels:
    monitor: 'node'
scrape_configs:
  - job_name: 'prometheus' 
    static_configs: 
      - targets: ['localhost:9090'] 
  - job_name: 'node-exporter' 
    dns_sd_configs:
    - names:
      - 'tasks.node-exporter'
      type: 'A'
      port: 9100 
  - job_name: 'cadvisor'
    dns_sd_configs:
    - names:
      - 'tasks.cadvisor'
      type: 'A'
      port: 8080