change to promtail yaml config not reflected in grafana/loki

Question

I've made a revision to a /usr/local/bin/config-promtail.yaml file on server a, something like this:

  - job_name: error_file
    static_configs:
      - targets:
          - localhost
        labels:
          job: bridge_errors
          __path__: home/user/logs/new_error_file.json
          host: server_a

where 'new_error_file.json' used to be 'old_error_file.json'. This is sending the logs to Loki/Grafana running on server b. I could see the old error file(s) in the Loki data adapter in Grafana just fine, but then updated the file name in the yaml config, as above, restarted the Promtail service and...nothing. The new files are not showing up. I can check the status of the service and can see " level=info msg="Seeked..." entries for the new file, so it's working at least as far as the Promtail service in concerned. Any ideas?? I'm a bit new to Grafana (like, only a couple days' in), so am probably missing something?? Thanks for any assistance!

Answer 1

I also stumbled upon the same issue. For some reason I currently can't explain, for Promtail to reflect config changes, you need to

1. Stop Loki sudo systemctl stop loki
2. Stop Promtail sudo systemctl stop promtail
3. Delete /tmp/positions.yaml, sudo rm -rf /tmp/positions.yaml
4. Start Loki sudo systemctl start loki
5. Start Promtail sudo systemctl start promtail
6. Refresh Grafana Explore screen a few times until a new/modified job becomes visible