How to track usage of OpenGrok service

Question

What are some ways we could monitor OpenGrok, both usage and specific searches? Is there any systems either built in or that can be added as extensions?

We are running OpenGrok on the intranet but we have no visibility of how it's being used and how much usage it gets.

Install

Currently OpenGrok is being installed through a puppet pipeline.

class opengrok {
  package {[
    'opengrok-tomcat',
    'ctags',
  ]

...

file { '/usr/share/tomcat8/webapps/opengrok-1.0':
  ensure  => directory,
  owner   => 'root',
  group   => 'root',
  mode    => '0755',
}

What are some ways I could monitor OpenGrok, both usage and specific searches and/or more?

Preliminary ideas:

1) Add a proxy forwarding endpoint in front of opengrok to record the "hits" and send it to some DB store. That wouldn't track low level usage of opengrok (i.e. search queries), just the hits.

2) Somehow enable tomcat logs and parse through the logs? I'm not sure how much info I would get from the logs and the parsing might get involved. Then send this info to some DB store.

Answer 1

We use Nginx like a proxy server to OpenGrok, so we have all OpenGrok searches logged at /var/log/nginx/access.log* files, where you can get a lot of information like, for example, the following:

COMPUTER-IP - USER [27/Sep/2018:10:17:30 -0300] "GET /xxxxx/search?project=sandbox_helloworld%28master%29&q=This+is+a+test&defs=&refs=&path=&hist=&type=sh HTTP/1.1" 200 3345 "https://OPENGROK-SERVER/xxxxx/search?project=sandbox_helloworld%28master%29&q=This+is+a+test&defs=&refs=&path=&hist=&type=" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"

Answer 2

Recent OpenGrok versions have monitoring capabilities that can provide high level (i.e. not per user or project) overview. Specifically you can get the metrics in the Prometheus format via HTTP GET request to /metrics/prometheus. For example if your web application is on http://example.com/source/ then you can view the metrics using e.g. curl http://example.com/source/metrics/prometheus. The access to this endpoint is not authorized. These metrics can be then fed into Grafana or the like. The requests_* meters are tagged with category of the request (e.g. search, api, history, ...) so you can get nice breakdown, something like this:

For more info see https://github.com/oracle/opengrok/wiki/Monitoring#web-application