Splunk dashboard and reports source backup for versioning

Question

We are trying to take a backup of Splunk dashboards and reports source code for versioning. we are on an enterprise implementation where our rest calls are restricted. we can create and access dashboards and reports via Slunk UI, but would like know if we can automatically take backup of them and store in our versioning system.

Answer 1

Automated versioning will be quite a challenge without REST access. I assume you don't have CLI access or you wouldn't be asking.

There are apps available to do this for you. See https://splunkbase.splunk.com/app/4355/ and https://splunkbase.splunk.com/app/4182/ .

There's also a .conf presentation on the topic. See https://conf.splunk.com/files/2019/slides/FN1315.pdf

Answer 2

For the time being, I have written a Python script to read/intercept the UI inspet -> performance -> network responses of Splunk's reports URL which lists the complete set of reports under my app and as well as its full details.

from time import sleep
import json
from selenium import webdriver
from selenium.webdriver import DesiredCapabilities

# make chrome log requests
capabilities = DesiredCapabilities.CHROME
capabilities["goog:loggingPrefs"] = {"performance": "ALL"}  # newer: goog:loggingPrefs

driver = webdriver.Chrome(
    desired_capabilities=capabilities, executable_path="/Users/username/Downloads/chromedriver_92"
)

spl_reports_url="https://prod.aws-cloud-splunk.com/en-US/app/sre_app/reports"

driver.get(spl_reports_url)
sleep(5)  # wait for the requests to take place

# extract requests from logs
logs_raw = driver.get_log("performance")
logs = [json.loads(lr["message"])["message"] for lr in logs_raw]

# create directory to save all reports as .json files
from pathlib import Path
main_bkp_folder='splunk_prod_reports'
# Create a main directory in which all dashboards will be downloaded to
Path(f"./{main_bkp_folder}").mkdir(parents=True, exist_ok=True)


# Function to write json content to file
def write_json_to_file(filenamewithpath,json_source):
    with open(filenamewithpath, 'w') as jsonfileobj:
        json_string = json.dumps(json_source, indent=4)
        jsonfileobj.write(json_string)
        
def log_filter(log_):
    return (
        # is an actual response
        log_["method"] == "Network.responseReceived"
        # and json
        and "json" in log_["params"]["response"]["mimeType"]
    )

counter = 0

# extract Network entries from each log event
for log in filter(log_filter, logs):
    #print(log)
    request_id = log["params"]["requestId"]
    resp_url = log["params"]["response"]["url"]
    # print only results_preview 
    if  "searches" in resp_url:
        print(f"Caught {resp_url}")
        counter = counter + 1
        nw_resp_body = json.loads(driver.execute_cdp_cmd("Network.getResponseBody", {"requestId": request_id})['body'])
        for each_report in nw_resp_body["entry"]:
            report_name = each_report['name']
            print(f"Extracting report source for {report_name}")
            report_filename = f"./{main_bkp_folder}/{report_name.replace(' ','_')}.json"
            write_json_to_file(report_filename,each_report)
            print(f"Completed.")

print("All reports source code exported successfully.")

The above code is far from production version as yet to add error handling, logging, and modularization. Also, to note that the above script uses browser UI, in production the scrip will be run in a docker image with ChromeOptions to use headless mode.

Instead of

driver = webdriver.Chrome(
    desired_capabilities=capabilities, executable_path="/Users/username/Downloads/chromedriver_92"
)

Use:

from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--window-size=1420,2080')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-gpu')
driver = webdriver.Chrome(
    desired_capabilities=capabilities,options=chrome_options, executable_path="/Users/username/Downloads/chromedriver_92"
)

Here you can customize as per your need.