How do we access databricks job parameters inside the attached notebook?

Question

In Databricks if I have a job request json as:

{
  "job_id": 1,
  "notebook_params": {
    "name": "john doe",
    "age": "35"
  }
}

How do I access the notebook_params inside the job attached notebook?

Answer 1

In notebooks, you can access parameters via Widgets using the dbutils.widgets.get function. For your example, it will be:

name = dbutils.widgets.get("name")
age = dbutils.widgets.get("age")

Please note that by default the value is string, so if you need to have age as number, you need to convert it.

Also, if you want to debug output before you put it as a job, then you need to declare corresponding widgets using one of the dbutils.widgets functions, otherwise you'll get error that widget isn't defined when doing dbutils.widgets.get. For example, you can use dbutils.widgets.text that allows to enter any text):

dbutils.widgets.text("name", "<default_name>", "Enter name")
dbutils.widgets.text("name", "<default_age>", "Enter age")

Answer 2

I found a solution that is useful when you want to read all the parameters without having to define each single one of them.

def get_parameters():
    
    all_args = dict(dbutils.notebook.entry_point.getCurrentBindings())

    # remove '--' substring
    all_args = {key.replace('--', ''): value for key, value in all_args.items()}

    # parse values to correct format
    all_args = {key: ast.literal_eval(value) for key, value in all_args.items()}

    return all_args

Be aware, you will only be able to get the parameters if you are running the notebook through a Job.