Can You Persist a Model in Databricks Community Edition?

Question

Is there a way to persist a Python machine learning model when using the free Databricks community edition?

• It looks like the DBFS is not available. This means that I can't use tools like joblib to save the model in the file system.
• ML Flow is not available in community edition, so that's not an option either.

Is there some other way I'm missing?

Answer 1

You can always persist the model to the local disk, like, /tmp/my-model, and then use dbutils.fs.cp function to copy it to DBFS to persist, like this:

dbutils.fs.cp('file:/tmp/my-model', '/FileStore/my-model')

When you will need it again, just copy in the reverse direction & load:

dbutils.fs.cp('/FileStore/my-model', 'file:/tmp/my-model')