I'm trying to populate a database with django models and I've used the Swifter structure to parallelize a pandas apply, like this:
df.swifter.force_parallel(enable=True).apply(lambda row: migrate(row), axis = 1)
When apply function loaded 1000 objects into database it gave me "AppRegistryNotReady: Apps aren't loaded yet." error.
The function migrate is used for each row in dataframe to create the data on db with django models
Seems that swifter (or what's under the hood) processes disrespects the django global context.
The workaround is to invoke
django.setup()inside of your apply function.That's fast, but it can mess some things. For example, for me logging doesn't work no more for the usual context.