Can I stop a Foundry Data Connection ingest if it runs for too long?

Question

Recently, a Data Connection ingest ran without making progress for 6 hours (versus the typical ingest time of 30 mins). This caused the source database server to grind to a halt, so we needed to kill the ingest process.

Is there a way to automatically abort an ingest if it runs for too long, so that if this job hangs again it won't cause the same issue?

Answer 1

Yes. You can set maxAllowedDuration on the jobspec in ISO-8601 format, e.g, maxAllowedDuration: PT10M will automatically cancel the job after 10 minutes. Jobs will sometimes hang because of a database issue.