Scheduling U - SQL Job

Question

I am trying to schedule a U SQL job. Please let me know whether I can schedule the U SQL job.If so,how can I schedule.

Thanks, Vinoth

Answer 1

You could use Azure Automation (with the help of the Azure Data Lake Analytics Cmdlets) or Azure Data Factory to schedule a U-SQL script in the cloud.

Answer 2

You can get some guidance regarding creating a ADF Pipeline here: https://azure.microsoft.com/en-us/documentation/articles/data-factory-build-your-first-pipeline-using-editor/

Answer 3

To my mind, the best way to orchestrate your U-SQL job along with concomitant data management such as getting source data, pushing output data and etc is Azure Data Factory V2. ADF has reach API. Basically, you can run your jobs using either PowerShell or C# or a trigger.

See my very simple example of the job and how to add a trigger below. In this example, I process the documents with my U-SQL job and then push output file (CSV or Avro file) into Azure SQL Server