Approach for performing long running tasks in .NET

Question

I'm developing a rather large website with which included a customer base and billing plans. The monthly billing should happen automatically (however, an employee should, manually, click a button to initiate the process every month due to business rules) and the number of customers are between 3000 - 5000. On every single invoice, the price lines should be calculated based on quite a lot of parameters and even external data from a webservice.

The above will definitely be a (very) long running task and since the application is an ASP.NET MVC website running on IIS, I don't want the process to run in it's own thread on IIS. My concerns are that if anything happens to IIS/the webserver, the billing calculation process stops aswell, which would be critical, to say the least.

Therefore I'm looking for alternatives and have read about creating a Windows Service for tasks like this. Is this the way to go? Or is there other, maybe more "modern" ways of running long processes? The process has to able to be started from the ASP.NET MVC application though.

Any help/hint is greatly appreciated! :-)

Answer 1

Looks like a good solution for SignalR: http://www.asp.net/signalr

Normally, this would be the ideal case for a SQL stored proc / agent job, but not if you need to call an external webservices for each bill to gather the data needed.

As long as you have a 'header' record for the bill run, you can easily store which record you're currently on and whether the run has completed or not. This will always be a consideration whether you run as a windows service or a thread in IIS - someone could turn off the machine at any point.

SignalR will allow you to

• initiate the run as a background task
• easily pass back progress to the UI
• allow you to stop the run from the UI
• reconnect to the in-progress run if the user closes browser
• listen for 'completed' events etc
• allow progress to be seen from multiple clients

Answer 2

If it's just one or two long running tasks, keep life simple and solve this with the Windows task scheduler and a C# console app. If you've got access to the console/RDP then create a scheduled task which would:

1. Execute periodically, say every five minutes checking for work to do.

2. if it's as simple as running just one task, then in your web front end, when the user wants to kick off your billing run then the web front end flips a status flag in your database.

3. Scheduled task notices your status flag has been flipped to "run mode"

4. Run billing code.

5. When done reset the "run mode"

Pre-requisites -

1. Factor out your billing run code such that that it can be called from a console exe.

2. Ensure the aforementioned scheduled task is configured not to start a new instance if the task is already running (the default, but double check the settings tab in the "Create Task" dialogue).

3. Add some logging so you know how much of the task has been completed/progress

Other Solution Considerations:

• Long running IIS tasks: your web server isn't designed to execute long running tasks such as batch jobs, which is essentially what your billing task is.

• Windows Services: why re-invent the wheel with the added complexity of learning to write a Windows service.

Answer 3

I actually had a requirement a few years back to do something similar to this. It was for processing monthly membership dues. The way I would do it is simple. First, in your database, add a table similar to this:

--------------------------------------
|Monthly Processing Table            |
|------------------------------------|
|MonthlyProcessorId       int     PK |
|ProcessMonth             int        |
|ProcessYear              int        |
|HasBeenProcessed         bool       |
|CanBeProcessed           bool       |
|ProcessDate              DateTime   |
|------------------------------------|

Of course, you can add other fields as you see fit. This is only for a demonstration.

Next, create your web page that contains the button that should get pressed monthly to begin the processing. When the button gets hit, it should perform the following logic:

1. Get the current month and year (DateTime.Now)
2. Search the monthly processing table for rows that match ProcessMonth equal to DateTime.Now.Month AND ProcessYear equal to DateTime.Now.Year.
3. If there are any rows returned, then check the row and see if HasBeenProcessed is true and ProcessDate to see when and report this to the user and you are done. This is an important step because if rows are returns and HasBeenProcessed is true, then the monthly dues have probably already run for this month and you do not want to double charge people. So don't skip this step and stop the program execution logic here if HasBeenProcessed is true!
4. If there are no rows returned, then create a new row. Set ProcessMonth equal to DateTime.Now.Month, ProcessYear to DateTime.Now.Year. Set CanBeProcessed to true and HasBeenProcessed to false.

The last piece of the puzzle is to have a Windows Service that sits in the background and periodically pulls all rows from the database where HasBeenProcessed is equal to false AND CanBeProcessed is equal to true.

If any rows are returned, then you just process them and after processing them, set the ProcessDate to DateTime.Now and set HasBeenProcessed to true. Finally, your service just goes back to periodically checking. When I say "periodically checking", I'm talking about maybe once a day or maybe even once a week, depending on your requirements. This service shouldn't bog down your SQL server in any way.

Instead of a Windows Service, you could have a Console Application that is scheduled to run with Windows Scheduler. I prefer the Windows Service way though, because it just "feels more right" to me.

Now if you wanted to get really fancy about it, you could write your schedule time and logic in an interface. This way you could reuse this Windows Service down the road.