I have started this web scraping project backwards by beginning with node.io. I have it working such that:
node.io myjob arg1 arg2
Will emit the results in JSON format:
{
1 : 'apple',
2 : 'orange',
3 : 'banana'
}
My goal is to be able to run this in a web server so if I point to:
http://localhost/arg1/arg2
It will be able to execute the node.io myjob arg1 arg2
command with the arguments and retrieve my JSON output.
I have looked through node.js but I cannot find any examples of it using node.io or vice versa to create the web server. I know I am working backwards but this is where I am stuck currently. Any advice would be greatly appreciated.
Use the standard built-in web server with Node.js, and when a request comes in with the correct parameters, programmatically start your scraping job.