how to click a link with scraper/crawler

211 Views Asked by At

So i'm using mechanize/nokogiri with a rails project and need to be able to click a link on a web page.

the webpage is this: https://www.fanduel.com/p/Home

(must be signed in)

so what I want to do is click a link that matches "NBA Salary Cap 60k Mon Dec 15th" (or whatever todays date is)

after that link is clicked and the page is loaded, i've built a scraper to get the info i need so that isn't a problem.

I just need to be able to parse through that table and click a link.

Maybe it's my inexperience with mechanize but I think the table is generated by JS and I can't get access to those elements since mechanize is an html/css parser?

anybody have any ideas on how to do this? (my hunch is to use something like phantomjs but am unfamiliar with it)

0

There are 0 best solutions below