Translate javascript/jQuery to POST using httr/rvest

484 Views Asked by At

I am trying to scrap a web page. Part of the data on this web page is updated using following script:

        <script>
            (function ($) {
                $(document).ready(function () {
                    var timeoutID = window.setTimeout(function () {
                        var $promise = $.ajax({
                            method: 'POST',
                            url: 'https://www.somedomain.com/cms/wp-admin/admin-ajax.php',
                            data: {
                                action: 'get_cache_guide',
                                symbol_id: 6533,
                                _wpnonce: 'b0ce23fbe8'
                            }
                        });
                        $promise.always(function (data) {
                            Chart_Guide({charts: {guide: JSON.parse(data)}});
                        });
                    }, 500);
                });
            }(jQuery));
        </script>

So, my question is how I can get this data (JSON) based on which some table and graphs are displayed using Chart_Guide function?

I have tried the following (parsing text of above script element):

                method <- strsplit(strsplit(script, "method: '")[[1]][2], "',\n")[[1]][1]
                urlOfInterest <- strsplit(strsplit(script, "url: '")[[1]][2], "',\n")[[1]][1]
                action <- strsplit(strsplit(script, "action: '")[[1]][2], "',\n")[[1]][1]
                symbolID <- as.numeric(strsplit(strsplit(script, "symbol_id: ")[[1]][2], ",\n")[[1]][1]) 
                wpnonce <- strsplit(strsplit(script, "_wpnonce: '")[[1]][2], "'\n")[[1]][1]

                data <- strsplit(strsplit(script, "data: ")[[1]][2], "});")[[1]][1]

                data <- gsub("\\\n", "", data)

Then I used rvest and httr in the following way(s):

pageSession <- request_POST(pageSession, url=urlOfInterest, body=data)

Then I tried:

values <- list("action"=action, "symbol_id"=symbolID, "_wpnonce"=wpnonce)
pageSession <- request_POST_1(pageSession, url=urlOfInterest, body=values)

And it all fails...

Suggestions on how to accomplish this task highly appreciated.

0

There are 0 best solutions below