Adding numbers to variables in WebHarvest

285 Views Asked by At

I should start off by saying that I'm very new to javascript.

I need to feed a bunch of urls to webharvest based on a number. It's a long story why, but my url structure looks kind of like this: http://www.example.com/foo/bar?page=0. ?page= increases by 25 each step. So the next page will be http://www.example.com/foo/bar?page=25 then http://www.example.com/foo/bar?page=50 and so forth. There's a maximum, which I can set by another variable, call it ${maxpages}.

So what I need to do is modify a variable to feed into the otherwise fully predictable url, so that the variable has 25 added to it each time. I'm thinking of doing a while loop, like this:

<var-def name="pageNo">0</var-def>
<while condition="${pageNo} < ${maxpages}">
    <body>
        <html-to-xml><http url="${url}?${pageNo}"/></html-to-xml>
        <var-def name="pageNo">
            <var name="pageNo">[this is where I want to add 25]</var>
        </var-def>
    </body>
</while>

So I'm really unsure of the syntax here.

My questions are:

  1. How can I check that my variable pageNo is less than maxpages in the while condition?

  2. Can you add integers to variables in webharvest? How?

1

There are 1 best solutions below

0
On

I think I'm developing a habit of answering myself.

After an hour of trial and error, I have this:

<var-def name="commentCount">432</var-def>
<var-def name="pageNo">0</var-def>
<while condition="${pageNo.toInt() &lt; commentCount.toInt()}">
    <html-to-xml>
        <http url="http://www.example.com/foo/bar?${pageNo}"/>
    </html-to-xml>
    <var-def name="pageNo"><template>${pageNo.toInt() + 25}</template></var-def>
</while>