Read data recursively from a web API (OSISoft PI) using R

739 Views Asked by At

Apologies in advance for a somewhat vague post - I'm reading confidential data and can't share the data itself or the exact means to access it.

I'm writing an R function to retrieve some experimental data between two timestamps (start_time & end_time) from PI Web API for a given equipment (webid). In some cases I may want to retrieve up to 100,000 rows but each API call returns a maximum of 1000 rows of data. The observations are at unequal intervals so it's not always known how many rows are to be retrieved for a given start_time and end_time.

My current approach is to use a while loop - simplified logic below:

while(difftime(time1 = end_time, time2 = t1, units = hours) > 1){
  dat <- rbind(dat, PiRead(start_time = t1, end_time = end_time, webid = webid)
  t1 <- max(dat$time)
}

Where PiRead is a function that returns data for a given start_time, end_time and webid (equipment), and t1 is the max time in the data retrieved from the current API call. I'm always retrieving historical data and a gap of 1 hour is acceptable. The while loop works but I'd like to make a recursive function.

Here's what I have so far:

recursiveRead <- function(start_time, end_time, webid){
  dat <- PiRead(start_time = start_time, end_time = end_time, webid = webid)
  if(abs(difftime(time1 = dat[, max(time)], time2 = end_time, units = 'hours'))) <= 1){
    return(dat)
  }else{
    cat(sprintf('\nDesired End Time: %s\tCurrent End Time: %s\n', end_time, dat[, max(time)]))
    return(rbind(dat, 
                 recursiveRead(start_time = dat[, max(time)], end_time = end_time, webid = webid)))
  }
  
}

When executed, the two times printed to console (via the cat statement) are identical and R is stuck an infinite loop. Any thoughts on what I'm missing here?

0

There are 0 best solutions below