How do I perform koyck lag transformations in PMML?

Question

I'm using PMML to transfer my models (that I develop in R) between different platforms. One issue I often face is that given input data I need to do a lot of pre-processing. Most times this is rather straightforward in PMML but I cannot figure out how to do it when I need a Koyck lag transformation. Now the first few lines of the input data set looks like this:

         Y Z         S       Xa       Xb       Xc
1 11.37738 1 0.8414710      0.0      0.0 581102.6
2 21.29848 2 0.9092974 700254.1      0.0  35695.1
3 14.30348 3 0.1411200      0.0 384556.3      0.0
4 18.07305 4 0.0000000 413643.2      0.0      0.0
5 29.02756 5 0.0000000 604453.3      0.0 350888.2
6 20.73336 6 0.0000000      0.0      0.0 168961.2

and is generated by:

df<-structure(list(Y = c(11.3773828021943, 21.2984762226498, 14.3034834956969, 
                         18.0730530464578, 29.0275566937015, 20.7333617643781,     30.9707039948106, 
                         30.2428379202751, 22.1677291047936, 19.7450403054104, 18.4642890388219, 
                     28.4145184014117, 28.5224574661743, 40.5073319897728, 40.8853498146471, 
                     20.7173713186907, 35.8080372291603, 37.6213598048788, 38.3123458040493, 
                     25.143519382411), 
               Z = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20), 
               S = c(0.841470984807897, 0.909297426825682, 0.141120008059867, 
                     0, 0, 0, 0.656986598718789, 0.989358246623382, 
                     0.412118485241757, 0, 0, 0, 0.420167036826641, 0.99060735569487,
                     0.650287840157117, 0, 0, 0, 0.149877209662952, 0.912945250727628), 
               Xa = c(0, 700254.133201206, 0, 413643.212229974, 604453.339408554, 
                      0, 623209.174415675, 1042574.05046884, 0, 0, 397257.053501325, 
                      441408.09060313, 0, 0, 597980.888163467, 0, 121672.230528635,
                      199542.274825303, 447951.083632432, 84751.5842557032), 
               Xb = c(0, 0, 384556.309344495, 0, 0, 0, 0, 0, 0, 0, 0, 
                      179488.805498654, 31956.7161910341, 785611.676606721, 
                      65452.7295721654, 0, 231214.563631705, 0, 0, 
                      176249.685091327), 
               Xc = c(581102.615208462, 35695.0974169599, 0, 0, 350888.245086195, 
                      168961.239749307, 458076.400377529, 218707.589596171, 
                      0, 506676.223324812, 0, 25613.8139087091, 429615.016105429, 
                      410675.885159107, 0, 229898.803944166, 2727.64268459058, 
                      711726.797796325, 354985.810664457, 0)), 
          .Names = c("Y", "Z", "S", "Xa", "Xb", "Xc"), 
          row.names = c(NA, -20L), 
          class = "data.frame")

I want to create a new variable M using koyck lags of the variables Xa, Xb and Xc like this:

lagIt<-function (x, d, ia = mean(x)) 
{
  y <- x
  y[1] <- y[1] + ia*d
  for (i in 2:length(x)) y[i] <- y[i] + y[i-1] * d
  y
}

df2<-transform(df, M=(lagIt(tanh(Xa/300000), 0.5) + 
                      lagIt(tanh(Xb/100000), 0.7) + lagIt(tanh(Xc/400000), 0.3)))

> head(df2)
#          Y Z         S       Xa       Xb       Xc        M
# 1 11.37738 1 0.8414710      0.0      0.0 581102.6 1.460318
# 2 21.29848 2 0.9092974 700254.1      0.0  35695.1 1.637388
# 3 14.30348 3 0.1411200      0.0 384556.3      0.0 1.767136
# 4 18.07305 4 0.0000000 413643.2      0.0      0.0 1.960151
# 5 29.02756 5 0.0000000 604453.3      0.0 350888.2 2.796750
# 6 20.73336 6 0.0000000      0.0      0.0 168961.2 1.761774

and finally build a model:

fit<-lm(Y~Z+S+M, data=df2)

Using the pmml library in R I can get the PMML XML output like this.

library(pmml)
pmml(fit)

However, I want to include a section of where the creation of the variable M takes place. How can I write that section conforming to PMML? Again the input data is the df data.frame and I want all pre-processing of data to be defined in PMML.

Answer 1

PMML operates on single-valued data records, but you're trying to use vector-valued data records. Most certainly, you cannot do (for-)loops in PMML.

Depending on your deployment platform, you might be able to use extension functions. Basically, this involves 1) programming Koyck lag transformation, 2) turning it into a standalone extension library and 3) making the PMML engine aware of this extension library. This extension function can be called by name just like all other built-in and user-defined functions.

The above should be doable using the JPMML library.