Finding model (returned from for loops) with lowest AIC in R

Question

I am trying to find model with lowest AIC. Models are returned from two for loops that make possible combinations of columns. I am unable to make the function return model with lowest AIC. The code below demonstrates where I got stuck:

rm(list = ls())

data <- iris

data <- data[data$Species %in% c("setosa", "virginica"),]

data$Species = ifelse(data$Species == 'virginica', 0, 1)

mod_headers <- names(data[1:ncol(data)-1])

f <- function(mod_headers){
    for(i in 1:length(mod_headers)){
    tab <- combn(mod_headers,i)
    for(j in 1:ncol(tab)){
      tab_new <- c(tab[,j])
      mod_tab_new <- c(tab_new, "Species")
      model <- glm(Species ~., data=data[c(mod_tab_new)], family = binomial(link = "logit"))
    }
    }
  best_model <- model[which(AIC(model)[order(AIC(model))][1])]
  print(best_model)
}

f(mod_headers)

Any suggestions? Thanks!

Answer 1

glm() uses an iterative re-weighted least squares algorithm. The algorithm reaches the maximum number of iterations before it converges - changing this parameter helps in your case:

 glm(Species ~., data=data[mod_tab_new], family = binomial(link = "logit"), control = list(maxit = 50))

There was another issue using which, I replaced it with an if after each model fit to compare to the lowest AIC so far. However, I think there are better solutions than this for-loop approach.

f <- function(mod_headers){
  lowest_aic <- Inf     # added
  best_model <- NULL    # added

  for(i in 1:length(mod_headers)){
    tab <- combn(mod_headers,i)
    for(j in 1:ncol(tab)){
      tab_new <- tab[, j]
      mod_tab_new <- c(tab_new, "Species")
      model <- glm(Species ~., data=data[mod_tab_new], family = binomial(link = "logit"), control = list(maxit = 50))
      if(AIC(model) < lowest_aic){ # added
        lowest_aic <- AIC(model)   # added
        best_model <- model        # added
      }
    }
  }
  return(best_model)
}

Answer 2

I replaced your for loops with vectorised alternatives

library(tidyverse)
library(iterators)
# Column names you want to use in glm model, saved as list
whichcols <- Reduce("c", map(1:length(mod_headers), ~lapply(iter(combn(mod_headers,.x), by="col"),function(y) c(y))))

# glm model results using selected column names, saved as list
models <- map(1:length(whichcols), ~glm(Species ~., data=data[c(whichcols[[.x]], "Species")], family = binomial(link = "logit")))

# selects model with lowest AIC
best <- models[[which.min(sapply(1:length(models),function(x)AIC(models[[x]])))]]

Output

Call:  glm(formula = Species ~ ., family = binomial(link = "logit"), 
data = data[c(whichcols[[.x]], "Species")])

Coefficients:
 (Intercept)  Petal.Length  
       55.40        -17.17  

Degrees of Freedom: 99 Total (i.e. Null);  98 Residual
Null Deviance:      138.6 
Residual Deviance: 1.208e-09    AIC: 4

Answer 3

Using your loop, just put all the models in one list. Then compute the AIC of all these models. Finally return the model with the minimum AIC.

f <- function(mod_headers) {

  models <- list()
  k <- 1
  for (i in 1:length(mod_headers)) {
    tab <- combn(mod_headers, i)
    for(j in 1:ncol(tab)) {
      mod_tab_new <- c(tab[, j], "Species")
      models[[k]] <- glm(Species ~ ., data = data[mod_tab_new], 
                         family = binomial(link = "logit"))
      k <- k + 1
    }
  }

  models[[which.min(sapply(models, AIC))]]
}