drm() Error Convergence failed for some data but not for similar?

Question

I'm trying to loop over some dose-dependent data and fit it using the drm() function. However, for some of the data I get the following error:

Error in optim(startVec, opfct, hessian = TRUE, method = optMethod, control = list(maxit = maxIt, : non-finite finite-difference value [4] Error in drmOpt(opfct, opdfct1, startVecSc, optMethod, constrained, warnVal, : Convergence failed

I don't see why this occurs for specific data-sets only, when almost identical data gets through just fine. I tried to google around a bit and see if someone has had similar problems and found this (however, it does not really provide a useful solution for me).

https://stats.stackexchange.com/questions/185542/questions-with-fitting-a-dose-response-curve-using-drc-package-in-r

These are 2 examples of data that does and doesn't generate any errors:

#Data
Data_OK <- c(1.53148, 1.51851, 1.53148, 1.51851, 1.51188, 1.54407, 1.51851, 1.50515, 1.51851, 1.53148, 1.53148, 1.53148, 1.53148, 1.51851, 1.51851, 1.50515)

Data_NotOK <- c(1.51851, 1.50515, 1.49136, 1.49136, 1.50515, 1.47712, 1.47712, 1.49136, 1.51851, 1.47712, 1.49136, 1.50515, 1.50515, 1.49831, 1.49136, 1.50515)

c <- c(0.00000000 0.00015625 0.00031250 0.00062500 0.00125000 0.00250000 0.00500000 0.01000000, 0.00000000 0.00015625 0.00031250 0.00062500 0.00125000 0.00250000 0.00500000 0.01000000)

#Output Data_OK

library(drc)

fit_OK <- drm(Data_OK ~ c, fct = LL.5())
plot(fit_OK)

This will give you the following plot

Whilst this

#Output Data_NotOK

library(drc)

fit_NotOK <- drm(Data_NotOK ~ c, fct = LL.5())
plot(fit_NotOK)

Generates the following error

Error in optim(startVec, opfct, hessian = TRUE, method = optMethod, control = list(maxit = maxIt, : non-finite finite-difference value [4] Error in drmOpt(opfct, opdfct1, startVecSc, optMethod, constrained, warnVal, : Convergence failed

Interestingly, I noticed that the error doesn't occur if I change the concentration vector.

library(drc)

Data_NotOK <- c(1.51851, 1.50515, 1.49136, 1.49136, 1.50515, 1.47712, 1.47712, 1.49136, 1.51851, 1.47712, 1.49136, 1.50515, 1.50515, 1.49831, 1.49136, 1.50515)

c1 <- c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)

fit_NotOKc1 <- drm(Data_NotOK ~ c1, fct = LL.5())
plot(fit_NotOKc1)

I also tried to change the fit to LL.4(), LL.3() etc, but that does not seem to help...

Any idea why this occurs and how to solve it?

EDIT

I also found this now Regression of multiple dose-response curves using the drc package in R

where they suggest using try/catch, which I'm going to test. I'll post an update on how it goes. However, I would still like to get some sort of plot of the data. Any idea how to implement that into a loop?

Answer 1

Here is a solution using dplyr and tidyr on a tibble. The code uses your variables above. However, I recommend not using function names as variables.

The code first fits a model for each subset of the data, identifies the subsets that have a solution and then plots the results for both subsets in one curve.

library(dplyr)
library(tidyr)
library(tibble)
library(drc)

# Format data as a tibble
df = tibble::tibble(dose     = c(rep(c, 2), c1),
                    response = c(Data_OK, Data_NotOK, Data_NotOK),
                    group    = c(rep("OK",   length(c)),
                                 rep("FAIL", length(c)),
                                 rep("OK!",  length(c1))))

#' Wrap DR analysis in tryCatch to checks if there is a solution
#' @param .x tibble with response and dose columns
.has_solution = function(.x) {
  base::tryCatch(drc::drm(data = .x, response ~ dose, fct = LL.5()),
                 error=function(e){FALSE}) %>%
    base::isFALSE()
}

# Fit curve for each subset individually and only keep the ones that work
df = df %>%
  tidyr::nest(data=-group) %>%
  dplyr::mutate(use_it = purrr::map_lgl(data, ~ !.has_solution(.x))) %>%
  dplyr::filter(use_it)

# Unnest the data and fit again using the curveid parameter
df %>%
  tidyr::unnest(data) %>%
  dplyr::select(-use_it) %>%
  drc::drm(data=., response ~ dose, fct = drc::LL.5(), curveid = group) %>%
  plot()

Answer 2

For some reason to me it worked when I changed the values to be vectors.

ss97 <- c(st$AVG[68:75])
ss107 <- c(st$AVG[76:83])
ss110 <- c(st$AVG[84:91])

This worked, while

ss97 <- st$AVG[68:75]
ss107 <- st$AVG[76:83]
ss110 <- st$AVG[84:91]

gave me the following error:

Error in optim(startVec, opfct, hessian = TRUE, method = optMethod, control = list(maxit = maxIt,  : non-finite value supplied by optim
Error in drmOpt(opfct, opdfct1, startVecSc, optMethod, constrained, warnVal,  : 
Convergence failed