wai-logger FileLogSpec as Scotty Middleware reporting openFile: resource busy (file is locked)

Question

I am using Scotty for a backend application in Haskell and I am interested in logging all requests to a file. The existing wai-middleware requestlogger is not enough as I would like the properties of FileLogSpec, since I like have multiple but short logfiles, which FileLogSpec permits.

I am also open to alternative solutions on the logging issue, but I am not interested in switching from Scotty.

My problem is, that after the first request I get this error:

openFile: resource busy (file is locked)
openFile: resource busy (file is locked)

and the application stops logging entirely.

My current implementation making wai-logger into a middleware is:

-- type Middleware = Application -> Application
-- Type Application = Request -> (Response -> IO ResponseReceived) -> IO ResponseReceived
mw :: Wai.Middleware
mw (app::Wai.Application) (req::Request) (fRes::(Response -> IO ResponseReceived)) =
    app req myResponseFunction
    where
        myResponseFunction :: Response -> IO ResponseReceived
        myResponseFunction response = do
            logger' <- logger
            logger' req (Wai.responseStatus response) Nothing
            fRes response
        logger :: IO Logger.ApacheLogger
        logger = createLoggerActions >>= return . Logger.apacheLogger
        createLoggerActions :: IO Logger.ApacheLoggerActions
        createLoggerActions = let
            -- 5 MB in bytes = 5242880
            mb5InBytes = 5242880
            -- From fast-logger default
            defaultBufSize = 4096 in
                Logger.initLogger
                Logger.FromFallback
                (Logger.LogFile Logger.FileLogSpec {Logger.log_file = "reqLogger", Logger.log_file_size = mb5InBytes, Logger.log_backup_number = 5} defaultBufSize)
                ( return $ B.pack "%d/%b/%Y:%T %z")

I have tried using bracket to clean up the logger as follows:

mw :: Wai.Middleware    
mw (app::Wai.Application) (req::Request) (fRes::(Response -> IO ResponseReceived)) =
    let loggerActions = createLoggerActions in
        bracket_ setup (teardown loggerActions) $ 
        app req (myResponseFunction loggerActions)
    where
        myResponseFunction :: IO Logger.ApacheLoggerActions -> Response -> IO ResponseReceived
        myResponseFunction loggerActions response = do
            logger' <- logger loggerActions
            logger' req (Wai.responseStatus response) Nothing
            fRes response
        logger :: IO Logger.ApacheLoggerActions -> IO Logger.ApacheLogger
        logger loggerActions = loggerActions >>= return . Logger.apacheLogger
        createLoggerActions :: IO Logger.ApacheLoggerActions
        createLoggerActions = let
            -- 5 MB in bytes = 5242880
            mb5InBytes = 5242880
            -- From fast-logger default
            defaultBufSize = 4096 in
                Logger.initLogger
                Logger.FromFallback
                (Logger.LogFile Logger.FileLogSpec {Logger.log_file = "reqLogger", Logger.log_file_size = mb5InBytes, Logger.log_backup_number = 5} defaultBufSize)
                ( return $ B.pack "%d/%b/%Y:%T %z")
        setup :: IO ()
        setup = return ()
        teardown :: IO (Logger.ApacheLoggerActions) -> IO ()
        teardown loggerActions = loggerActions >>= Logger.logRemover

And here is how I use it:

someFunc :: IO ()
someFunc = do
    appConfig <- loadAppConfig
    workingDir <- Dir.getCurrentDirectory
    putStrLn $ "Working directory: " <> workingDir
    case appConfig of
        Nothing -> putStrLn "Failed to load appconfig.json"
        Just appConfig' ->
            let checkSizeRequirement' = checkSizeRequirement (sizeLimitGB appConfig') in
                scotty 3000 $ do
                    --middleware logStdoutDev
                    middleware $ mw
                    middleware $ staticPolicy (addBase "./ElmFeFMI/")
                    post "/api/fmichecker" ...

Unfortunately, this led to the same result.

I have created an issue on wai logger as well: https://github.com/kazu-yamamoto/logger/issues/183

The full source is available here: https://github.com/into-cps-association/utilities_backend

Answer 1

sicklorkin from irc freenode #haskell responded with the rather obvious solution -

Make sure you havn't accidentally opened the logger twice

and asked me to try passing in the logger.

So the new and working solution is:

someFunc :: IO ()
someFunc = do
    appConfig <- loadAppConfig
    workingDir <- Dir.getCurrentDirectory
    putStrLn $ "Working directory: " <> workingDir
    fileLogger <- createLogger 
    case appConfig of
        Nothing -> putStrLn "Failed to load appconfig.json"
        Just appConfig' ->
            let checkSizeRequirement' = checkSizeRequirement (sizeLimitGB appConfig') in
                scotty 3000 $ do
                    --middleware logStdoutDev
                    middleware $ (mw fileLogger)

And further down the mw has been split from createLogger:

mw :: Logger.ApacheLogger -> Wai.Middleware
mw logger (app::Wai.Application) (req::Request) (fRes::(Response -> IO ResponseReceived)) =
    app req myResponseFunction
    where
        myResponseFunction :: Response -> IO ResponseReceived
        myResponseFunction response = do
            logger req (Wai.responseStatus response) Nothing
            fRes response

createLogger :: IO Logger.ApacheLogger
createLogger = createLoggerActions >>= return . Logger.apacheLogger