Using an external service or agent to Log all the file accesses and processing done my Java web-app?

Question

I work to maintain an Enterprise Java web app. Its extremely configurable, with over thousands of config files and xml definitions.

Background

At times, some config file gets corrupted leading to a blocking exception. The file name and file path are generally not logged in our current framework. Making it extremely difficult to identify on client side which config file is corrupted as there are thousands of them. So we want to log all the file accesses and processing done by our app.

What has been done -We have created two debug logs and one error log using log4j. -We log each file access in config processing or xml processing classes. -We log try catch any exception that might occur. This require manually refactoring code in hundreds of classes.

Code

    public static final Logger log = Logger.getLogger(ThisLoggerClass.class);
    
    public static void logConfigFileException(Exception ex , String fileName,  String fullPathToFile) {
        log.error(" file: [ " + fileName + " ] was either corrupted or not present at [ " + fullPathToFile + " ] causing error: " + ex.getMessage(), ex);
    }
    
    public static void logFileAccessForRead(String fileName, String fullPathToFile) {
        log.debug("file: [ " + fileName + " ] was read from the location: [ " + fullPathToFile + " ] ");
    }
    
    public static void logFileAccessForWrite(String fileName, String fullPathToFile) {
        log.debug("file: [ " + fileName + " ] was written to the location: [ " + fullPathToFile + " ] ");
    }

}

REAL PROBLEMS

As can be seen its very had to manually refactor thousands of classes that access and then process thousand of files in different manner.

• Developer might skip places.
• Excessive try-catch makes code unreadable
• Try catch has to be done taking lots of things in account eg: like should be rethrow ex or not (requires deep untestable of each code flow)

DESIRED SOLUTION

Log all the files written or read by our app with timestamps through an external agent or app.

Are there any external agents or apps that can Log all the file access activity done by a process or app on the server?

Answer 1

There are a few things you could do. Which is best depends on your application.

1. Find out why files are getting corrupted and fix that problem.
2. Have a configuration checking stage at application startup which checks all your configuration files to check that they are valid XML. This avoids the need to catch exceptions where your existing code reads the configuration.
3. Wrap all the java.io classes you use so that they log accesses without changing your existing code (other than changing the import statements)

I'm sure there are other approaches too.