Security Monitoring and Tracking Standards

Question

I am working on a system that handles authentication/authorization, and needs to keep track of the attempts of individuals logging in, making changes to the permissions/users, failed attempts, etc. We want to be able to parse this information into a database for further analysis/retrieval at a later time.

In our current implementation, we are using a home-brewed standard that is logged using a logging framework (Log4j in this case, but that is not what is important). Is a Logging framework the right mechanism to keep track of this information? It seems to me like it isn't; I had always understood logging to be a form of autopsy for the code - more to tell what happened when for purposes of debugging etc. This seems more like a reporting mechanism to me. Are there any standards for this type of a problem? Are there standard solutions/formats that people use? Is using a logging framework the right solution for this, or is there a better way to handle this type of data? What sources can I reference when looking at this information and presenting it to the stakeholders?

I should note - the data that is being logged is already being filtered based on compliance/security standards (no passwords, etc) and all of the logging occurs in our internal environment. I am more looking for a way that we can manage the change information of the authentication and authorization system.

Answer 1

It seems like you are using log4J for auditing (and probably for logging diagnostic or tracing information as well). To answer your question:

Is a Logging framework the right mechanism to keep track of this information?

the straightforward answer is "No, a logging framework is not the right mechanism". There are certain attributes, which if present in a logging framework, would lend it the capability of being used as an audit framework.

Some of these requirements are presented below, and log4j can be used to cater to some of these. This is not exhaustive, and I would recommend you to look at the implementation of auditing frameworks (like LAUS) to derive a more comprehensive list.

• An audit framework must ensure fail safe auditing of events. This might depend on how the application uses the framework, but the underlying requirement is that if auditing fails, then so should the application. No attempt should be made by the application to process any request if the event cannot not be audited. A logging framework typically fails to adhere to this requirement.
• An audit framework should ideally provide a store that is write-once and read-only. In other words, events written to the audit log must and should not be erased. An audit framework usually does not implement this protection on it's own instead relying on a combination of other factors to ensure that the log is tamper resistant.
• An audit framework should allow for storage of audit logs on a different system. This would ensure that a compromise of one system, does not automatically result in a compromise of the audit logs.
• The framework should also allow for capture of important information, and should ideally not leave this to programmers. Important information would constitute the timestamp from a synchronized timesource, the user responsible for the request (or any information to identify the user), the source of the request, the status of the request (whether it succeeded or failed), any errors encountered during processing of the request etc.

Answer 2

Logging framework is not nesseccarily a bad thing to use here. Logging to a file is usually faster than logging to a database. Though, with Log4j you can implement or use existing JDBC appender, that will insert all your data into database as it is being logged. For reliabilty you can have both file and database appenders configured for your audit logging to have some back up in case logging database fails.

Other alternatives could be AOP aspects around your security/business logic that will insert data directly into logging database.

I don't think there is any common standard for this kind of data though.