Future of logging infrastrucutres

During the last decade probably the logging infrasturcture has changed least, however this fieeld is very appreciated. The log is the book which contaions info as records about events on our system. Its target was oroginally fine-tuning and troubleshooting. Logging has a big tradition in POSIX environmnents, that is why we say UNIX is gassy. In the MSDOS/Windows world this was not always true, users had to find missconfigured options blindly.

We are controlling more and more valuable and important systems by compures from powerplants, tap water systems, cars and even the HVACs. If we are lucky, these systems send messages about unusual events as log entries. Serious systems every single parts - modules of th OS, applications, security subsystems and even network devices (routers and swtiches) are able to create such messages.

The world hase changed. Nowadays tons of regulations press for log collection and secure storage (sometimes years before). Not all the developers takes it seriously and it is a serious problem of harvesting storing logs. Finally the hardest part is analyzing these messages. Companies try to add appropriate answares to the problems. It is important to create company policy and task priority. After a careful planning the suitable logging infrastructure can help a lot. Finally do not forget support the IT staff doing the log management.

The infrastructure

If only we want the use our harvested messages well (analysis, statistics), it is essential a well oparating logging infrastructure, It contains tasks of log generations, collections and storing tasks. Itprepares them for the analysis, sends alerts if it is needed, prepares riports and integrates to the management system. A modern system helps the logprocessing with several functions:

• Parsing: breaking up smaller parts and recongizing;
• Filtering: handling messages uppon the content. It also contains classification, which is tagging messages (just like blog posts). According to the message tag we can do extra filtering. Artificial ingnorance can do filtering the unknown and unclassifies messages.
• Contraction: its target is storing repeating messages only once.
  
• Rotation: it prevents from too large logfiles.
• Archiving: the process of storing old messages on an external media.
  
• Compression: is for storing on smaller space. No need to explain.
• Conversion: changing between logging protocols or log message formats;
• Normalization: uniform messages parts (eg timestamps).
  
• Checking integrity: searching for unneeded change during message transport.
  
• Correlation analysis: searching for messages generated by an event on different path (eg. firewall, ids, network devices and aplication server).
  
• Display: a tool for viewing messages.
• Reporting: producing a "boss friend" doc about the log.
• Cleaning: deleting old messages from the system which can also law regulation too.

There are different standards for log transfer. The legacy syslog (RFC3164) and its impoved version (RFC5424-5428) are for online log trasfer in a unified way. The first one was designed to the old UNIX systems therefore it has lot solutins not very suitables for moden systems. The new protocol supports secure log transfer and supports windows speciality. There are also SIEM soultions, however these are not protocols. They sometimes use agents, sometimes not.

We also need a log storage and arhiving system, a diplay application and another for report genetration which can automatically create the needed riports.

Planning

Implementation of a logging infrastructure start with planning. Almoast all the system users has some connection with the logs.

• Admins and network team uses them for troubleshooting and fine tuning.
• Security admins are responsible for managing the infrastructure and prohibit disconnecting any system from the infrastructure therefore hiding any malicious activity.
  
• CSIRS are searching for evidences during a forensics analysis.
• Developpers are responsible for creating applications which produces suitable ammount of messages.
  
• Auditors usually checks system logs too.
  
• CIO and CSO are sesponsible for managing the logging infrastructure.
  
• The management also can support introduction a log infrastructure by commitment and budget too.
  

Except end-users everyone hase some connection to the logs, therefore it is essential to create a suitable log policy. It is important to define which parts of the system have to crate and transfer logs and how we have to secure harvested log messages. It also aplies to the archive. Finally defines viewpoints of analysis, access to them, maybe the reaction process of the events. The final policy must be cheked if it can be observed, flexible enough, suitable for the campany and laws.

When we have the policy we can plan the implementation. It is important to plan the expected ammount of logs and its load on the network. We also have to exactly define the trasport and storing secutity realization.

Management

We have to connect all the clients and servers to the implemented infrastructure. Security admins responsibility is to prepare all the systems and set them to keep the rules - generating and sending logs. From the point of log storage messages can be:

• Not stored messages, which hase minor meaning therefor we do not store them log term.
  
• Messages stored on the system, which has low priority, therefore long term storing and arhiving is not essential.
• Messages stored on the system and the infrastructure are the most valuable messages. We need to store and archive them.
• Messages stored only on the infrastructure are messages which cannot be stored on the system but they are valuable (eg. network devices).

The normal daily life conteins evenets and handling processes. It is very critical in logging. It is a natural need from the CSO not to handle (serve) any user request if logs cannot be prodced, the infrastructure is not available. On the other hand business processes needs full time availability in the face of not being able to store evidences. If any of the log storage is not available we have different choices:

• Stopping logging, but it is unacceptable from security reasons.
• Overwriting, which only acceptable on low priority messages.
• Stopping the application which can cause business loss.

It is also important to monitor the infrastructure and the log files and sending alerts. To create a complete protection we need:

• Control who can access log messages.
• Avoid logging sensitive and unneccessary messages.
• Secure archiving.
• Harden the application which produces business critical messages.
  
• Set the applications carefully and look for the applications reactions to the errors of log infrastructures.
  
• Use secure protocols for message trasfer.

Important part of log management is analysis. Its target is after filtering messages of the ususal activity finding mailious or unusual events. Its base is collenting and understanding and finding coherent logs. Coherency is not only meant in messages of only one application but messages of every system every messages. This is called correlation. It is not a easy job at all! Just like priorizing them which is not of necessity has the same priority as the admins handle them. It is godd if the log infrastrucutre can set this priority or filter according to them. Finally we need to react, sending alerts to the management system therefore admins or CSIRT can react immediately.

Finally harvested messages must be stored. Out-to date messages must be deleted or archived:

• Choose suitable format. If we need to read them 5 to 10 years later and we need to understad them, we need a suitable format.
• Check the integrity of the archive and if we can reread.
• Store the media in a safe palce eg. in a trezor.

Collected messages need to reported daily and send them who concerned. To make the report we need to define viewpoints which defines selections. There are reporting applocations available on the market which produces the reports, sometimes comliant to the regulations (eg. SOX report).

Finally we need to test the infrastructure. During the passive test we check the configuration and the infrastructure, while the active test tries them with real messages.

Summary

Logging is a valuable part of our systems today. However implementation and management is not an easy job, a system cannot be build without it. We need to stress the importance of the planning regarding to internal and extrenal regulations, laws. If we have a suitable logging infrastructure it can help to build a successful company. Be honest! Is sour present system has remind you to my sketch?