Log files are present on almost every computer system or networking device. They contain details about events happening on each system. They can prove invaluable when troubleshooting various issues. They can also reveal malicious activities and can, therefore, become a useful mean of ensuring security. But who has time to even look at the log files? With the typical administrator managing dozens of devices, some of them logging several events every second, there is no way anyone can keep track. This is why log monitoring tools were invented. They consolidate all event logs in a single location and often provide analysis tools and services that will go through the logs and raise alerts whenever something out of the ordinary is observed. Many different log monitoring tools are available and picking the best one can prove to be a challenge. To assist you, we’ve assembled this list of some of the very best log monitoring tools.
We’ll start off our discussion by exploring system logs, what they are and how they work. Next, we’ll talk about monitoring logs. Just like before, we’ll have a look at what it means and how it’s done. We’ll then provide you with more details about log analysis as this is the feature that makes log monitoring tools the most useful. Like before, we’ll describe what it is and the different forms of analysis that are available. Finally, we’ll review some of the very best log monitoring tools we could find and tell you about their main features.
System Logs In A Nutshell
In one sentence, a log file, or system log, is a file that records events that occur in an operating system or other software. Logging is the act of keeping a system log. In the simplest of cases, messages are simply written to a single log file. While most systems primarily use text files for logging events, some modern systems use some form of database to log them.
No matter how and where events are logged, some systems allow you to define the level of logging that you require. This is particularly true with networking equipment where each event has a severity level and logging parameters can be set to only record event of a certain severity level or higher. Other types of systems provide similar functionality as well.
About Monitoring Logs
Monitoring logs is a two-part process. The first—and the most important—part is the gathering of log data from various systems. This is accomplished in different ways. Some systems can be configured to automatically send logs to a centralized server through the Syslog protocol. Log Monitoring tools typically have a built-in syslog server to receive event data directly. Other systems, such as Windows, for example, work differently. Various means of acquiring log data from these systems exist such as using the Windows Management Instrumentation or using local agents running on Windows hosts. No matter how its done, every log monitoring system includes the required functionality to receive and consolidate log data from multiple sources.
The Next Step – Log Analysis
The second task of any useful log monitoring tool is the log analysis. This is where tools differ the most. Some will only offer very basic analysis such as triggering an alert when the number of events per unit of time reaches a given threshold. More advanced tools will examine each event and look for specific indications of problems. For instance, a large number of failed logins could be a sign of an ongoing intrusion attempt. We could spend pages describing the different forms of log analysis that are available. Instead, we invite you to have a look at the different product review below for details on what each one offers.
The Best Log Monitoring Tools
As we indicated earlier, there are many different tools available with varying degrees of functionality. Not everyone needs a tool with extensive analysis and high-security features so we included a mix of tools that provide various feature sets. Some are simpler tools while others are more complex. It is up to you to determine which tool offers the best fit for your needs. Fortunately, all of the tools on our list have a free trial available so nothing stops you from trying a few, something we’d highly recommend.
SolarWinds is a common name in the monitoring world. The company has been around for over 20 years and its flagship product, called the Network Performance Monitor, is recognized by many as one of the best SNMP monitoring tool available. And as if that wasn’t enough, SolarWinds is also known for its numerous free tools. These are smaller tools, each addressing a specific need of network administrators. The Advanced Subnet Calculator and the SolarWinds TFTP server are two excellent examples of these free tools.
As for the SolarWinds Log & Event Manager (LEM), it is exactly what its name implies. The tool is so feature-rich that many consider it as a full-fledged Security Information and Event Management tool. When it comes to monitoring and managing logs, it is likely one of the most interesting log management tools you can find. It has very useful log management and correlation features as well as an impressive reporting engine.
- FREE TRIAL: SolarWinds Log & Event Manager
- Download Link: https://www.solarwinds.com/log-event-manager-software/registration
The SolarWinds Log & Event Manager can help improve security and compliance by detecting suspicious activity and identifying threats faster with event-time detection of suspicious activity. You can also use the tool to conduct security event investigations and forensics for mitigation and compliance. This feature is why many consider the product as a SIEM tool. In addition, this tool helps with regulatory compliance readiness. You can use it to demonstrate compliance, thanks to its audit-proven reporting for HIPAA, PCI DSS, SOX, DISA STIG, and more.
The SolarWinds Log & Event Manager’s event response features leave nothing to be desired. The detailed real-time response system will actively react to every threat. Being based on behaviour rather than signature analysis means that you’re even protected against unknown or future threats. But the tool’s dashboard is possibly its best asset. With a simple design, you’ll have no trouble quickly identifying anomalies.
Pricing for the SolarWinds Log & Event Manager is based on the number of monitored nodes. Various levels of licenses from 30 to 2500 nodes are available starting at $4 665. And if you want to try the product before purchase, a free fully functional 30-day trial version is available for download.
Next on our list is another product from SolarWinds called the Log Manager for Orion. Orion, in case you’re not familiar with SolarWinds’ products, was the company’s top platform a few years back. It is still the underlying architecture on top of which many of SolarWinds’ best products are built. If you’re using any one of the Network Performance Monitor, the NetFlow Traffic Analyzer, the Network Configuration Manager, the Virtualization Manager, the Server and Application Monitor or the Storage Resource Monitor, you are using Orion.
- FREE TRIAL: SolarWinds Log Manager for Orion
- Download Link: https://www.solarwinds.com/log-manager-for-orion-software/registration
The SolarWinds Log Manager for Orion adds log management capabilities to any of the Orion-based monitoring and management tools. In summary, the product features powerful and intuitive log aggregation, tagging, filtering, and alerting. Its integration with the Orion platform products offers a unified view of IT infrastructure monitoring and associated logs. The product was created in collaboration with network and systems engineers to ensure their problems—and how to solve them—were understood.
Despite its integration with the Orion platform, the Log Manager can be installed by itself and does not require any other Orion tool to be installed. Pricing starts at $1 495 and a free 30-day trial version is available should you want to give the product a test run and see how it fits your needs.
Next is yet another product from SolarWinds called Papertrail. This one is very different from the previous two as it is a cloud-based, Software as a Service (SaaS) offering. The powerful tool was already enjoying some popularity when SolarWinds acquired it, a few years back. It aggregates log files from a multitude of products such as Apache or MySQL as well as Ruby on Rails apps, several cloud hosting services and other standard text log files.
To help diagnose bugs and performance issues, you can use the Papertrail very effective and lightning fast search engine which can search both stored and streaming logs. The product integrates with a few other SolarWinds products such as Librato and Geckoboard for graphing results. Papertrail is also easy to implement, use, and understand. It will provide you with instant visibility across all systems in minutes.
Papertrail is available under several plans including a free plan. It is somewhat limited and only allows 50 MB of logs each month. It will, however, allow 16 GB of logs in the first month which is equivalent to giving you a free and unlimited 30-day trial. Paid plans start at $7/month for 1GB/month of logs, 1 year of archive and 1 week of index. The $75/month plan with 8 GB of logs is the most popular. Noise filtering allows the tool to preserve data by not saving useless logs.
4. PRTG Network Monitor
The PRTG Network Monitor from Paessler AG is an integrated, all-in-one monitoring system that can be used to monitor almost anything, thanks to its clever sensor-based architecture. One of the best features of this is enterprise-grade product is certainly its setup speed. According to Paessler, the PRTG Network Monitor can be set up in just a couple of minutes. Although it might not be that fast for everyone, it is still one of the easiest and quickest monitoring tools to set up, thanks in part to its auto-discovery process.
The PRTG Network Monitor is a feature-rich product. At the base, it is primarily a network monitoring tool that uses SNMP to poll devices and display their interfaces utilization on chronological graphs. However, through the use of additional sensors, PRTG can monitor just about anything. Sensors are somewhat similar to add-ons except that they are included with the product. And there are available sensors for various servers, services and applications. In all, the product includes over 200 sensors.
For log monitoring and management, two different sensors are available. The Event Log Windows API sensor captures all the log messages that Windows generates. This sensor monitors the rate of log messages rather than their contents and it will generate an alarm if the rate of event log messages reaches a critical threshold.
The other interesting sensor, the Syslog Receiver sensor, receives, monitors, and saves syslog messages from any device. It won’t just aggregate logs from various sources, though. Its monitoring functionality will trigger alarms whenever worrying conditions arise, such as an increase in the rate of log reception.
The PRTG Network Monitor is available in two versions. The free version is full-featured but it will limit your monitoring ability to 100 sensors. When using SNMP, each monitored parameter counts as one sensor. For example, if you monitor two interfaces on a router, it will count as two sensors. Each instance of a specific monitoring sensor also counts as one. If you need more than 100 sensors, you’ll need to purchase a license which starts at $1 600 for 500 sensors. A free, sensor-unlimited and full-featured 30-day trial version is available.
5. ManageEngine EventLog Analyzer
ManageEngine is another well-known maker of network administration tools among IT professionals. The company offers a log management system called the ManageEngine EventLog Analyzer. The product collects, manages, analyzes, correlates, and searches through the log data of over 700 sources using a combination or agentless and agent-based log collection as well as log import.
The ManageEngine EventLog Analyzer’s capacity is impressive. It can processes log data at a rate of up to 25 000 logs/second and detect attacks in real time. The tool can also quickly perform forensic analysis thereby reducing the potential impact of a breach. The system’s auditing capabilities extend to the network perimeter devices’ logs, user activities, server account changes, user accesses, and more, helping you meet security auditing needs.
The tool’s real-time event log correlation instantly detects attack attempts and traces potential security threats by correlating log data with over 30 predefined rules to detect brute force attacks, account lockouts, data theft, web server attacks, and many more. It also features a custom log parser that can extract fields from any human-readable log format. The product truly provides a single console for viewing all your security log data.
The ManageEngine EventLog Analyzer is available in a feature-reduced free edition which only supports 5 log sources or in a premium edition which starts at $595 and varies according to the number of devices and applications. A free, full-featured 30-day trial version is also available.
Graylog is a free, open-source log management platform with plenty of interesting features. The tool can parse and enrich logs and event data from almost any data source. Its processing pipelines allow for some flexibility in routing, blacklisting, modifying and enriching messages in real-time. The tool will search through terabytes of log data to discover and analyze important information. Its powerful and rather unique search syntax lets you find exactly what you are looking for.
With Graylog, you have the ability to create customized dashboards which let you visualize specific metrics and observe trends from one central location. You can use field statistics, quick values, and charts from the search results page to drill down for deeper analysis of your data. In addition, the product offers the option to trigger actions or issue notifications upon events such as such as failed login attempts, exceptions or performance degradation.
Graylog is available either as a free and open-source limited version which also has limited support. There is also an enterprise version with extended features and unlimited support. It is free as well for up to 5GB of logs per day. Depending on how big and busy your network is. It could be enough for your need. License and support prices can be obtained by contacting Graylog sales.
7. WhatsUp Log Management Suite
The WhatsUp Log Management Suite is an excellent tool from Ipswitch. Ipswitch, is there need to remind you, is the company behind WhatsUp Gold, the super popular network monitoring tool. This one is an automated tool which collects, stores, archives and saves system logs, Windows events, and W3C/IIC logs. The doesn’t just aggregate logs and events, though, its continuous log surveillance and analysis will alert you of any abnormal activity.
The WhatsUp Log Management Suite will follow frequently audited events such as access rights and file, folder and object privileges and generate alerts as needed. It also uses collected events to build compliance reports for HIPAA, SOX, FISMA, PCI, MiFID, or Basel II compliance. This software can also help transform your raw log data into meaningful information for managers or IT security teams, using its powerful automated filtering, correlating, reporting, and converting features.
The WhatsUp Log Management Suite is actually a set of applications which include the following tools:
- Event Archiver: This tool automates log collection, clearing and consolidation.
- Event Alarm: A tool to monitor log files and receive real-time notification on key events.
- Event Analyst: Analyzes and reports on log data and trends; automatically distribute reports to management, security officers, auditors and other stakeholders.
- Event Rover: A unified console for in-depth forensics across all servers and workstations to increase efficiency and save time.
Pricing information for the Log Management Suite is not readily available from Ipswitch. The product can be purchased either directly from the publisher or through Ipswitch’s reseller network. A free trial version is, of course, also available.
LogDNA is said to be “the fastest, most intuitive, and cost-effective log management system”. This tends to be true. Right from the start, the product’s installation only takes a couple of minutes before you can start collecting and monitoring logs. No matter how logs are generated and transmitted, hundreds of custom integration schemes are available within the product to help you centralize logs into a single location.
LogDNA is available in either a cloud-based or a self-hosted version, depending on your preference. It is a highly scalable product that can handle hundreds of thousands of logs per second and dozens of terabytes per day while offering the utmost security as well as real-time log analysis. Both the company and its products are SOC2, PCI, and HIPAA compliant as well as being Privacy Shield certified.
LogDNA’s simple pay-per-GB pricing model eliminates contracts and fixed data allocations, which makes for one of the lowest total cost of ownership of any paid log monitoring and management solution. Several subscription plans are available with increasing features. The bottom-tier plan is free and prices for the paid plans vary from $1.50/GB/month to $3/GB/month depending on the retention duration and the number of users. A free, full-featured and unlimited 14-day trial is also available.