Log files in Linux systems serve as chronological records of events, errors, and activities. These files play a crucial role in Linux systems by recording system events, errors, and activities.
They provide a detailed record of the system’s events, offering valuable insights for troubleshooting, monitoring, and auditing.
Administrators frequently refer to log files as the starting point of diagnosing issues, tracking system performance, and ensuring the security and reliability of their Linux environments.
From identifying hardware or software errors to detecting security breaches or unauthorized access attempts, log files are vital for maintaining the health and integrity of Linux systems.
As you will see in this guide, log files are essential for understanding the state and behavior of the system and facilitating effective management and maintenance of Linux environments.
In this comprehensive guide, we will take a close look at Linux log files and how you can set up and view these files in Ubuntu and CentOS environments.
Table Of Contents
- What are Linux Log Files
- How to View Linux Logs
- Important Linux Logging Aspects
- Significant Logs for Linux Systems
- How to Set Up Log Files in CentOS and Ubuntu?
- Conclusion
- FAQs
What are Linux Log Files
Log files in a Linux system can be divided into the following categories based on the type of information stored in these files. Note that almost all events are time-stamped with the date and time of the event.
Event Logging
These log files generally contain the records for all system events, especially the following:
- System startup
- System restart/shutdown
- User login
- Application and script launches
Error Logging
These log files contain all error messages and alerts generated during system operation. Entries in these logs are detailed and include associated error codes, stack traces, and associated information required for troubleshooting.
Activity Logging
This set of logs contains information about the user activity on the system. These logs are critical for investigating issues connected to file access and permission issues. You will find the following specific information in these logs:
- User activities, including file access, permission changes, and system configuration modifications
- User terminal commands (useful for security audits)
- System-level service activities such as web server requests, db queries, cron job management, and execution
Security Logging
These logs contain security-related information that you can use in security event investigations. Security audit software use these blogs to detect and flag security incidents and potential breaches.
In particular, you will find the following information in these logs:
- Unauthorized access attempts to access the system
- Failed logins and authentication failures
Custom Logging
In addition to system-level logs, Linux environments allow individual applications to create their custom logs and record application information and incidents. In rare cases, the information is duplicated in system and application logs.
You will find the following information in these custom application logs:
- Application-specific metrics
- User interactions with the application
- Debug information and messages
The Location(s) of Log Files
When working with log files in Linux systems, the first thing you should know is the location of these files. In most Linux distributions, log files are typically stored in the /var/log, log directory, or its subdirectories. Here are some default locations for system and application logs:
/var/log/messages: General system messages and kernel logs./var/log/syslog files: has System-wide syslog messages./var/log/auth.log: Authentication-related logs, including user logins and sudo activity./var/log/secure: Security-related logs, particularly on systems using the rsyslog service./var/log/apache2/: Apache web server logs./var/log/mysql/: MySQL database server logs./var/log/nginx/: NGINX web server logs./var/log/journal/: systemd journal logs (on systemd-based systems).
How to View Linux Logs
Now that you know the types of logs in a Linux system and where to find them, let’s see how to view Linux logs.
Linux provides several ways of reading Linux logs through commands and tools. This presents a degree of flexibility that allows you to select the right combination of utilities to fit your requirements.
But first, let’s look at the prerequisites:
The Prerequisites
- A system with a mainstream Linux distribution
- A user account with root and sudo privileges
View Linux Log Files With the cat Command
You can use the cat command to display the entire contents of a log file. For example, run the following command to print the contents of the syslog log file:
# cat /var/log/syslog
Use Linux Log Files With the less Command
If you wish to view log files interactively, use the less command to benefit from scrolling and search capabilities. For instance, consider the following command to view the messages log:
# less /var/log/messages
You can close the command output by simply pressing q.
Use the tail command for Real-time Monitoring
By default, the tail command prints the last ten lines of the specified file. This means you can get a real-time view of the log updates by using the command to print the most recent entries.
For example, consider the following command that offers a near real-time view of the auth log:
# tail /var/log/auth.log
Use the grep Command to Filter Log Entries
Log files are usually long, and finding specific information items can take some time. In such cases, we suggest using the grep utility to filter log entries based on specific patterns or keywords. For instance, run the following command to search for interrupt statements in the kernel log:
# grep "interrupt" /var/log/kern.log
Use the journalctl Command
If you are on a systemd-based distribution, we recommend using the journalctl command to view logs from the systemd journal.
For this, run the journalctl command in the terminal:
# journalctl
View the Kernel Buffer Messages
The dmesg command displays kernel ring buffer messages, providing insights into kernel-level events and errors. The output of the command can be long, so we recommend piping it into less for better viewing experience:
# dmesg | less
Important Linux Logging Aspects
Working with Linux logs requires considering the following factors and aspects.
1. Levels of Logs
All log entries are not equal because of the severity and urgency of information included in these entries. Overtime, the following scale has emerged to help administrators prioritize their action plan:
- Emergency: The most severe level that denotes a system malfunction.
- Alert: Signals the need for quick action.
- Critical: Denotes a serious situation that you need to attend ASAP.
- Error: Points to a condition that requires attention.
- Warning: Points to a possible problem or situation that requires your attention.
- Notice: Signals typical yet noteworthy occurrences.
- Info: Broad announcements of information.
- Debug: Comprehensive information you can use for debugging.
2. Facilities for Logging
Log facilities classify logs according to the subsystem or software that produces them. Among the frequently-used utilities are daemon, syslog daemon, mail, kernel, cron, auth, and local0 through local7. Facilities assist in filtering and organizing logs.
3. Log Records
Usually, Linux logs are kept in one of the subdirectories under /var/log. Different software components and distributions may have different log file locations and naming patterns. Common log files include syslog, auth.log, kern.log, messages, secure, apache2/error.log, and mysql/error.log.
4. Rotation of Logs
Linux provides log rotation utilities to control log file sizes and stop them from using too much disc space. These programs automatically rotate and archive log files according to predefined standards, including size, time, or number of entries. Logrotate is the most common log rotation tool in Linux.
Significant Logs for Linux Systems
Logs provide valuable insights into a system’s operations. Understanding each type of log file is crucial for distinguishing and interpreting their respective contents accurately.
Application Logs
- Location: Varies by application; often found in /var/log/.
- Description: Application-specific logs, detailing the behavior and issues related to various programs. Examples include web server logs (Apache, NGINX), database logs (MySQL, PostgreSQL), and email server logs (Postfix, Sendmail).
Authentication Logs
- Location: /var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RHEL/CentOS/Fedora).
- Description: Records authentication activities, such as user logins, sudo commands, SSH logins, and other authorization mechanisms. Useful for security auditing and identifying unauthorized access attempts.
System Logs
- Location: /var/log/syslog (Debian/Ubuntu) or /var/log/messages (RHEL/CentOS/Fedora).
- Description: Contains a broad range of system messages, including those from the kernel, system daemon log, and other low-level system components. This log is crucial for general troubleshooting.
Kernel Logs
- Location: /var/log/kern.log.
- Description: Dedicated to messages generated directly from the Linux kernel. Useful for diagnosing hardware and driver issues and some low-level system errors.
Boot Logs
- Location: /var/log/boot.log and the standard output of dmesg.
- Description: Contains messages related to the system boot process. Useful for troubleshooting issues occurring during the system startup sequence.
Package Management Logs
- Location: /var/log/dpkg.log (Debian/Ubuntu) or /var/log/yum.log (RHEL/CentOS/Fedora).
- Description: Records activities related to package management, such as software installations, updates, and removals. It helps track changes made to the system software.
How to Set Up Log Files in CentOS and Ubuntu?
The process of setting up log files and a logging mechanism is similar in all major Linux distributions, including Ubuntu and CentOS. Here are the major steps of the process:
Step #1: Specify Log File Locations
Determine the log files you want to configure and their respective locations. We recommend sticking to common log file locations (such as /var/log/) and applications subdirectories to ensure compatibility with log utilities and applications.
Step #2: Configure Log Rotation
We will use logrotate as the main log rotation utility. Here are the steps involved in the configuration of this step.
You might need to install logrotate if it’s not available on your system. On most Linux distributions, you can install it using the system’s package manager. Since we have Ubuntu on our test system, we will run the following command in the terminal:
# sudo apt-get install logrotate
logrotate uses configuration files to define how logs should be rotated. The main configuration file is usually located at /etc/logrotate.conf, with additional custom configurations stored in /etc/logrotate.d/.
To set up log rotation, you must define a scheme in a configuration file. This includes specifying the log files to rotate, the rotation interval (daily, weekly, monthly), the number of rotations to keep, and other options like compression.
Note that logrotate can compress old log files to save space. You need to enable this option in the configuration. Similarly, you can opt to enable emailing log files before they are rotated. For this, we recommend changing the mailfirst or maillast options to control whether the email is sent before or after rotation.
Finally, you can execute custom scripts before or after log rotation. For this, use the prerotate/postrotate script blocks.
Here’s an example configuration that demonstrates some of these features:
/var/log/myapp/*.log {
daily
rotate 7
compress
missingok
notifempty
create 640 root root
mail [email protected]
maillast
prerotate
# Script to execute before log rotation begins
/usr/bin/myscript.sh pre
endscript
postrotate
# Script to execute after log rotation
/usr/bin/myscript.sh post
endscript
}
This configuration specifies that logs in /var/log/myapp/ should be rotated daily, kept for 7 days, compressed, and emails sent after rotation. It also includes pre- and post-rotation scripts.
Step #3: Setting Up Log Levels
Log levels determine the severity of messages in the logs. You can set up these levels to determine the nature and impact of the log entries. You can configure log levels for individual components, services, and applications. Additionally, you can control the verbosity of information in the logs.
You can set the log level in their configuration files for many applications and services (like web servers and databases). The specific setting and syntax will depend on the application. For example, in an Apache httpd.conf file, you might set:
LogLevel warn
This tells Apache to log warnings, errors, and critical messages but ignore info and debug messages.
Step #4: Define Log Formats
You can customize log formats to suit your preferences and requirements. FOr this, we recommend using command line tools like rsyslog or syslog-ng for advanced log management and customization.
You can set the log formats in config files, such as /etc/rsyslog.conf.
For instance, Apache allows you to define custom log formats in its configuration file, typically located at /etc/apache2/apache2.conf (Debian/Ubuntu) or /etc/httpd/conf/httpd.conf (CentOS/RHEL).
We suggest opening the file in your preferred text editor and using the LogFormat directive to define a custom format and the CustomLog directive to apply it. For example:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" custom
CustomLog /var/log/apache2/custom_access.log custom
Step #5: Test and Monitor the Logging Process
After configuring the logging process, we recommend testing the functionality to ensure logs are generated as expected. For instance, you should monitor log files regularly for system health, security incidents, and application errors. For this, use tools like tail, grep, journalctl, and log management platforms for efficient log analysis and monitoring.
Documentation and Maintenance
Document the log file configurations for future reference, including locations, rotation settings, log levels, and formats. Regularly review and update log file configurations as system requirements change or new applications/services are deployed.
Conclusion
Mastering the process of viewing and interpreting Linux log files is essential for effective system management. By utilizing commands like cat, less, tail, grep, journalctl, and dmesg, administrators gain valuable insights into system events, errors, and activities. Whether troubleshooting issues, monitoring system performance, or analyzing security incidents, the ability to navigate and understand log files is paramount. With a solid grasp of log file management techniques, Linux administrators can ensure their systems’ stability, security, and reliability.
Ready to dive into Linux log files for your high-config bare metal servers? Discover how to master the command line with our guide!
Explore bare metal hosting solutions with RedSwitches. Enjoy a perfect hosting experience with RedSwitches, the world’s leading dedicated hosting company.
We offer the best dedicated server pricing and deliver instant dedicated servers, usually on the same day the order gets approved. Whether you need a dedicated server, a traffic-friendly 10Gbps dedicated server, or a powerful bare metal server, we are your trusted hosting partner.
FAQs
Q. What is Linux logging?
Linux logging is recording events and activities on a Linux system. It helps administrators track system behavior, identify issues, and troubleshoot problems. Various components and services generate logs, providing valuable information about system status, user activities, network events, etc.
Q. How are log files stored in Linux?
Log files in Linux are stored as plain text files. Each log file contains information about a specific aspect of the system or application. Linux log files can be found in the /var/log directory, and different files serve different purposes, such as system logs, service logs, authentication logs, and more.
Q. How can I view Linux log files using the command line?
To view Linux log files using the command line, you can use the “less” or “cat” command. For example, to view the content of a log file named “example.log,” you can run the command “less example.log” or “cat example.log.”
Q. How can I view the contents of a specific log file?
To view the contents of a specific log file, you need to specify the file path in the command. For example, to view the content of the system log file (/var/log/syslog), you can use the command “less /var/log/syslog” or “cat /var/log/syslog.”
Q. How can I rotate log files in Linux?
To rotate log files in Linux, you can use the logrotate utility. The logrotate utility allows you to specify the rotation frequency, compression options, and more. By default, logrotate is typically configured to rotate log files weekly and keep a certain number of backup files.
Q. How can I centralize Linux logs from multiple systems?
To centralize Linux logs from multiple systems, you can use a centralized logging solution such as syslog-ng or rsyslog. These tools allow you to configure a central log server and collect log messages from remote systems over the network. Centralized logging can help streamline log analysis, troubleshooting, and monitoring of multiple Linux systems.
Q. How do I read Linux log files with regular expressions?
Reading Linux log files with regular expressions can be done using tools like grep or awk. Regular expressions allow you to search for patterns or keywords within the log files. For example, you can use the command “grep ‘ERROR’ /var/log/syslog” to search for log entries containing the word “ERROR” in the system log file.
Q. What is the default log daemon in Linux?
The default log daemon in most Linux distributions is syslogd or rsyslogd. These log daemons log events and messages generated by the system and applications.
Q. How can I monitor system log files in real time?
To monitor system log files in real-time, you can use tools like tail or journalctl. For example, you can run the command “tail -f /var/log/syslog” to continuously monitor the system log file and display new log entries as they are added.
Q. How can I analyze log files on Linux?
Analyzing log files on Linux can be done using various tools and techniques. You can use tools like grep, awk, or sed to extract specific information from log files. Additionally, log analysis tools provide advanced features like log parsing, filtering, and visualization.
