Log File Rotation on Linux

Logging in Linux

In the world of system administration, managing log files is a crucial task. Logs provide valuable insights into the health, performance, and security of your Linux system. However, if left unchecked, log files can consume valuable disk space, degrade system performance, and become challenging to analyze effectively. This is where log file rotation comes to the rescue, and when it comes to log rotation on Linux systems, rsyslog is an indispensable tool.

Many programs use the syslog protocol to log events to the system. rsyslogcomes by default in many Linux distros.  rsyslog –   “rocket-fast system for log processing,” is a powerful and flexible log management system used in Linux environments. Run the command below to verify its prescence.

[root@rocky9 ~]# rsyslogd -v
rsyslogd  8.2102.0-113.el9_2 (aka 2021.02) compiled with:
  PLATFORM:				x86_64-redhat-linux-gnu
  PLATFORM (lsb_release -d):		
  FEATURE_REGEXP:				Yes
  GSSAPI Kerberos 5 support:		Yes
  FEATURE_DEBUG (debug build, slow code):	No
  32bit Atomic operations supported:	Yes
  64bit Atomic operations supported:	Yes
  memory allocator:			system default
  Runtime Instrumentation (slow code):	No
  uuid support:				Yes
  systemd support:			Yes
  Config file:				/etc/rsyslog.conf
  PID file:				/var/run/rsyslogd.pid
  Number of Bits in RainerScript integers: 64

See https://www.rsyslog.com for more information.

Understanding Log Rotation

Log rotation is the process of managing log files by periodically creating new log files, compressing or archiving old ones, and deleting outdated logs. This practice ensures that logs in the /var/log  directory remain manageable in size, making it easier to diagnose issues, ensure that we dont run out of space due to the continous growth of the log files, track system behavior, and comply with regulatory requirements.

When a log file is rotated, it is renamed with an extension indicating the date it was rotated. For instance, the old /var/log/messages file may become /var/log/messages-20230927 if it is rotated on 2023-09-27. When rotation occurs on the old file, a new log file is created and the service that writes to it is notified.

Typically, after four weeks, the oldest log file is discarded to free disk space. There is a cron job that runs logrotate daily to check whether there any logs that needs to be rotated – mostly based on their sizes.

Configuration of Log File Rotation on Linux

Step 1: Install rsyslog (if not already installed)

#### On Debian Distros
sudo apt-get update
sudo apt-get install rsyslog

### On RHEL based Distros
dnf install logrotate

Step 2: Configure Log Rotation Policies

Navigate to the rsyslog configuration directory, usually located at /etc/rsyslog.d/. Create a new configuration file, for example, mongo_log_rotation.conf, and define your log rotation policies. Here’s an example configuration to rotate logs weekly and keep a maximum of 7 rotated files:

# /etc/rsyslog.d/mongo_log_rotation.conf
/var/log/mongodb/mongod.log {
    rotate 7
    weekly
    size 10M
    compress
    missingok
    notifempty
    create 0644 mongod mongod
}

Understanding the above directives:

• weekly  – Log  files are rotated once each weekday, or if the date is advanced by at least 7 days since the last rotation
• rotate 7   – Log files are rotated 7 times before being removed or mailed to the address specified in a mail directive.
• size 10M   – Log files are rotated only if they grow bigger than 10M.
• compress   – Old versions of log files are compressed with gzip by default.
• missingok  – If the log file is missing, go on to the next one without issuing an error message.

• notifempty – Do not rotate the log if it is empty (this overrides the ifempty option).

• create      – mode owner group. Immediately after rotation, the log file is created (with the same name as the log file just rotated). mode specifies the mode for the  log  file  in  octal. owner specifies the user who will own the log file, and group specifies the group the log file will belong to(mongod:mongod).

Step 3: Restart rsyslog

After the configuration is done, restart the rsyslog service to effect the changes.

# service rsyslog restart
Redirecting to /bin/systemctl restart rsyslog.service

Step 4: Verify Rotation

Monitor the log rotation process by observing the log file and its rotated versions in the specified directory i.e /var/log/mongo.

Conclusion

Log rotation is an essential practice in maintaining a healthy and well-managed Linux system. By following the steps outlined above, administrators can ensure that their log files remain manageable, disk space is used efficiently, and critical log data is preserved for analysis and compliance purposes.

Check out this:

• How To Install PIP Python Package Manager on Linux