In this article, we learn how to manage and configure logs using logrotate. This can be useful since logs can eat up a lot of space just in few days. Even at work, I faced this issue when one service was generating > 1GB of logs in a day, which is when I learned about logrotate.

We will also explore an example where we configure Postgres logs.

---

What is Logrotate?

From the man page -

Logrotate is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large.

That explains it pretty well, this system utility is used to manage how logs are to be stored (based on time, size, etc). It handles compression of log files. This is important because if logs are not rotated and eventually removed after a period of time, the whole of your disk space will be consumed by logs.

In Unix, the default lograte file in stored at /etc/logrotate.conf Default configs look like this -

# see "man logrotate" for details
# rotate log files weekly
weekly

# use the syslog group by default, since this is the owning group
# of /var/log/syslog.
su root syslog

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}

/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

# system-specific logs may be configured here

Apart from the config file, there is also a directory /etc/logrotate.d that contains the packages you install that need help with log rotation. Eg. nginx, postgresql-common,dpkg etc. This directory contains config files with the name of the package i.e these package-specific configs are only applicable to a particular package.

In case you want to explore more on logrotate, check the resources section at the end of the article.

---

Managing Postgres logs with Logrotate

In the previous section, we learned where are the configs stored. Let’s check the logrotate configs for postgres.

cat /etc/logrotate.d/postgresql-common

The default configs would be like this -

/var/log/postgresql/*.log {
       weekly
       rotate 10
       copytruncate
       delaycompress
       compress
       notifempty
       missingok
       su root root
}

Let me explain some of the parameters used above -

• weekly - Log files are rotated every week. You can set this option to daily, monthly, yearly also.
• rotate 10 - This means that Log files are rotated 10 times before they are removed. In case you want to store all logs, you can also automatically configure to mail the logs which are going to be removed. In case you don’t want to store any past logs, just set the count to 0.
• compress - Older log files are first compressed and then stored. By default, gzip is used. You can always unzip to read the logs later using gzip -d filename.gz
• notifempty - It tells logrotate to avoid rotation in case the original log file is empty. (This can happen if there are no logs being produced)

Now you understand the basic usage of logrotate. Let’s see another scenario where you want to configure logrotate.

Rotate based on the size of log files

You can use the size option in logrotate for this. Eg. size 100k, size 10M for 100KB and 10MB respectively.

Create log files with specific permission

For this, we can specify with create option. Eg -

**create**  _mode owner group_
# example
create 0640 mohit mohit

Mail log files before removal

As mentioned above, if you want to store all logs you can simply set up a mail where these logs will be sent.

mail yourEmail@domain.com
# Example
mail code@xyz.com

Suppose the rotate count is set as 10 then when the number of logs reaches 11, the oldest log will be mailed to the provided address.

This is how I keep my Postgres log configs -

/var/log/postgresql/*.log {
       weekly
       size 100M
       rotate 8
       missingok
       compress
       delaycompress
       nomail
       notifempty
       create 0660 root root
}

You can also configure logs using postgres configs. Check details here (runtime logging configs).

Note : Logrotate runs based on cron jobs. If you want to do a force reload, this can be achieved manually using -

# for postgres
sudo logrotate -vf /etc/logrotate.d/postgresql-common

---

Resources

• Man page
• Manage Logfiles with Logrotate
• Postgres - Where are my logs

Awesome! You just learned how to manage logs using the lograte command. Do try to configure your logs according to your need as an exercise!

Liked the article? Consider supporting me ☕️

---

I hope you learned something new. Feel free to suggest improvements ✔️

I share regular updates and resources on Twitter. Let’s connect!

Keep exploring 🔎 Keep learning 🚀