Get latest articles directly in your inbox
In this article, we learn how to manage and configure logs using logrotate. This can be useful since logs can eat up a lot of space just in few days. Even at work, I faced this issue when one service was generating > 1GB of logs in a day, which is when I learned about logrotate.
We will also explore an example where we configure Postgres logs.
What is Logrotate?
From the man page -
Logrotate is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large.
That explains it pretty well, this system utility is used to manage how logs are to be stored (based on time, size, etc). It handles compression of log files. This is important because if logs are not rotated and eventually removed after a period of time, the whole of your disk space will be consumed by logs.
In Unix, the default lograte file in stored at
/etc/logrotate.conf Default configs look like this -
# see "man logrotate" for details
# rotate log files weekly
# use the syslog group by default, since this is the owning group
# of /var/log/syslog.
su root syslog
# keep 4 weeks worth of backlogs
# create new (empty) log files after rotating old ones
# uncomment this if you want your log files compressed
# packages drop log rotation information into this directory
# no packages own wtmp, or btmp -- we'll rotate them here
create 0664 root utmp
create 0660 root utmp
# system-specific logs may be configured here
Apart from the config file, there is also a directory
/etc/logrotate.d that contains the packages you install that need help with log rotation. Eg.
dpkg etc. This directory contains config files with the name of the package i.e these package-specific configs are only applicable to a particular package.
In case you want to explore more on logrotate, check the resources section at the end of the article.
Managing Postgres logs with Logrotate
In the previous section, we learned where are the configs stored. Let’s check the logrotate configs for postgres.
The default configs would be like this -
su root root
Let me explain some of the parameters used above -
weekly- Log files are rotated every week. You can set this option to daily, monthly, yearly also.
rotate 10- This means that Log files are rotated 10 times before they are removed. In case you want to store all logs, you can also automatically configure to mail the logs which are going to be removed. In case you don’t want to store any past logs, just set the count to
compress- Older log files are first compressed and then stored. By default,
gzipis used. You can always unzip to read the logs later using
gzip -d filename.gz
notifempty- It tells logrotate to avoid rotation in case the original log file is empty. (This can happen if there are no logs being produced)
Now you understand the basic usage of logrotate. Let’s see another scenario where you want to configure logrotate.
Rotate based on the size of log files
You can use the
size option in logrotate for this.
size 10M for 100KB and 10MB respectively.
Create log files with specific permission
For this, we can specify with create option. Eg -
**create** _mode owner group_
create 0640 mohit mohit
Mail log files before removal
As mentioned above, if you want to store all logs you can simply set up a mail where these logs will be sent.
Suppose the rotate count is set as 10 then when the number of logs reaches 11, the oldest log will be mailed to the provided address.
This is how I keep my Postgres log configs -
create 0660 root root
You can also configure logs using postgres configs. Check details here (runtime logging configs).
Note : Logrotate runs based on cron jobs. If you want to do a force reload, this can be achieved manually using -
# for postgres
sudo logrotate -vf /etc/logrotate.d/postgresql-common
Awesome! You just learned how to manage logs using the
lograte command. Do try to configure your logs according to your need as an exercise!
Liked the article? Consider supporting me ☕️
I hope you learned something new. Feel free to suggest improvements ✔️
I share regular updates and resources on Twitter. Let’s connect!
Keep exploring 🔎 Keep learning 🚀
Liked the content? Do support :)