If you need answer to any of the questions you've come to the right place. Read on and I guarantee you'll know EVERYTHING practical there is to know about log rotation on Unix.
Ubuntu is a popular Unix operating system; so I'll be using Ubuntu in my answer. I am using Ubuntu 10.4. This article can be easily adapted to other flavors of the Linux operating system including Debian GNU/Linux, Fedora, SUSE Linux Enterprise, Gentoo.
How Log Rotation Works
Log rotation is triggered by a cron job. On Ubuntu 10.4 logrotate is one of the commands listed in /etc/cron.daily. By default logrotate uses /etc/logrotate.conf as the default configuration file. Here's the content of /etc/logrotate.conf:
# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# uncomment this if you want your log files compressed
#compress
# packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
missingok
monthly
create 0664 root utmp
rotate 1
}
/var/log/btmp {
missingok
monthly
create 0660 root utmp
rotate 1
}As you can see this configuration file includes directory /etc/logrotate.d/. Let's navigate to /etc/logrotate.d/ and you'll see a list of files like the following. $ cd /etc/logrotate.d $ ll drwxr-xr-x 2 root root 4096 2013-07-11 06:15 ./ drwxr-xr-x 95 root root 4096 2013-07-11 02:06 ../ -rw-r--r-- 1 root root 293 2012-03-05 16:44 apache2 -rw-r--r-- 1 root root 126 2010-04-19 08:56 apport -rw-r--r-- 1 root root 173 2010-04-15 04:33 apt -rw-r--r-- 1 root root 79 2010-04-09 12:17 aptitude -rw-r--r-- 1 root root 135 2010-02-19 06:07 consolekit -rw-r--r-- 1 root root 112 2010-04-15 17:23 dpkg -rw-r--r-- 1 root root 609 2011-07-27 17:58 landscape-client -rw-r--r-- 1 root root 880 2012-02-27 17:55 mysql-server -rw-r--r-- 1 root root 256 2013-07-11 06:09 nginx ...Each of these files is created by the corresponding program to help them manage their own log files. For example nginx, a popular web server, creates its log rotation configuration file, /etc/logrotate.d/nginx, which looks like the following.
/var/log/nginx/*.log {
daily
missingok
rotate 52
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
[ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid`
endscript
}If you are interested in how another program such as Apache or MySQL server handle their log rotation, open its corresponding configuration file instead (apache2 for Apache and mysql-server for MySQL server).As you can see, nginx's default log rotation setting says that each of its logs rotates every day to the maximum of 52 times, meaning you'll always keep the most recent 52 days worth of logs on your file system.
How to let the log rotation program rotate my own logs too?
Suppose I have my own logs located in /home/ubuntu/webserver-log/ and I'd like log rotation program to handle them too. I change this file's content to the following.
/var/log/nginx/*.log /home/ubuntu/webserver-log/*/*.log {
weekly
missingok
rotate 10
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
[ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid`
endscript
}Note I change the configuration to indicate the weekly rotation and the max rotation of 10 times. One thing to note is this line of command:create 640 root adm
This command will alter the permission of all the matched logs to '640 root adm'. If this is a problem you can change it. For example, my web server (user is www-data) writes to one of these logs. If permission is '640 root adm' my web server won't be able to write to it. Therefore, I changed 'create 640 root adm' to 'create 777 www-data adm'.
One day I discovered that the permission of /home/ubuntu/webserver-log/email.log is reverted to "-rw-r----- 1 root adm". And this is because /etc/logrotate.d/nginx has this line: create 640 root adm. So I needed to change it to "create 777 www-data adm" to fix my-web-server-has-no-permission-to-edit-email-log issue.
To test your changes run the following command:
cd
sudo logrotate -d -f /etc/logrotate.conf 2> logrotate.debug
The -d flag makes sure no changes will be made to the logs or to the logrotate state file. -f means force. If you want to test the changes you made in /etc/logrotate.d/nginx, replace /etc/logrotate.conf with /etc/logrotate.d/nginx in the command.
A description of what logrotate will do when it actually runs is put into a file called logrotate.debug. In this file, I search for "webserver-log" and here is the relevant text:
rotating pattern: /var/log/nginx/*.log /home/ubuntu/webserver-log/*/*.log forced from command line (10 rotations) empty log files are not rotated, old logs are removed switching euid to 0 and egid to 84 considering log /home/ubuntu/webserver-log/chtoen/access.log log needs rotating considering log /home/ubuntu/webserver-log/chtoen/chinese-to-english.log log does not need rotatingYou get an idea of what logrotate will do when it runs.
Once you've modified the configuration file, you don't need to do anything to have your changes reflected. Next time when logrotate runs you will see the results of the changes you've made. However, if you want to force logrotate to run right now according to the settings in /etc/logrotate.d/nginx, run this command:
cd
sudo logrotate -s /var/lib/logrotate/status /etc/logrotate.d/nginx -f
sudo logrotate -s /var/lib/logrotate/status /etc/logrotate.d/nginx -f
-s is the status file. -f means force.
When you are done, go to /home/ubuntu/webserver-log/ and observe the new logs as a result of running logrotate.
If you see this error:
error: skipping "/home/ubuntu/webserver-log/access.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.
Simply run "sudo chmod 755 chtoen" and try again.
Questions? Let me know!
Posts
