rsyslog not rotating logs

Post Reply
jiffysound
Posts: 66
Joined: Wed Sep 07, 2016 9:49 pm
languages_spoken: english
ODROIDs: odroid c2
Contact:

rsyslog not rotating logs

Unread post by jiffysound » Wed Dec 12, 2018 2:08 am

I have tried to remove --purge rsyslog and reinstall hoping it would fix the issue but alas here we are with no log rotation. Here are some configs I am sure you guys will ask of me. First off here is the rsyslog status, as you can see it's *active* but with some sort of error.

Code: Select all

rsyslog.service - System Logging Service
   Loaded: loaded (/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2016-02-11 17:28:02 CET; 2 years 9 months ago
     Docs: man:rsyslogd(8)
           http://www.rsyslog.com/doc/
 Main PID: 553 (rsyslogd)
   CGroup: /system.slice/rsyslog.service
           └─553 /usr/sbin/rsyslogd -n -i/var/run/rsyslogd.pid

Feb 11 17:28:02 odroid64 rsyslogd[553]: action 'action-2-builtin:omfile' resumed (module 'builtin:omfile') [v8.39.0 try http://www.rsyslog.com/e/
Feb 11 17:28:02 odroid64 rsyslogd[553]: action 'action-2-builtin:omfile' suspended (module 'builtin:omfile'), retry 0. There should be messages b
Feb 11 17:28:02 odroid64 rsyslogd[553]: action 'action-2-builtin:omfile' resumed (module 'builtin:omfile') [v8.39.0 try http://www.rsyslog.com/e/
Feb 11 17:28:02 odroid64 rsyslogd[553]: action 'action-2-builtin:omfile' suspended (module 'builtin:omfile'), retry 0. There should be messages b
Feb 11 17:28:02 odroid64 rsyslogd[553]: action 'action-2-builtin:omfile' resumed (module 'builtin:omfile') [v8.39.0 try http://www.rsyslog.com/e/
Feb 11 17:28:02 odroid64 rsyslogd[553]: action 'action-2-builtin:omfile' suspended (module 'builtin:omfile'), retry 0. There should be messages b
Feb 11 17:28:02 odroid64 rsyslogd[553]: action 'action-2-builtin:omfile' resumed (module 'builtin:omfile') [v8.39.0 try http://www.rsyslog.com/e/
Feb 11 17:28:02 odroid64 rsyslogd[553]: action 'action-2-builtin:omfile' suspended (module 'builtin:omfile'), retry 0. There should be messages b
Feb 11 17:28:02 odroid64 rsyslogd[553]: action 'action-2-builtin:omfile' suspended (module 'builtin:omfile'), next retry is Thu Feb 11 17:28:32 2
Feb 11 17:28:02 odroid64 systemd[1]: Started System Logging Service.
/etc/rsyslog.conf

Code: Select all

root@odroid64:~# cat /etc/rsyslog.conf
#  /etc/rsyslog.conf	Configuration file for rsyslog.
#
#			For more information see
#			/usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
#
#  Default logging rules can be found in /etc/rsyslog.d/50-default.conf


#################
#### MODULES ####
#################

$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog   # provides kernel logging support (previously done by rklogd)
#$ModLoad immark  # provides --MARK-- message capability

# provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

# provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514


###########################
#### GLOBAL DIRECTIVES ####
###########################

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Filter duplicated messages
$RepeatedMsgReduction on

#
# Set the default permissions for all log files.
#
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog

#
# Where to place spool files
#
$WorkDirectory /var/spool/rsyslog

#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf

root@odroid64
/etc/rsyslog.d/50-default.conf

Code: Select all

root@odroid64:/etc/rsyslog.d# cat 50-default.conf
#  Default rules for rsyslog.
#
#			For more information see rsyslog.conf(5) and /etc/rsyslog.conf

#
# First some standard log files.  Log by facility.
#
auth,authpriv.*			/var/log/auth.log
*.*;auth,authpriv.none		/var/log/syslog
syslog.*                        /var/log/rsyslog.log #rsyslog error messages
#cron.*				/var/log/cron.log
#daemon.*			/var/log/daemon.log
kern.*				/var/log/kern.log
#lpr.*				/var/log/lpr.log
mail.*				/var/log/mail.log
#user.*				/var/log/user.log

#
# Logging for the mail system.  Split it up so that
# it is easy to write scripts to parse these files.
#
#mail.info			/var/log/mail.info
#mail.warn			/var/log/mail.warn
mail.err			/var/log/mail.err

#
# Logging for INN news system.
#
news.crit			/var/log/news/news.crit
news.err			/var/log/news/news.err
news.notice			/var/log/news/news.notice

#
# Some "catch-all" log files.
#
#*.=debug;\
#	auth,authpriv.none;\
#	news.none;mail.none	/var/log/debug
#*.=info;*.=notice;*.=warn;\
#	auth,authpriv.none;\
#	cron,daemon.none;\
#	mail,news.none		/var/log/messages

#
# Emergencies are sent to everybody logged in.
#
*.emerg                                :omusrmsg:*

#
# I like to have messages displayed on the console, but only on a virtual
# console I usually leave idle.
#
#daemon,mail.*;\
#	news.=crit;news.=err;news.=notice;\
#	*.=debug;*.=info;\
#	*.=notice;*.=warn	/dev/tty8

# The named pipe /dev/xconsole is for the `xconsole' utility.  To use it,
# you must invoke `xconsole' with the `-file' option:
#
#    $ xconsole -file /dev/xconsole [...]
#
# NOTE: adjust the list below, or you'll go crazy if you have a reasonably
#      busy site..
#
#
# As this functionality is almost never needed, it is commented out. If you
# need it, be sure to remove the comment characters below.
#daemon.*;mail.*;\
#	news.err;\
#	*.=debug;*.=info;\
#	*.=notice;*.=warn	|/dev/xconsole
root@odroid64:/etc/rsyslog.d#
/etc/logrotate.d/rsyslog

Code: Select all

root@odroid64:~# cat /etc/logrotate.d/rsyslog
/var/log/syslog
{
	rotate 7
	daily
	missingok
	notifempty
	delaycompress
	compress
	postrotate
		invoke-rc.d rsyslog rotate >/dev/null
	endscript
}

/var/log/mail.info
/var/log/mail.warn
/var/log/mail.err
/var/log/mail.log
/var/log/daemon.log
/var/log/kern.log
/var/log/auth.log
/var/log/user.log
/var/log/lpr.log
/var/log/cron.log
/var/log/rsyslog.log
/var/log/debug
/var/log/messages
{
	rotate 4
	weekly
	missingok
	notifempty
	compress
	delaycompress
	sharedscripts
	postrotate
		invoke-rc.d rsyslog rotate >/dev/null
	endscript
}
root@odroid64:~#
My /var/log looks like this, everything in each log, gets logged in the primary, not the log.1. I know this little guy is getting old but I don't want to give up on him just yet.

Code: Select all

root@odroid64:/var/log# ls
alternatives.log    auth.log    dist-upgrade    journal     lightdm   unattended-upgrades  Xorg.0.log.old
alternatives.log.1  auth.log.1  dpkg.log        kern.log    ntpstats  wtmp
apcupsd.events      btmp        dpkg.log.1      kern.log.1  syslog    wtmp.1
apt                 btmp.1      fontconfig.log  lastlog     syslog.1  Xorg.0.log
Upon further investigation, it seems there was an error during the last install purge/install process, here is the log, notice that I reinstalled it today using synaptic instead of terminal and no errors.

Code: Select all

Start-Date: 2018-12-10  01:09:58
Commandline: apt-get remove rsyslog --purge
Requested-By: odroid (1000)
Purge: rsyslog:arm64 (8.39.0-0adiscon5xenial1)
End-Date: 2018-12-10  01:10:02

Start-Date: 2018-12-10  01:10:47
Commandline: apt-get install rsyslog
Requested-By: odroid (1000)
Install: rsyslog:arm64 (8.39.0-0adiscon5xenial1)
Error: Sub-process /usr/bin/dpkg returned an error code (1)
End-Date: 2018-12-10  01:10:55

Start-Date: 2018-12-11  19:10:32
Commandline: /usr/sbin/synaptic
Requested-By: odroid (1000)
Purge: rsyslog:arm64 (8.39.0-0adiscon5xenial1)
End-Date: 2018-12-11  19:10:37

Start-Date: 2018-12-11  19:11:12
Commandline: /usr/sbin/synaptic
Requested-By: odroid (1000)
Install: rsyslog:arm64 (8.39.0-0adiscon5xenial1)
End-Date: 2018-12-11  19:11:21
root@odroid64:/var/log/apt#

User avatar
mad_ady
Posts: 5457
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1, H2, N2
Location: Bucharest, Romania
Contact:

Re: rsyslog not rotating logs

Unread post by mad_ady » Wed Dec 12, 2018 3:15 am

It's not rsyslog which rotates its log, but logrotate.
You can run logrotate verbosely (but I don't know the exact switch, possibly -v) to see what's going on.
logrotate is typically called by cron, so check if cron is running too.

jiffysound
Posts: 66
Joined: Wed Sep 07, 2016 9:49 pm
languages_spoken: english
ODROIDs: odroid c2
Contact:

Re: rsyslog not rotating logs

Unread post by jiffysound » Wed Dec 12, 2018 8:20 am

Well I solved one part of the puzzle, the strange error messages when I do service rsyslog status

Code: Select all

rsyslog.service - System Logging Service
   Loaded: loaded (/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2016-02-11 17:28:02 CET; 2 years 9 months ago
     Docs: man:rsyslogd(8)
           http://www.rsyslog.com/doc/
 Main PID: 553 (rsyslogd)
   CGroup: /system.slice/rsyslog.service
           └─553 /usr/sbin/rsyslogd -n -i/var/run/rsyslogd.pid

Feb 11 17:28:02 odroid64 rsyslogd[553]: action 'action-2-builtin:omfile' resumed (module 'builtin:omfile') [v8.39.0 try http://www.rsyslog.com/e/
Feb 11 17:28:02 odroid64 rsyslogd[553]: action 'action-2-builtin:omfile' suspended (module 'builtin:omfile'), retry 0. There should be messages b
Turns out all I had to do was edit /etc/rsyslog.conf like so

Code: Select all

#
# Set the default permissions for all log files.
#
#$FileOwner syslog
#$FileGroup adm
#$FileCreateMode 0640
#$DirCreateMode 0755
#$Umask 0022
#$PrivDropToUser syslog
#$PrivDropToGroup syslog


Now the status looks good

Code: Select all

root@odroid64:~# service rsyslog status
● rsyslog.service - System Logging Service
   Loaded: loaded (/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2016-02-11 17:28:03 CET; 2 years 9 months ago
     Docs: man:rsyslogd(8)
           http://www.rsyslog.com/doc/
 Main PID: 617 (rsyslogd)
   CGroup: /system.slice/rsyslog.service
           └─617 /usr/sbin/rsyslogd -n -i/var/run/rsyslogd.pid

Feb 11 17:28:03 odroid64 systemd[1]: Starting System Logging Service...
Feb 11 17:28:03 odroid64 rsyslogd[617]: imuxsock: Acquired UNIX socket '/run/systemd/journal/syslog' (fd 3) from systemd.  [v8.39.0
Feb 11 17:28:03 odroid64 rsyslogd[617]:  [origin software="rsyslogd" swVersion="8.39.0" x-pid="617" x-info="http://www.rsyslog.com"
Feb 11 17:28:03 odroid64 systemd[1]: Started System Logging Service.
I added this bit in case anyone else had this issue. I found the answer here https://askubuntu.com/questions/1066997 ... med-module

User avatar
mad_ady
Posts: 5457
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, N1, H2, N2
Location: Bucharest, Romania
Contact:

Re: rsyslog not rotating logs

Unread post by mad_ady » Wed Dec 12, 2018 3:04 pm

What about logrotate? What does it have to say?

User avatar
beroot
Posts: 16
Joined: Sun Jan 07, 2018 12:38 am
languages_spoken: english, german
ODROIDs: C2
Contact:

Re: rsyslog not rotating logs

Unread post by beroot » Tue Dec 18, 2018 3:22 am

Hello,

at your statements I'm missing a (max)size and minsize.
My answer just give you an example. 4M is not that large or that small to find issues. The expected rotation will be compared with all values before the rotation cycle rotates the logfile. Just some, as well as the Apache, syslog and other logfiles can grow very fast. That depends at the services you offer or use. Logfiles could explode your available diskspace.

- make sure that all your config files have an empty last line
- please install gzip and gunzip before
- If "man logrotate" or "man rsyslog" is not installed at your system please check an internet database like "duckduckgo.com" for "man rsyslog" && || "man logrotate" for more detailed information
- keep the logfiles small.

size 4M
minsize 4M

Please remove the leading "#" from your conf file:
#$FileOwner syslog
#$FileGroup adm
#$FileCreateMode 0640
#$DirCreateMode 0755
#$Umask 0022
#$PrivDropToUser syslog
#$PrivDropToGroup syslog
The cronjob needs to know the owner, the group, and all other values to create and/or secure logfiles against improper access...


An example of
cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
extension gz
minsize 4M
size 4M
notifempty
compress
weekly

# use the syslog group by default, since this is the owning group
# of /var/log/syslog.
su root syslog

# keep 4 weeks worth of backlogs
rotate 8

# create new (empty) log files after rotating old ones
create 644 root

# uncomment this if you want your log files compressed
#compress

# packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp, or btmp -- we'll rotate them here
/var/log/wtmp {
missingok
monthly
create 0664 root utmp
rotate 1
}

/var/log/btmp {
missingok
monthly
create 0660 root utmp
rotate 1
}

# system-specific logs may be configured here
The Extension "gz" specifies the compression method.


At each of the logfiles found in "/etc/logrotate.d/", I changed some values or enhanced with new entries

"minsize ? M (size in 4, 8, 12, 16, or any Number you want in (M)egabytes)
"weekly" (monthly - only one of those statements)
"rotate ?" (how many logfiles will be kept 4, 8, 12, 16, or any Number you want)
"notifempty" (selfexplaining)
"missingok" (selfexplaining)


2 Examples of my logrotate files found in:

cat /etc/logrotate.d/apache2
Please ignore the statements starting and ending by IF / FI;
Some default values are given and should only be changed if you really know what you're doing (read man) ;)
/var/log/apache2/*.log {
monthly
minsize 4M
missingok
rotate 16
compress
delaycompress
notifempty
create 644 root adm
sharedscripts
postrotate
if invoke-rc.d apache2 status > /dev/null 2>&1; then \
invoke-rc.d apache2 reload > /dev/null 2>&1; \
fi;
endscript
prerotate
if [ -d /etc/logrotate.d/httpd-prerotate ]; then \
run-parts /etc/logrotate.d/httpd-prerotate; \
fi; \
endscript
}

cat /etc/logrotate.d/alternatives
/var/log/alternatives.log {
monthly
minsize 4M
rotate 16
compress
delaycompress
missingok
notifempty
create 644 root root
}

At least my complete

cat /etc/rsyslog.conf
# /etc/rsyslog.conf Configuration file for rsyslog.
#
# For more information see
# /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
#
# Default logging rules can be found in /etc/rsyslog.d/50-default.conf


#################
#### MODULES ####
#################

module(load="imuxsock") # provides support for local system logging
#module(load="immark") # provides --MARK-- message capability

# provides UDP syslog reception
#module(load="imudp")
#input(type="imudp" port="514")

# provides TCP syslog reception
#module(load="imtcp")
#input(type="imtcp" port="514")

# provides kernel logging support and enable non-kernel klog messages
module(load="imklog" permitnonkernelfacility="on")

###########################
#### GLOBAL DIRECTIVES ####
###########################

#
# Use traditional timestamp format.
# To enable high precision timestamps, comment out the following line.
#
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Filter duplicated messages
$RepeatedMsgReduction on

#
# Set the default permissions for all log files.
#
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog

#
# Where to place spool and state files
#
$WorkDirectory /var/spool/rsyslog

#
# Include all config files in /etc/rsyslog.d/
#
$IncludeConfig /etc/rsyslog.d/*.conf

Enable your changes with the command:
service rsyslog restart

Remember that the "/etc/logrotate.conf" is the default file which will be overwritten by the specified files and values in "/etc/logrotate.d/*"

Also remeber that the logrotation process been started by cron daylie will check the size of the files and other values found in "/etc/logrotate.d/*"

Watch out your spelling. Wrong charakters or complete false spelled words like "1 M" will not be tolerated.

At my system is running "mariadb" but I don't use the rsyslog extension with the database.

COMPLETION 2

ls -ali /var/log
524290 drwxrwxr-x 10 root syslog 4096 Dez 5 14:30 .
2 drwxr-xr-x 13 root root 4096 Nov 4 05:25 ..
526038 -rw-r--r-- 1 root root 4464 Dez 2 12:38 alternatives.log
526105 -rw-r--r-- 1 root root 2004 Nov 3 13:46 alternatives.log.1
526268 drwxr-xr-x 2 root adm 4096 Dez 3 00:00 apache2
524478 drwxr-xr-x 2 root root 4096 Dez 17 16:13 apt
527237 -rw-r----- 1 syslog adm 805267 Dez 17 19:39 auth.log
526106 -rw-r----- 1 syslog adm 26874 Nov 4 06:25 auth.log.1
526107 -rw-r--r-- 1 root root 0 Nov 3 08:12 bootstrap.log
527240 -rw-rw---- 1 root utmp 0 Nov 4 06:25 btmp
526108 -rw-rw---- 1 root utmp 1200 Nov 3 10:33 btmp.1
527943 drwxr-xr-x 2 root root 4096 Dez 5 13:10 corosync
527279 -rw-r----- 1 syslog adm 13510 Dez 9 09:13 debug
526117 -rw-r--r-- 1 root root 240739 Dez 17 16:15 dpkg.log
526109 -rw-r--r-- 1 root root 102483 Nov 4 06:01 dpkg.log.1
526110 -rw-r--r-- 1 root root 3584 Dez 5 13:10 faillog
527614 -rw-r--r-- 1 hacluster haclient 12549 Dez 5 14:30 ha-log
527747 -rw-r--r-- 1 hacluster haclient 12296 Dez 9 09:14 ha.log
524480 drwxr-sr-x+ 3 root systemd-journal 4096 Jan 28 2018 journal
527233 -rw-r----- 1 syslog adm 927717 Dez 9 09:09 kern.log
526111 -rw-r----- 1 syslog adm 217560 Nov 3 16:07 kern.log.1
526112 -rw-rw-r-- 1 root utmp 296296 Dez 17 18:45 lastlog
526149 -rw-r----- 1 syslog adm 470 Nov 23 10:16 mail.err
527266 -rw-r----- 1 syslog adm 321746 Dez 17 16:19 mail.info
527232 -rw-r----- 1 syslog adm 323033 Dez 17 16:19 mail.log
526495 -rw-r----- 1 syslog adm 2723 Nov 4 06:17 mail.log.1
527287 -rw-r----- 1 syslog adm 6040 Nov 23 10:16 mail.warn
526485 -rw-r----- 1 syslog adm 3918941 Dez 17 19:29 messages
526707 drwxr-s--x 2 mysql adm 4096 Nov 3 13:46 mysql
527251 -rw-r--r-- 1 root root 4394 Dez 17 06:59 noip.log
526171 drwxr-xr-x 2 ntp ntp 4096 Jul 6 21:08 ntpstats
528208 -rw-rw---- 1 hacluster haclient 2865818 Dez 17 19:57 pacemaker.log
526834 -rw------- 1 root root 8343 Dez 9 09:09 php7.2-fpm.log
527627 drwxr-x--- 2 root adm 4096 Nov 16 14:19 samba
526113 -rw-r----- 1 syslog adm 2798895 Dez 17 19:57 syslog
527230 -rw-r----- 1 syslog adm 4698902 Dez 2 00:00 syslog.1
524340 -rw-r----- 1 syslog adm 204355 Nov 4 06:25 syslog.2.gz
526114 -rw------- 1 root root 7168 Dez 5 13:10 tallylog
526686 -rw-r----- 1 syslog adm 655 Nov 3 15:17 ufw.log
527409 drwxr-x--- 2 root adm 4096 Nov 5 06:28 unattended-upgrades
527239 -rw-rw-r-- 1 root utmp 199200 Dez 17 18:45 wtmp
526115 -rw-rw-r-- 1 root utmp 70400 Nov 4 05:54 wtmp.1
Files " *.1" have been rotated but not compressed.
Files " *.2.gz" is the last rotated file compressed by gzip and the values given by the conf file.



Have fun

Post Reply

Return to “Ubuntu”

Who is online

Users browsing this forum: No registered users and 2 guests