2018-04-11T00:59:36  *** cthugha has joined #opensuse-admin
2018-04-11T01:03:22  *** ldevulder has quit IRC
2018-04-11T01:56:15  *** dddh has quit IRC
2018-04-11T01:56:39  *** dddh has joined #opensuse-admin
2018-04-11T01:56:39  *** dddh has joined #opensuse-admin
2018-04-11T02:14:22  *** nicolasbock has quit IRC
2018-04-11T02:20:08  *** plinnell has quit IRC
2018-04-11T02:20:31  *** plinnell has joined #opensuse-admin
2018-04-11T02:20:31  *** plinnell has joined #opensuse-admin
2018-04-11T02:21:22  *** okurz has quit IRC
2018-04-11T02:21:43  *** okurz_ has joined #opensuse-admin
2018-04-11T02:21:52  *** okurz_ is now known as okurz
2018-04-11T02:22:51  *** Son_Goku has joined #opensuse-admin
2018-04-11T03:59:31  *** Son_Goku has quit IRC
2018-04-11T04:12:30  *** lcp has joined #opensuse-admin
2018-04-11T05:39:39  *** plinnell_ has joined #opensuse-admin
2018-04-11T05:40:56  *** plinnell has quit IRC
2018-04-11T06:40:39  *** matthias_bgg has joined #opensuse-admin
2018-04-11T06:47:19  *** deneb_alpha has joined #opensuse-admin
2018-04-11T06:47:19  *** deneb_alpha has joined #opensuse-admin
2018-04-11T07:01:18  *** ancorgs has joined #opensuse-admin
2018-04-11T07:28:53  *** cthugha is now known as ldevulder
2018-04-11T07:41:15  *** sven15 has joined #opensuse-admin
2018-04-11T08:19:56  <heroes-bot> PROBLEM: SSH on rpmlint.infra.opensuse.org - SSH CRITICAL - OpenSSH_7.2 (protocol 2.0) version mismatch, expected OpenSSH_6.6.1 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=rpmlint.infra.opensuse.org&service=SSH
2018-04-11T10:07:17  *** sven15 has quit IRC
2018-04-11T10:10:40  *** Son_Goku has joined #opensuse-admin
2018-04-11T10:14:33  *** cboltz has joined #opensuse-admin
2018-04-11T10:14:33  *** cboltz has joined #opensuse-admin
2018-04-11T10:20:44  *** sven15 has joined #opensuse-admin
2018-04-11T10:23:21  *** Son_Goku has quit IRC
2018-04-11T10:25:14  *** lcp has quit IRC
2018-04-11T10:37:40  *** nicolasbock has joined #opensuse-admin
2018-04-11T11:15:51  *** Son_Goku has joined #opensuse-admin
2018-04-11T11:21:34  *** deneb_alpha has quit IRC
2018-04-11T11:52:21  *** asmorodskyi has quit IRC
2018-04-11T11:56:19  *** asmorodskyi has joined #opensuse-admin
2018-04-11T12:02:43  *** cboltz has quit IRC
2018-04-11T12:39:34  *** deneb_alpha has joined #opensuse-admin
2018-04-11T12:39:34  *** deneb_alpha has joined #opensuse-admin
2018-04-11T13:29:30  *** ancor has joined #opensuse-admin
2018-04-11T13:30:22  *** ancorgs has quit IRC
2018-04-11T13:34:10  *** ancor is now known as ancorgs
2018-04-11T14:05:56  *** lcp has joined #opensuse-admin
2018-04-11T14:30:55  *** Son_Goku has quit IRC
2018-04-11T15:03:39  *** Son_Goku has joined #opensuse-admin
2018-04-11T15:49:02  *** Son_Goku has quit IRC
2018-04-11T15:59:00  *** Son_Goku has joined #opensuse-admin
2018-04-11T16:03:50  *** sven15 has quit IRC
2018-04-11T16:08:44  *** Son_Goku has quit IRC
2018-04-11T16:17:51  *** deneb_alpha has quit IRC
2018-04-11T16:19:18  *** ancorgs has quit IRC
2018-04-11T16:38:03  *** plinnell_ has quit IRC
2018-04-11T16:38:29  *** plinnell_ has joined #opensuse-admin
2018-04-11T16:45:21  <heroes-bot> PROBLEM: NRPE on sarabi.infra.opensuse.org - connect to address 192.168.47.15 port 5666: Connection refused ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=sarabi.infra.opensuse.org&service=NRPE
2018-04-11T16:47:07  *** Son_Goku has joined #opensuse-admin
2018-04-11T17:05:00  *** fvogt has joined #opensuse-admin
2018-04-11T17:38:16  *** cboltz has joined #opensuse-admin
2018-04-11T17:38:16  *** cboltz has joined #opensuse-admin
2018-04-11T17:55:21  *** Son_Goku has quit IRC
2018-04-11T18:03:12  *** plinnell_ has quit IRC
2018-04-11T18:12:42  *** plinnell has joined #opensuse-admin
2018-04-11T18:12:43  *** plinnell has joined #opensuse-admin
2018-04-11T18:15:21  *** matthias_bgg has quit IRC
2018-04-11T18:32:42  *** lcp_ has joined #opensuse-admin
2018-04-11T18:34:26  *** lcp has quit IRC
2018-04-11T18:34:26  *** lcp_ is now known as lcp
2018-04-11T18:49:57  *** Son_Goku has joined #opensuse-admin
2018-04-11T19:06:37  <pjessen> anyone here
2018-04-11T19:06:54  <pjessen> relay2 is rejcted mails
2018-04-11T19:06:57  <pjessen> NOQUEUE: reject: MAIL from unknown[149.44.160.134]: 452 4.3.1 Insufficient system storage; proto=ESMTP helo=<relay2.suse.de>
2018-04-11T19:07:47  <cboltz> nice[tm]
2018-04-11T19:08:09  <pjessen> warning: not enough free space in mail queue: 3518464 bytes < 1.5*message size limit
2018-04-11T19:08:32  <cboltz> thomic (or whoever has access to relay2.suse.de) - can you please check/fix that?
2018-04-11T19:08:50  <pjessen> would be nice[tm]
2018-04-11T19:09:51  <pjessen> found another appamor issue :-)
2018-04-11T19:10:00  <pjessen> Could not mkdir(/var/spool/mlmmj/opensuse-bugs//requeue/783041) for requeueing. Mail cannot be requeued.: Permission denied
2018-04-11T19:10:19  <cboltz> what's the audit.log line?
2018-04-11T19:10:44  <pjessen> just checking. might not be apparmor
2018-04-11T19:10:47  *** Son_Goku has quit IRC
2018-04-11T19:12:06  <pjessen> nope, isn't AA
2018-04-11T19:12:49  *** Son_Goku has joined #opensuse-admin
2018-04-11T19:13:01  <cboltz> :-)
2018-04-11T19:13:03  <pjessen> never mind, only one occurence
2018-04-11T19:14:00  <cboltz> just wondering - is the double slash (...bugs//requeue...) somewhat correct, or is a directory name missing in between?
2018-04-11T19:14:49  <pjessen> christian, would you update status.o.o - all mailing lists are down since 15:35 UTC.
2018-04-11T19:15:12  <cboltz> are you afraid of doing it yourself? ;-)
2018-04-11T19:15:12  <pjessen> it looks like mails are queued on relay[12].suse.de
2018-04-11T19:15:18  <cboltz> status.o.o doesn't bite...
2018-04-11T19:15:31  <pjessen> I'm too busy with other stuff, and I haven't done it before :-)
2018-04-11T19:15:51  <cboltz> it's not hard, but "too busy" is a valid argument
2018-04-11T19:15:56  <pjessen> bitte ....
2018-04-11T19:15:56  <cboltz> I'll do it ;-)
2018-04-11T19:21:31  <pjessen> the requeue is the aa profile for mlmmj-send. I'll send you a patch tomorrow.
2018-04-11T19:22:04  <cboltz> :-)
2018-04-11T19:22:11  <cboltz> status.opensuse.org updated
2018-04-11T19:22:28  <cboltz> BTW: I mentioned DKIM, DMARC etc. at the heroes meeting - see https://listi.jpberlin.de/pipermail/postfixbuch-users/2018-February/065826.html (whenever you have time to read it)
2018-04-11T19:22:39  <pjessen> just saw it thanks.
2018-04-11T19:23:57  *** Son_Goku has quit IRC
2018-04-11T19:32:04  *** fvogt has quit IRC
2018-04-11T19:32:38  *** fvogt has joined #opensuse-admin
2018-04-11T19:37:50  <pjessen> uh, my mistake. It is baloo that is rejecting mails from relay[12].suse.de. I'll have a look.
2018-04-11T19:42:35  <pjessen> looks like logrotate isn't wokring on baloo
2018-04-11T19:43:20  <cboltz> looks like we should have done a "how to configure logrotate" at the heroes meeting
2018-04-11T19:43:31  <cboltz> this isn't the first machine with missing logrotate config ;-)
2018-04-11T19:44:41  <cboltz> oh, mail.info has "just" 4.3 GB? nice ;-)
2018-04-11T19:46:42  <cboltz> interestingly, logrotate.timer is enabled, and    systemctl status logrotate.service    says "Started Rotate log files..."
2018-04-11T19:50:01  <pjessen> yeah
2018-04-11T19:50:17  <pjessen> looks like mail.info and mail.warn were not \rotated.
2018-04-11T19:50:54  <pjessen> most likely due to my never using those files, so I dropped them from logrotate ..... arggg
2018-04-11T19:51:44  <cboltz>  /etc/logrotate.d/syslog includes mail.info and mail.warn, so in theory they should be rotated
2018-04-11T19:51:55  <pjessen> hehe. just added them :-)
2018-04-11T19:52:18  <cboltz> ah, that explains it ;-)
2018-04-11T19:53:35  <pjessen> I'll amend the rsyslog config, nobody needs those files
2018-04-11T19:54:19  <pjessen> I'm compressing one mail.info, but the lists look like they are running again.
2018-04-11T19:56:02  <cboltz> FYI: the rsyslog config is managed with salt, so if you don't want your changes overwritten, you should do them in salt ;-)
2018-04-11T19:56:19  <cboltz> rsyslog.conf is in salt/profile/log/files/etc/rsyslog.conf
2018-04-11T20:00:04  <heroes-bot> PROBLEM: HAProxy on elsa.infra.opensuse.org - HAPROXY CRITICAL - Active service dale is DOWN on dale proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=elsa.infra.opensuse.org&service=HAProxy
2018-04-11T20:00:05  <heroes-bot> PROBLEM: HAProxy on anna.infra.opensuse.org - HAPROXY CRITICAL - Active service dale is DOWN on dale proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=anna.infra.opensuse.org&service=HAProxy
2018-04-11T20:00:26  <pjessen> also for baloo?
2018-04-11T20:01:46  <cboltz> check   head -n1 /etc/rsyslog.conf   ;-)
2018-04-11T20:01:57  <pjessen> duh
2018-04-11T20:02:29  <pjessen> but /etc/logrotate.conf is not?
2018-04-11T20:02:53  <cboltz> no
2018-04-11T20:03:25  <pjessen> okay.  I have deleted the large mail.info, we have 48% free space, problem solved.
2018-04-11T20:04:06  <cboltz> indeed :-)
2018-04-11T20:04:39  <pjessen> gotta go, I'm really late for picking up my son from the sbahn .... can you update status.o.o for me again?
2018-04-11T20:04:50  <cboltz> yes, of course
2018-04-11T20:07:55  <pjessen> hey, there is one more screw-up for your presentation ....
2018-04-11T20:08:08  <pjessen> ciao
2018-04-11T20:08:18  <cboltz> ;-)
2018-04-11T20:08:51  <cboltz> I have a feeling that I could do a whole presentation about "how not rotating your logs can break all your systems" ;-)
2018-04-11T20:21:20  *** awafaa_ has joined #opensuse-admin
2018-04-11T20:23:50  *** DimStar|1fk has joined #opensuse-admin
2018-04-11T20:25:54  *** lnussel_ has joined #opensuse-admin
2018-04-11T20:28:37  *** lnussel has quit IRC
2018-04-11T20:28:38  *** awafaa has quit IRC
2018-04-11T20:28:39  *** DimStar|afk has quit IRC
2018-04-11T20:28:39  *** bmwiedemann has quit IRC
2018-04-11T20:28:44  *** awafaa_ is now known as awafaa
2018-04-11T20:30:37  *** bmwiedemann has joined #opensuse-admin
2018-04-11T20:35:33  *** fvogt has quit IRC
2018-04-11T20:40:06  <heroes-bot> PROBLEM: HAProxy on mufasa.infra.opensuse.org - HAPROXY CRITICAL - Active service riesling is DOWN on riesling proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=mufasa.infra.opensuse.org&service=HAProxy
2018-04-11T21:18:42  *** matthias_bgg has joined #opensuse-admin
2018-04-11T21:59:27  *** Fraser_Bell has joined #opensuse-admin
2018-04-11T21:59:28  *** Fraser_Bell has joined #opensuse-admin
2018-04-11T22:10:34  *** Fraser_Bell has quit IRC
2018-04-11T22:23:45  *** matthias_bgg has quit IRC
2018-04-11T23:34:41  *** cboltz has quit IRC