2018-01-14T01:09:24 *** matthias_bgg has quit IRC 2018-01-14T02:04:53 *** Son_Goku has joined #opensuse-admin 2018-01-14T02:17:52 *** Son_Goku has quit IRC 2018-01-14T02:54:51 *** Son_Goku has joined #opensuse-admin 2018-01-14T03:29:43 *** okurz has quit IRC 2018-01-14T03:30:38 *** okurz has joined #opensuse-admin 2018-01-14T03:52:15 *** default__ has joined #opensuse-admin 2018-01-14T03:55:43 *** cthugha has quit IRC 2018-01-14T04:27:58 PROBLEM: Hosts syslog on monitor.infra.opensuse.org - CRITICAL: Found files older than 600 minutes /var/log/opensuse/hosts//galera2.log was last modified on Fri Jan 12 15:53:16 2018 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=monitor.infra.opensuse.org&service=Hosts%20syslog 2018-01-14T06:26:02 *** Son_Goku has quit IRC 2018-01-14T07:00:44 *** Son_Goku has joined #opensuse-admin 2018-01-14T07:02:39 *** BManojlovic has joined #opensuse-admin 2018-01-14T07:23:14 *** Son_Goku has quit IRC 2018-01-14T08:46:23 are the email lists down ? 2018-01-14T11:10:25 *** fvogt has joined #opensuse-admin 2018-01-14T13:07:45 *** cboltz has joined #opensuse-admin 2018-01-14T13:07:45 *** cboltz has joined #opensuse-admin 2018-01-14T13:22:04 *** nicolasbock has joined #opensuse-admin 2018-01-14T13:57:39 *** fvogt has quit IRC 2018-01-14T14:16:31 cboltz: on monitor.i.o.o:/var/log/opensuse/hosts: 2018-01-14T14:16:33 -rw-r--r-- 1 root root 10255840 Jan 14 14:15 water.log 2018-01-14T14:16:35 -rw-r--r-- 1 root root 64429 Jan 3 18:24 wikisearch.log 2018-01-14T14:16:38 they are the same machine? 2018-01-14T14:18:54 water is the VM with the wiki search backend, so most probably yes 2018-01-14T14:20:35 okay I'll take a tarball of the old one then 2018-01-14T14:32:34 RECOVERY: memcached data stored on riesling.infra.opensuse.org - MEMCACHE OK: memcached 1.4.33 on 127.0.0.1:11211, up 8 minutes 17 seconds - bytes is 5070285 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=riesling.infra.opensuse.org&service=memcached%20data%20stored 2018-01-14T14:41:42 RECOVERY: Hosts syslog on monitor.infra.opensuse.org - OK: Tested /var/log/opensuse/hosts/ - no files older than 240 minutes found ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=monitor.infra.opensuse.org&service=Hosts%20syslog 2018-01-14T14:45:47 *** Son_Goku has joined #opensuse-admin 2018-01-14T15:56:33 cboltz: after the latest updates, some new packages need to be added to the monitoring, is there a command to generate the list of non trusted packages again? 2018-01-14T15:56:42 ideally to remove packages that are not included any more 2018-01-14T16:11:51 the easiest way is to install them - just wait until the monitoring complains ;-) (the detail page has the list of non-whitelisted packages) 2018-01-14T16:12:52 for cleaning up the whitelist, I'm afraid you'll need rpm -qa --qf '%{distribution}\t%{name}\n' | sort and manual checking 2018-01-14T16:21:13 what's the command that the monitoring is running? 2018-01-14T16:39:02 okay found it 2018-01-14T16:39:06 I think I can script it 2018-01-14T17:14:47 *** cboltz has quit IRC 2018-01-14T17:22:47 *** Son_Goku has quit IRC 2018-01-14T18:08:32 *** nicolasbock has quit IRC 2018-01-14T18:09:41 *** nicolasbock has joined #opensuse-admin 2018-01-14T18:28:14 *** nicolasbock has quit IRC 2018-01-14T19:18:49 *** cboltz has joined #opensuse-admin 2018-01-14T19:20:19 *** cboltz has quit IRC 2018-01-14T19:20:30 *** cboltz has joined #opensuse-admin 2018-01-14T19:20:31 *** cboltz has joined #opensuse-admin 2018-01-14T20:34:25 *** fvogt has joined #opensuse-admin 2018-01-14T20:40:01 PROBLEM: HAProxy on elsa.infra.opensuse.org - HAPROXY CRITICAL - Active service sarabi is DOWN on elections proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=elsa.infra.opensuse.org&service=HAProxy 2018-01-14T20:40:02 PROBLEM: HAProxy on anna.infra.opensuse.org - HAPROXY CRITICAL - Active service sarabi is DOWN on elections proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=anna.infra.opensuse.org&service=HAProxy 2018-01-14T20:40:03 PROBLEM: HAProxy on mufasa.infra.opensuse.org - HAPROXY CRITICAL - Active service narwal4 is DOWN on static proxy ! Active service riesling is DOWN on riesling proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=mufasa.infra.opensuse.org&service=HAProxy 2018-01-14T20:56:19 cboltz: !147 you missed it or no idea? 2018-01-14T20:59:46 more or less "no idea" - I don't know keepalived 2018-01-14T21:00:03 also, you probably already know my opinion about *_master and *_slave roles ;-) 2018-01-14T21:00:11 okay 2018-01-14T21:01:49 about the roles, we could get rid of them if we make sure that the shuttle net address and the external IP are properly whitelisted 2018-01-14T21:01:56 and get rid of the virtual IPs 2018-01-14T21:04:43 that, or use subrole_keepalived (similar to subrole_ntp) 2018-01-14T21:04:49 and possibly tell the haproxy to keep the connection to the initial server instead of going through them 2018-01-14T21:13:59 *** fvogt has quit IRC 2018-01-14T21:15:46 *** fvogt has joined #opensuse-admin 2018-01-14T21:51:14 for !153 - what about using FORMULAS.yaml in the saltmaster config instead of needing another check? 2018-01-14T21:53:47 it is hard, it can read only pillars 2018-01-14T21:53:47 *** fvogt has quit IRC 2018-01-14T21:55:56 *** fvogt has joined #opensuse-admin 2018-01-14T21:58:53 there might be an easy way if you accept not to use salt-formula for the formulas ;-) 2018-01-14T21:59:18 we could write a /etc/salt/master.d/formulas that only contains the formulas 2018-01-14T21:59:44 and move/change FORMULAS.yaml to be pillar 2018-01-14T21:59:56 *** fvogt has quit IRC 2018-01-14T22:00:10 after that, adjusting the get_formulas.py script should be easy 2018-01-14T22:08:10 yep that's possible 2018-01-14T22:40:07 *** BManojlovic has quit IRC 2018-01-14T22:49:11 *** BManojlovic has joined #opensuse-admin 2018-01-14T23:22:58 !154 looks technically correct, but - why? 2018-01-14T23:27:52 for speed and maybe it is a workaround for an old bug that produces warnings 2018-01-14T23:28:05 I think it's fixed on our version, but not on the sle version 2018-01-14T23:29:06 so it produces warnings for some hosts we want to get rid of / updated? 2018-01-14T23:29:09 https://github.com/saltstack/salt/issues/40912 this is the ticket 2018-01-14T23:29:26 sounds like you should at least add a TODO to remove disabling IPv6 when that's done 2018-01-14T23:30:09 I'm using this repo as base for the internal suse repo, so I'd like this to be there if there is no objection 2018-01-14T23:32:54 I'd like to have an explanation (as comment or at least in the commit message) 2018-01-14T23:33:00 besides that, no objections 2018-01-14T23:34:03 okay 2018-01-14T23:48:40 cboltz: any idea how to get rid of this warning? https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=baloo.infra.opensuse.org&service=Apache+baloo.infra.opensuse.org%3A80+Status 2018-01-14T23:50:22 did I already mention that I don't know much about monitoring? ;-) 2018-01-14T23:50:46 * cboltz tends to zypper in user for the monitoring