2018-07-26T01:00:05 *** teclator has quit IRC 2018-07-26T01:01:21 *** teclator has joined #opensuse-admin 2018-07-26T01:26:18 *** plinnell has quit IRC 2018-07-26T01:26:33 *** plinnell has joined #opensuse-admin 2018-07-26T02:23:33 *** lcp has joined #opensuse-admin 2018-07-26T02:42:38 *** okurz has quit IRC 2018-07-26T02:43:30 *** okurz has joined #opensuse-admin 2018-07-26T03:30:19 *** Son_Goku has joined #opensuse-admin 2018-07-26T04:23:02 *** Fraser_Bell has joined #opensuse-admin 2018-07-26T04:23:02 *** Fraser_Bell has joined #opensuse-admin 2018-07-26T04:33:09 *** Son_Goku has quit IRC 2018-07-26T06:16:34 *** Fraser_Bell has quit IRC 2018-07-26T06:37:56 *** duncanmv has joined #opensuse-admin 2018-07-26T07:31:44 *** matthias_bgg has joined #opensuse-admin 2018-07-26T07:40:33 *** lcp has quit IRC 2018-07-26T07:40:52 *** lcp has joined #opensuse-admin 2018-07-26T07:45:40 *** lcp has quit IRC 2018-07-26T07:46:08 *** lcp has joined #opensuse-admin 2018-07-26T07:48:53 *** lcp has quit IRC 2018-07-26T07:51:40 *** ldevulder_ is now known as ldevulder 2018-07-26T07:55:46 *** lcp has joined #opensuse-admin 2018-07-26T07:56:25 *** lnussel has quit IRC 2018-07-26T07:57:02 *** duncanmv has quit IRC 2018-07-26T08:02:48 *** dakechi has quit IRC 2018-07-26T08:10:23 *** duncanmv has joined #opensuse-admin 2018-07-26T08:13:56 *** enavarro has joined #opensuse-admin 2018-07-26T08:17:34 *** dakechi has joined #opensuse-admin 2018-07-26T08:22:43 *** marxin has quit IRC 2018-07-26T08:22:56 *** marxin has joined #opensuse-admin 2018-07-26T08:28:35 *** marxin has quit IRC 2018-07-26T08:35:36 *** marxin has joined #opensuse-admin 2018-07-26T08:41:33 *** dakechi has quit IRC 2018-07-26T08:53:04 *** dakechi has joined #opensuse-admin 2018-07-26T09:08:54 *** megamaced has quit IRC 2018-07-26T09:09:16 *** megamaced has joined #opensuse-admin 2018-07-26T09:56:36 PROBLEM: PerfData on monitor.infra.opensuse.org - CRITICAL: Found files older than 2880 minutes /var/lib/pnp4nagios/perfdata//proc_Rsyslog_Process.xml was last modified on Thu Jun 21 09:42:54 2018 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=monitor.infra.opensuse.org&service=PerfData 2018-07-26T10:12:13 *** Son_Goku has joined #opensuse-admin 2018-07-26T10:16:52 *** cboltz has joined #opensuse-admin 2018-07-26T10:16:53 *** cboltz has joined #opensuse-admin 2018-07-26T10:25:44 *** Son_Goku has quit IRC 2018-07-26T10:30:36 *** dakechi has quit IRC 2018-07-26T10:44:35 *** dakechi has joined #opensuse-admin 2018-07-26T11:15:49 *** Son_Goku has joined #opensuse-admin 2018-07-26T13:15:25 *** ldevulder has quit IRC 2018-07-26T13:28:32 *** Son_Goku has quit IRC 2018-07-26T13:40:56 *** ldevulder has joined #opensuse-admin 2018-07-26T14:11:14 tampakrap: Hi - can you please help me find openqaworker1 again? The machine is not accessible by ssh anymore 2018-07-26T14:13:55 DimStar: hello, I don't have access to the openqa infrastructure. Max should be able to help you though 2018-07-26T14:14:05 nickname mmaher on the internal IRC 2018-07-26T14:14:18 thanks.. will reach out to him 2018-07-26T14:16:25 tampakrap: any idea where Jared hides? ;-) 2018-07-26T14:16:58 cboltz: who is jared? 2018-07-26T14:17:31 cboltz: can you try to log in to freeipa? 2018-07-26T14:17:59 the Provo admin who can fix the news.o.o RSS feed 2018-07-26T14:18:35 no idea 2018-07-26T14:18:45 vacation maybe? 2018-07-26T14:19:20 might be, but without any autoresponder, we can only guess 2018-07-26T14:19:49 from my first mail, it would be ~3 weeks of vacation 2018-07-26T14:23:35 I'm unable to log in to freeipa, and I have no idea what's the issue 2018-07-26T14:25:37 and darix is on vacation to help me debug 2018-07-26T14:25:39 damn :( 2018-07-26T14:30:16 I fixed freeipa and I have no idea why it broke in the first place :( 2018-07-26T14:30:37 cboltz: why don't you move news.o.o to nue? 2018-07-26T14:32:40 cboltz: different question, mediawiki-test is still needed? 2018-07-26T14:42:34 *** dakechi has quit IRC 2018-07-26T14:48:33 news.o.o - moving it is the long term plan, but I don't want to have the RSS broken until then 2018-07-26T14:48:43 for mediawiki-test - what exactly do you mean? 2018-07-26T14:49:51 (if you mean en-test.o.o - yes, that's still needed) 2018-07-26T14:53:15 192.168.47.55 2018-07-26T14:55:02 *** dakechi has joined #opensuse-admin 2018-07-26T15:04:16 sorry, I meant .54 2018-07-26T15:20:32 good question, that IP doesn't ring a bell for me 2018-07-26T15:21:10 I can only guess that it _might be_ the first VM I used for playing with salt and the wiki - if so, it's no longer needed 2018-07-26T15:21:36 (actually I'm not sure if I ever want to see my first steps in salt again - if I'd review that today, I'd probably be scared ;-) 2018-07-26T15:21:48 okay review please and let me know 2018-07-26T15:22:31 it seems .54 is down (ping says "Destination Host Unreachable") 2018-07-26T15:22:35 which makes a review hard ;-) 2018-07-26T15:29:42 *** Son_Goku has joined #opensuse-admin 2018-07-26T15:32:55 cboltz: now it should be up 2018-07-26T15:40:27 ping 192.168.47.54 says From 192.168.253.1 icmp_seq=2 Destination Host Unreachable :-( 2018-07-26T15:40:59 and what is the one I brought up then? 2018-07-26T15:41:14 mystery solved 2018-07-26T15:41:22 it is even before we split the networks 2018-07-26T15:41:31 running 42.2 2018-07-26T15:41:36 I'll destroy it 2018-07-26T15:50:23 *** duncanmv has quit IRC 2018-07-26T16:19:40 *** matthias_bgg has quit IRC 2018-07-26T16:22:57 *** mcaj has quit IRC 2018-07-26T17:04:10 *** enavarro has quit IRC 2018-07-26T17:06:19 *** dakechi has quit IRC 2018-07-26T17:24:39 *** Son_Goku has quit IRC 2018-07-26T17:59:46 *** Son_Goku has joined #opensuse-admin 2018-07-26T18:03:01 *** Son_Goku has quit IRC 2018-07-26T18:07:10 *** Son_Goku has joined #opensuse-admin 2018-07-26T18:35:51 *** Son_Goku has quit IRC 2018-07-26T18:53:53 *** ldevulder has quit IRC 2018-07-26T19:10:09 *** ldevulder has joined #opensuse-admin 2018-07-26T19:16:14 tampakrap: have a look at paste.o.o please... 2018-07-26T19:22:49 *** Fraser_Bell has joined #opensuse-admin 2018-07-26T19:22:50 *** Fraser_Bell has joined #opensuse-admin 2018-07-26T19:37:11 yeah I see it 2018-07-26T19:37:15 susepaste.org is working though 2018-07-26T19:37:21 so the problem should be on our proxy 2018-07-26T19:38:33 ah it's not going through the proxy, it's a cname 2018-07-26T19:38:42 so the problem is on the server itself 2018-07-26T19:39:23 so we need |miska| for that 2018-07-26T19:43:55 I have something else that you might be able to fix (actually okurz noticed it): 2018-07-26T19:44:31 ntpdate ntp{2,3}.infra.o.o results in 26 Jul 19:42:50 ntpdate[4592]: no server suitable for synchronization found 2018-07-26T19:45:05 (yesterday, ntp1.infra.o.o aka freeipa resulted in the same, but whatever you did to it today seems to have fixed it) 2018-07-26T19:45:32 as a result, the time on (probably) all VMs is wrong by half an hour 2018-07-26T19:46:04 my *guess* is that the ntpd -> chrony switch could have broken it 2018-07-26T19:47:55 yes I saw the ticket 2018-07-26T19:48:11 so today I restarted the whole freeipa service, that looks to be like an umbrella service 2018-07-26T19:48:25 which restarts services like ldap, ntp, kerberos and others 2018-07-26T19:48:42 so this probably did it, and we need to fix ntp[2-3].i.o.o as you mention 2018-07-26T19:49:06 I'm pretty sure though that we tested it after the migration and it worked, so possibly an update broke it 2018-07-26T19:53:24 is there any VM right now that is behind? 2018-07-26T19:53:43 try salt \* cmd.run date ;-) 2018-07-26T19:54:11 (I fixed the time manually only on water) 2018-07-26T19:54:21 pinot.infra.opensuse.org: 2018-07-26T19:54:23 Thu Jul 26 20:25:43 UTC 2018 2018-07-26T19:54:29 this one is on the future, what is it? 2018-07-26T19:54:47 all the others are fine 2018-07-26T19:55:02 the future VM for counter.o.o (and maybe also planet) 2018-07-26T19:55:17 maybe we can first abuse it as test VM for the new Leap 15 JeOS ;-) 2018-07-26T19:55:32 okay 2018-07-26T19:56:05 it doesn't have chrony or ntp running, so let's ignore it for now 2018-07-26T19:56:16 feel free to destroy it 2018-07-26T20:00:03 PROBLEM: HAProxy on anna.infra.opensuse.org - HAPROXY CRITICAL - Active service dale is DOWN on dale proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=anna.infra.opensuse.org&service=HAProxy 2018-07-26T20:03:07 *** Son_Goku has joined #opensuse-admin 2018-07-26T20:05:46 *** Son_Goku has quit IRC 2018-07-26T20:40:01 PROBLEM: HAProxy on elsa.infra.opensuse.org - HAPROXY CRITICAL - Active service dale is DOWN on dale proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=elsa.infra.opensuse.org&service=HAProxy 2018-07-26T21:13:13 *** duncanmv has joined #opensuse-admin 2018-07-26T21:13:28 *** ldevulder has quit IRC 2018-07-26T21:22:49 PROBLEM: Hosts syslog on monitor.infra.opensuse.org - CRITICAL: Found files older than 600 minutes /var/log/opensuse/hosts//simba.log was last modified on Thu Jul 26 11:15:02 2018 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=monitor.infra.opensuse.org&service=Hosts%20syslog 2018-07-26T21:42:50 RECOVERY: Hosts syslog on monitor.infra.opensuse.org - OK: Tested /var/log/opensuse/hosts/ - no files older than 240 minutes found ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=monitor.infra.opensuse.org&service=Hosts%20syslog 2018-07-26T22:13:32 *** Fraser_Bell has quit IRC 2018-07-26T22:14:57 *** lcp has quit IRC 2018-07-26T23:04:15 *** cboltz has quit IRC 2018-07-26T23:08:17 *** duncanmv has quit IRC