2017-12-15T00:16:57 *** cboltz has quit IRC 2017-12-15T02:14:42 *** plinnell has quit IRC 2017-12-15T02:19:41 *** plinnell has joined #opensuse-admin 2017-12-15T02:30:54 *** heroes-bot has joined #opensuse-admin 2017-12-15T03:51:57 *** okurz has quit IRC 2017-12-15T03:53:09 *** okurz has joined #opensuse-admin 2017-12-15T04:30:00 *** matthias_bgg has joined #opensuse-admin 2017-12-15T04:31:53 *** plinnell has quit IRC 2017-12-15T04:37:25 *** plinnell has joined #opensuse-admin 2017-12-15T04:37:25 *** plinnell has joined #opensuse-admin 2017-12-15T06:49:53 *** sven15 has joined #opensuse-admin 2017-12-15T07:04:28 *** tigerfoot has joined #opensuse-admin 2017-12-15T07:15:46 *** kollex has joined #opensuse-admin 2017-12-15T07:36:31 *** asmorodskyi has joined #opensuse-admin 2017-12-15T08:12:39 *** matthias_bgg has quit IRC 2017-12-15T08:56:58 *** ldevulder has joined #opensuse-admin 2017-12-15T09:09:56 *** tigerfoot_ has joined #opensuse-admin 2017-12-15T09:10:17 *** tigerfoot has quit IRC 2017-12-15T09:10:23 *** tigerfoot_ is now known as tigerfoot 2017-12-15T09:14:37 *** solevi_ has joined #opensuse-admin 2017-12-15T09:20:20 *** matthias_bgg has joined #opensuse-admin 2017-12-15T09:38:30 *** jdsn has joined #opensuse-admin 2017-12-15T09:38:30 *** jdsn has joined #opensuse-admin 2017-12-15T10:11:12 *** deneb_alpha has joined #opensuse-admin 2017-12-15T10:14:05 *** cboltz has joined #opensuse-admin 2017-12-15T10:14:05 *** cboltz has joined #opensuse-admin 2017-12-15T11:56:50 *** asmorodskyi_ has joined #opensuse-admin 2017-12-15T11:58:48 *** asmorodskyi has quit IRC 2017-12-15T12:23:19 https://en.opensuse.org/Main_Page look broken 2017-12-15T12:24:12 should I report this somehow ? 2017-12-15T12:37:51 asmorodskyi_: can you define "broken"? 2017-12-15T12:37:59 it looks normal to me 2017-12-15T12:43:24 css is not loaded , something like this . So no text alignment , white background etc. 2017-12-15T12:44:18 I am using chrome 63.0.3239.84 (openSUSE Build) (64-bit) . Also might be the case that I am sitting from home under Nuremberg VPN ... 2017-12-15T12:45:11 if you saying that all is fine for you , might me some client side issue I will take a look closer later today 2017-12-15T12:45:26 can you check if the network console gives you some useful error messages? 2017-12-15T12:45:52 for the records: most of the wiki CSS comes from static.opensuse.org 2017-12-15T12:46:09 ERR_TOO_MANY_REDIRECTS for https://static.opensuse.org/themes/bento/css/style.fluid.css 2017-12-15T12:46:40 same for https://static.opensuse.org/themes/bento/js/global-navigation.js and https://static.opensuse.org/themes/bento/css/print.css 2017-12-15T12:51:15 testing with wget, I see a redirect to the same location, but interestingly after the redirect wget downloads style.fluid.css 2017-12-15T12:51:24 tampakrap: can you please check what's happening here? 2017-12-15T13:04:49 tampakrap: AFAIK static.o.o is served by narwal, narwal2 and narwal3, right? 2017-12-15T13:05:11 nginx is running on all of them and reachable locally (as in "curl -v localhost") 2017-12-15T13:05:55 but it seems something (firewall?) blocks access from the outside on narwal and narwal2 2017-12-15T13:06:12 so probably only narwal3 works as expected 2017-12-15T13:07:21 *** mcaj has quit IRC 2017-12-15T13:34:37 cboltz: I don't fully follow sorry 2017-12-15T13:34:53 how do I reproduce? 2017-12-15T13:35:16 maybe first answer if static.o.o is really served by 3 machines (narwal, narwal2 and narwal3) 2017-12-15T13:36:00 correct, you can see that in the haproxy config as well 2017-12-15T13:36:50 then try curl -v http://narwal3.infra.opensuse.org which will give you the static.o.o main page 2017-12-15T13:36:56 # https://static.opensuse.org 2017-12-15T13:36:58 # narwal1.infra.opensuse.org narwal2.infra.opensuse.org narwal3.infra.opensuse.org narwal4.infra.opensuse.org 2017-12-15T13:37:00 backend static 2017-12-15T13:37:02 option forwardfor 2017-12-15T13:37:04 option tcpka 2017-12-15T13:37:06 option httpchk OPTIONS / HTTP/1.1\r\nHost:\ static.opensuse.org 2017-12-15T13:37:08 server static1 192.168.47.12:80 check port 80 2017-12-15T13:37:10 server static2 192.168.47.13:80 check port 80 2017-12-15T13:37:12 server static3 192.168.47.10:80 check port 80 2017-12-15T13:37:14 server static4 130.57.72.1:80 check port 80 2017-12-15T13:37:32 also try curl -v http://narwal.infra.opensuse.org and http://narwal2.infra.opensuse.org 2017-12-15T13:37:59 ack, I see it now 2017-12-15T13:38:02 they both gave me a timeout when trying from another host 2017-12-15T13:39:19 I didn't know about narwal4, but it looks like it also refuses a connection from the outside 2017-12-15T13:39:38 indeed, I will remove it 2017-12-15T13:39:54 so from the proxy vm I can access narwal and narwal2 normally :) 2017-12-15T13:40:14 you checked any logs yet on the narwals? 2017-12-15T13:40:46 no 2017-12-15T13:40:58 I just tried curl localhost, which worked everywhere 2017-12-15T13:41:30 so either the webserver listens only on 127.0.0.1 (unlikely) or there's an overly restrictive firewall 2017-12-15T13:42:00 well, "everywhere" means narwal, narwal2 and narwal3 2017-12-15T13:42:10 narwal4 doesn't even respond to a ping 2017-12-15T13:42:26 it's the firewall 2017-12-15T13:42:34 I stopped it on narwal and narwal2 2017-12-15T13:42:42 and given the IP - is narwal4 in Provo? 2017-12-15T13:44:22 * cboltz wonders who sets up a firewall with port 80 closed on a webserver 2017-12-15T13:48:19 those machines need redeploy from scratch, they are old and dusty 2017-12-15T13:48:36 narwal4 is in provo yes, it's the machine we used for the power outage 2017-12-15T13:52:55 BTW: I noticed that status2 and provo-mirror work again. What was the problem with them? 2017-12-15T13:53:03 they were off 2017-12-15T13:53:24 any idea why? 2017-12-15T13:53:54 nope 2017-12-15T13:56:48 on status2, /v/l/messages ends at 2017-12-08T11:55:58.808700+00:00 without any messages that look like a shutdown 2017-12-15T13:57:07 followed by some null bytes which typically indicate a hard power-off 2017-12-15T13:57:28 the next log message is from booting it yesterday 2017-12-15T13:57:57 BTW: I enabled logrotate.timer on status2 2017-12-15T13:58:21 that sucks, so there might be a problem on the hypervisor 2017-12-15T13:58:33 or someone rebooted the hypervisor for whatever reason 2017-12-15T13:58:41 I don't know to which hypervisor they ran though 2017-12-15T14:04:11 have fun in finding out and making them reboot-proof ;-) 2017-12-15T14:09:48 *** ldevulder has quit IRC 2017-12-15T15:10:50 cboltz: how long you're going to be online today? 2017-12-15T15:12:11 *** tigerfoot has quit IRC 2017-12-15T15:54:49 I'll leave soon for a meeting with some Glühwein at a christmas market ;-) 2017-12-15T15:55:13 okay 2017-12-15T16:06:21 cboltz: I am wondering which locale settings I suppose to have to properly see u-umlaut ? de_DE.UTF-8 ? 2017-12-15T16:06:59 currently I have LC_CTYPE=en_US.UTF-8 and POSIX for everything else 2017-12-15T16:08:02 just thinking loud :) 2017-12-15T16:09:13 utf8 supports everything including klingon ;-) so it shouldn't matter if you have de_DE or en_US as long as it's utf8 2017-12-15T16:10:32 cool , but why I can't read Gluhwein than ? :) 2017-12-15T16:11:37 because it's something you have to drink, not to read? ;-) 2017-12-15T16:11:55 make sense :D 2017-12-15T16:14:07 I can at least offer some cookies: http://paste.opensuse.org/70395959 2017-12-15T16:15:08 *** sven15 has quit IRC 2017-12-15T16:17:10 looks really tasty , good addition to Gluhwein ;) 2017-12-15T16:21:47 *** cboltz has quit IRC 2017-12-15T16:49:03 *** matthias_bgg has quit IRC 2017-12-15T17:33:46 *** deneb_alpha has quit IRC 2017-12-15T17:38:22 asmorodskyi: for me over riot.im the message from cboltz also displayed wrong. Can you read Glühwein? 2017-12-15T18:14:11 *** Son_Goku has joined #opensuse-admin 2017-12-15T18:29:28 *** fvogt has joined #opensuse-admin 2017-12-15T18:29:44 *** nicolasbock has joined #opensuse-admin 2017-12-15T18:54:15 yeah :) 2017-12-15T18:54:29 *** Son_Goku has quit IRC 2017-12-15T18:55:03 okurz[m]: I can read your Glühwein properly :) 2017-12-15T18:55:41 or probably it's because I just drink it ? :D 2017-12-15T19:43:36 PROBLEM: MySQL WSREP recv on galera2.infra.opensuse.org - CRIT wsrep_local_recv_queue_avg = 1.000000 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera2.infra.opensuse.org&service=MySQL%20WSREP%20recv 2017-12-15T19:44:09 RECOVERY: MySQL WSREP recv on galera1.infra.opensuse.org - OK wsrep_local_recv_queue_avg = 0.048387 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera1.infra.opensuse.org&service=MySQL%20WSREP%20recv 2017-12-15T19:46:52 *** asmorodskyi_ has quit IRC 2017-12-15T20:00:05 PROBLEM: HAProxy on mufasa.infra.opensuse.org - HAPROXY CRITICAL - Active service keyserver-http is DOWN on keyserver proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=mufasa.infra.opensuse.org&service=HAProxy 2017-12-15T20:05:57 *** matthias_bgg has joined #opensuse-admin 2017-12-15T21:00:44 *** kollex has left #opensuse-admin 2017-12-15T21:15:59 *** cboltz has joined #opensuse-admin 2017-12-15T21:44:02 *** cboltz has quit IRC 2017-12-15T22:11:43 *** nicolasbock has quit IRC 2017-12-15T22:13:36 RECOVERY: MySQL WSREP recv on galera2.infra.opensuse.org - OK wsrep_local_recv_queue_avg = 0.470588 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera2.infra.opensuse.org&service=MySQL%20WSREP%20recv 2017-12-15T22:29:43 *** matthias_bgg has quit IRC 2017-12-15T23:28:48 *** fvogt has quit IRC