2020-02-05T01:10:11  -heroes-bot- PROBLEM: MySQL WSREP recv on galera1.infra.opensuse.org - CRIT wsrep_local_recv_queue_avg = 1437.618188 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera1.infra.opensuse.org&service=MySQL%20WSREP%20recv
2020-02-05T01:35:20  -heroes-bot- PROBLEM: HTTP l10n on nuka.infra.opensuse.org - CRITICAL - Socket timeout after 10 seconds ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=nuka.infra.opensuse.org&service=HTTP%20l10n
2020-02-05T01:45:11  -heroes-bot- RECOVERY: HTTP l10n on nuka.infra.opensuse.org - HTTP OK: HTTP/1.1 200 OK - 41228 bytes in 0.450 second response time ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=nuka.infra.opensuse.org&service=HTTP%20l10n
2020-02-05T02:39:21  -heroes-bot- PROBLEM: HTTP l10n on nuka.infra.opensuse.org - CRITICAL - Socket timeout after 10 seconds ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=nuka.infra.opensuse.org&service=HTTP%20l10n
2020-02-05T03:41:28  *** okurz_ is now known as okurz
2020-02-05T04:43:55  -heroes-bot- PROBLEM: PSQL locks on mirrordb1.infra.opensuse.org - POSTGRES_LOCKS CRITICAL: DB postgres total waiting locks: 8 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=mirrordb1.infra.opensuse.org&service=PSQL%20locks
2020-02-05T04:53:55  -heroes-bot- RECOVERY: PSQL locks on mirrordb1.infra.opensuse.org - POSTGRES_LOCKS OK: DB postgres total=20 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=mirrordb1.infra.opensuse.org&service=PSQL%20locks
2020-02-05T06:31:34  * lcp uploaded an image: Screenshot from 2020-02-05 07-31-15.png (85KB) < https://matrix.org/_matrix/media/r0/download/matrix.org/FoXKbdFeGYwYLwDCkrUjohZr >
2020-02-05T06:31:37  <lcp> cboltz: fixed it
2020-02-05T07:18:24  <lcp> alright, I wrote a set of liquid calculations to limit the number of the pages to 5
2020-02-05T07:55:52  <lcp> cboltz: I might also need some help with getting matrix-synapse packages in openSUSE:infrastructure, because I don't even know where to begin
2020-02-05T07:56:22  <lcp> I have a set of packages which are required to get it working on leap 15.2
2020-02-05T08:30:08  *** ldevulder__ is now known as ldevulder
2020-02-05T10:05:51  <okurz> lcp: matrix-synapse should be in Factory, branch from there?
2020-02-05T10:06:38  <okurz> or use https://build.opensuse.org/package/show/network:messaging:matrix/matrix-synapse
2020-02-05T10:31:02  <lcp> okurz: that much I know
2020-02-05T11:52:39  <enavarro_suse> Hi, does it changed the redirection of obs-reviewlab.opensuse.org ?  It was working yesterday, but not today. This should not return a 503: https://obs-reviewlab.opensuse.org/saraycp-responsive-uxlayout
2020-02-05T12:05:03  -heroes-bot- PROBLEM: PostgreSQL standby on mirrordb1.infra.opensuse.org - POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB mb_opensuse2 (host:mirrordb2) 331472936 and 129 seconds ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=mirrordb1.infra.opensuse.org&service=PostgreSQL%20standby
2020-02-05T12:35:03  -heroes-bot- RECOVERY: PostgreSQL standby on mirrordb1.infra.opensuse.org - POSTGRES_HOT_STANDBY_DELAY OK: DB mb_opensuse2 (host:mirrordb2) 0 and 2 seconds ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=mirrordb1.infra.opensuse.org&service=PostgreSQL%20standby
2020-02-05T12:55:37  *** adrianS_ is now known as adrianS
2020-02-05T16:21:25  <kl_eisbaer> http://monitor.opensuse.org/grafana/playlists/play/1
2020-02-05T16:31:02  -heroes-bot- RECOVERY: NRPE on svn.infra.opensuse.org - NRPE v3.2.1 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=svn.infra.opensuse.org&service=NRPE
2020-02-05T18:15:19  -heroes-bot- PROBLEM: MySQL WSREP recv on galera2.infra.opensuse.org - CRIT wsrep_local_recv_queue_avg = 4.119403 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera2.infra.opensuse.org&service=MySQL%20WSREP%20recv
2020-02-05T19:06:51  -heroes-bot- PROBLEM: SSH on riesling.infra.opensuse.org - SSH CRITICAL - OpenSSH_7.9 (protocol 2.0) version mismatch, expected OpenSSH_7.2 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=riesling.infra.opensuse.org&service=SSH
2020-02-05T19:30:50  <kbabioch> @all: is it really a good idea to have all of the monitoring data from grafana publicly accessible ...
2020-02-05T19:30:56  <kbabioch> http://monitor.opensuse.org/grafana/d/wGgaPlciz/postgresql-overview?orgId=1
2020-02-05T19:31:03  <kbabioch> (let's forget for a minute that it is even not https)
2020-02-05T19:32:45  <cboltz> it also supports https (maybe a redirect http -> https would be nice?)
2020-02-05T19:39:17  <cboltz> FYI: The wiki (finally) runs on Leap 15.1 since some minutes :-)
2020-02-05T19:47:22  <kbabioch> cboltz: cool, great job ... the infrastructure gets better (running on supported OS and everything) with steps like that
2020-02-05T19:48:18  <cboltz> now I "only" need to find some time to update MediaWiki ;-)
2020-02-05T19:48:46  <cboltz> (and before that, get a newer elasticsearch on water3)
2020-02-05T19:51:24  <cboltz> speaking about things getting better - can you please have a look at https://gitlab.infra.opensuse.org/infra/salt/merge_requests/281 (Ctrl-F "last blocker")?
2020-02-05T19:58:21  *** ldevulder__ is now known as ldevulder
2020-02-05T20:05:19  -heroes-bot- RECOVERY: MySQL WSREP recv on galera2.infra.opensuse.org - OK wsrep_local_recv_queue_avg = 0.492857 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera2.infra.opensuse.org&service=MySQL%20WSREP%20recv
2020-02-05T20:40:04  -heroes-bot- PROBLEM: HAProxy on elsa.infra.opensuse.org - HAPROXY CRITICAL - Active service obsreview is DOWN on obsreview proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=elsa.infra.opensuse.org&service=HAProxy
2020-02-05T20:40:06  -heroes-bot- PROBLEM: HAProxy on anna.infra.opensuse.org - HAPROXY CRITICAL - Active service obsreview is DOWN on obsreview proxy ! ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=anna.infra.opensuse.org&service=HAProxy
2020-02-05T23:55:04  -heroes-bot- PROBLEM: PostgreSQL standby on mirrordb1.infra.opensuse.org - POSTGRES_HOT_STANDBY_DELAY CRITICAL: DB mb_opensuse2 (host:mirrordb2) 654789544 and 87 seconds ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=mirrordb1.infra.opensuse.org&service=PostgreSQL%20standby