2022-11-04T01:39:02  *** teepee_ is now known as teepee
2022-11-04T09:20:45  *** teepee_ is now known as teepee
2022-11-04T11:25:24  *** teepee_ is now known as teepee
2022-11-04T11:31:50  <darix-> cboltz: zypper dup and tell me if the fix works for you as well
2022-11-04T11:31:55  <darix-> cboltz: TIA
2022-11-04T11:53:37  <cboltz> darix-: works, I can see the diffs again :-)
2022-11-04T13:31:36  <kervel> hello, i try to wget https://download.opensuse.org/tumbleweed/appliances/openSUSE-MicroOS.x86_64-OpenStack-Cloud.qcow2 but that gives me an error 502. status.opensuse.org reports everything running fine
2022-11-04T13:32:02  <kervel> (i'm trying to use https://github.com/kube-hetzner/terraform-hcloud-kube-hetzner)
2022-11-04T13:50:22  <anikitin> kervel, the service was restarted twice around that time because it had bad performance, is it still the case?
2022-11-04T13:52:54  <anikitin> (I moved DB away from the galera cluster back to local one, because it couldn't keep with load. Individual queries were fast, not sure why it doesn't scale with load)
2022-11-04T13:54:06  <kervel> now seems fine!
2022-11-04T13:56:35  <darix-> anikitin: maybe you could use your mariadb skills to help fixing it?
2022-11-04T14:00:31  <anikitin> I think it is expected that galera (or even binary logging enabled) cannot compete with single server without binlog. It is still able to do it, just slower. Plus other apps put some load there
2022-11-04T14:01:36  <anikitin> plus haproxy adds small overhead
2022-11-04T14:01:55  <darix-> nothing of that should make it unusable for you
2022-11-04T14:02:15  <darix-> plus we run galera for buildservice and that sees a very decent load as well and works just fine
2022-11-04T14:02:42  <bmwiedemann[m]> logs show 22k requests/minute
2022-11-04T14:04:51  <darix-> uhm
2022-11-04T14:19:41  <anikitin> do we have innodb buffer pool statistics? grafana doesn't seem to show anything useful
2022-11-04T14:19:50  <darix-> we do not right
2022-11-04T14:20:06  <darix-> we should resetup PMM
2022-11-04T14:33:37  <anikitin> my guess is that MirrorCache relies on logging stats and galera just do it slower, holding UI connections for a bit longer. (I guess we can optimize it further, but I am really not sure if I really want to use galera.) This is profiling for the same insert on relatively idle galera and on busy standalone server. I believe it gets worse for galera with load https://ibb.co/qjqB6Zw
2022-11-04T14:39:10  <bmwiedemann[m]> so galera is 5x slower there.
2022-11-04T14:43:15  <anikitin> it is also kind of expected, system with distributed consistency cannot compete with standalone in writes. Commit is the slowest , but typically batches are 300 rows, so it shouldn't have that big impact. MirrorCache definitely can do writing less stats, which will help here
2022-11-04T18:23:00  *** teepee_ is now known as teepee
2022-11-04T21:31:30  <darix-> that/cr