2025-07-05T01:34:33 Hello. how do i report an issue with mirrorcache? I'm trying to do a dup on tumbleweed but https://mirrorcache-au.opensuse.org  seems to be failing with 500 2025-07-05T03:59:57 *** teepee_ is now known as teepee 2025-07-05T06:21:24 Guest6: Yes, it is broken since some days. Please use download.o.o instead 2025-07-05T12:37:35 Jul 05 12:05:28 quiz quiz-update[5341]: Server for instance main still not running after 12 seconds 2025-07-05T12:37:41 which means quiz.o.o is currently down 2025-07-05T12:38:32 if someone wants to debug it, please do so in the next 20 minutes - otherwise the next timer run might fix it 2025-07-05T12:38:49 (the error mail shows the quiz got some translation updates) 2025-07-05T15:27:08 *** teepee_ is now known as teepee 2025-07-05T16:50:59 main quiz instance did come back by itself (or maybe it took longer than 12 seconds to boot? though that would be weird) but didn't reach the chmod part in rework_instances() 2025-07-05T16:51:18 (so repaired for now via `chmod 660 /run/quiz/main.sock`) 2025-07-05T16:51:36 might make sense to also check existing instances for the right socket mode as part of the rescan 2025-07-05T17:36:37 indeed 2025-07-05T17:36:41 thanks for checking and fixing! 2025-07-05T18:28:57 Can you guys please have a look at meet.o.o, we're getting kicked out all the time 2025-07-05T18:29:25 Maybe a reboot/restart ? 2025-07-05T18:52:49 hm.. bugzilla is going crazy, I received 9 emails for a single comment 2025-07-05T18:57:15 or mx1/mx2.o.o maybe, all emails have different 'Received' time 2025-07-05T19:09:51 +3 more copies 2025-07-05T19:10:03 it seems we indeed had some issues on mx2.o.o (spam filter timed out, seems to have fixed itsself in the meantime) 2025-07-05T19:10:20 that said - do you use your @o.o address in bugzilla? 2025-07-05T19:10:39 the emails are sent to opensuse-kde-bugs@opensuse.org 2025-07-05T19:10:56 then they're relayed to my own email 2025-07-05T19:11:14 ok, then they clearly went via mx*.o.o ;-) 2025-07-05T19:14:53 we still have several log messages like postfix/smtpd[31299]: warning: timeout talking to proxy ipv6-localhost:10024 2025-07-05T19:17:18 the load on mx* is quite high - mx1 load is currently > 20, and I catched mx2 with 15 (now down to 2) 2025-07-05T19:28:20 oh, and mailman.i.o.o has ~1200 mails in mailq, many of them with (lost connection with relay.infra.opensuse.org[2a07:de40:b27e:1203::10] while sending end of data -- message may be sent more than once) 2025-07-05T19:32:12 restarting clamd and spampd on mx* seems to have helped, the mailman queue is shrinking (now at 800 mails) 2025-07-05T19:36:08 however, mx* are both nearly out of memory (only ~300 MB for free, cache and available) which probably slows them down 2025-07-05T19:58:50 mailman mailq is down to 73 mails (interestingly all from security-announce to a single mail address, no idea why) 2025-07-05T20:19:02 ... and now the SUSE mailserver is able to deliver security announcements again, which fills the mailq again ;-) 2025-07-05T20:23:01 I guess it will take a while until all the external mailservers submitted whatever mails they have queued for us 2025-07-05T20:33:19 cboltz: Progress is gone crazy, keeps getting an email I sent hours ago and creating multiple tickets? 2025-07-05T20:33:47 right, that's a side effect of the mail issue, see above 2025-07-05T20:34:11 ahh ok 2025-07-05T20:34:38 the mailq is smaller now, so we shouldn't get too many additional duplicates - but they might still happen 2025-07-05T20:35:36 ok, I'll just ignore 2025-07-05T20:40:59 I won't stop you from closing some of the duplicates ;-) (which of course creates some more mails, but compared to the amount of security announcements, that doesn't really matter) 2025-07-05T20:41:33 cboltz: I will when it calms down 😉 2025-07-05T21:06:11 bonus fun: https://progress.opensuse.org/issues/185299 2025-07-05T21:09:35 cboltz: does the time being out by 3 seconds affect this? 2025-07-05T21:10:02 or is that just when it could log events... 2025-07-05T21:11:04 the timestamp diff isn't really relevant (actually I have enouch incoming bounces for this rewritten address over several hours, and just picked a close-enough one) 2025-07-05T21:13:03 looks like syncing the secret indeed helped - mx1 just did some successful rewrites for this address :-) 2025-07-05T21:14:29 ("some" actually means 200...) 2025-07-05T21:17:06 so a side effect of this fix is that mailman bounce handling will now work better, and unsubscribe some broken addresses 2025-07-05T21:42:03 egotthold: regarding what knurpht reported above, I notice jicofo.sh[1156]: SEVERE: Health check failed in PT0.0000036S: Result(success=false, hardFailure=true, responseCode=null, sticky=false, message=No operational bridges available (total bridge count: 1)) 2025-07-05T21:44:12 acidsys: for increasing the RAM on mx*, I guess you need to reboot the VMs? Feel free to do so at any time - a short downtime is better than the current RAM situation (we even got some OOM kills) 2025-07-05T21:44:50 and the videobridge seemed to be running but half-dead with some MergingDatagramSocket$SocketContainer.runInReaderThread#770: Failed to receive: java.net.SocketException: Socket closed, so I restarted the service 2025-07-05T21:45:21 cboltz: apply highstate on falkors and then shutdown VM when suitable 2025-07-05T21:46:29 and probably not need to add, but maybe wait for one to be back until shutting down the second one ;) 2025-07-05T21:46:48 I never did that, and probably shouldn't try it for the first time around midnight ;-) (especially because I'm not familiar with managing the VMs) 2025-07-05T21:47:03 therefore I'd prefer if you can do it 2025-07-05T21:52:31 done 2025-07-05T21:53:09 thanks! 2025-07-05T21:59:17 I also pinged leon regarding jitsi because not sure they read here 2025-07-05T22:00:52 acidsys: Thanks 2025-07-05T22:01:16 knurpht: does it work now at least? 2025-07-05T22:01:56 acidsys: You want us to test? 2025-07-05T22:03:09 well after I restarted jitsi-videobridge there don't seem to be any errors (but maybe there's no session) 2025-07-05T22:04:00 Onuralp and me are in there now 2025-07-05T22:04:52 great 2025-07-05T22:16:51 nice, the german mailinglist got "some" duplicated mails during the last hours - one mail was delivered 28 times 2025-07-05T22:18:33 unsurprisingly, people already wondered, and ironically the "I repeatedly got that mail" also got delivered to the list several times ;-) 2025-07-05T22:18:46 I just sent a short explanation mail 2025-07-05T22:19:54 security-announce also got a few duplicates, but really just a few 2025-07-05T22:20:14 on the positive side, other lists (at least those where I'm subscribed) don't seem to be affected/flooded by duplicate mails