2020-08-24T02:13:30  *** okurz_ is now known as okurz
2020-08-24T06:10:21  <pjessen> dns problem?
2020-08-24T06:33:07  <pjessen> dns problem?
2020-08-24T08:25:14  <lcp> pjessen: I found a way to unfuck the database with encoding, but it's gonna be a lot of work
2020-08-24T08:33:35  <pjessen> lcp: no surprise
2020-08-24T08:36:18  <lcp> not particularly happy about having to do it tbh
2020-08-24T08:37:03  <pjessen> yeah, nobody really wants top touch it
2020-08-24T08:37:18  <pjessen> i tried and wasxted many hours
2020-08-24T08:37:54  <lcp> also it's not a 100% solution as far as I can tell, because it still leaves some unencoded things behind, so that's annoying
2020-08-24T08:52:42  *** Martchus_ is now known as Martchus
2020-08-24T08:58:36  <pjessen> lcp: have you considered leaving the current forums as a read-only archive and then a clean start with discourse ?
2020-08-24T09:08:25  <lcp> I haven't yet
2020-08-24T09:08:29  <lcp> I will let you know when I do
2020-08-24T09:09:05  <lcp> our forums might be way bigger than expected by the import script tbh
2020-08-24T11:21:55  <lcp> it's worth noting, there are some greek subforums that conflict by name, and I wonder if we could somehow remove them
2020-08-24T11:22:22  <lcp> they don't exist publicly but I also don't see it from admin control panel
2020-08-24T11:23:23  <lcp> nvm, I found them
2020-08-24T11:26:23  <lcp> malcolmlewis: I assume that READY TO DELETE forums are actually something that needs to be deleted?
2020-08-24T12:13:48  <lcp> https://gist.github.com/gschlager/bad57293004c0bfcddba might be helpful 🤔
2020-08-24T12:15:30  <cboltz> well, more or less ;-)
2020-08-24T12:16:16  <cboltz> our problem is that we have a "mixed" charset - the old half of a table has different encoding than the newer half
2020-08-24T12:16:42  <cboltz> while that gist works on whole tables
2020-08-24T12:16:57  <lcp> I can no problem deal with utf-8 to windows-1525 and again utf-8, but latin-1 is the problematic factor
2020-08-24T12:16:58  <lcp> I wish this database wasn't double encoded but oh well
2020-08-24T12:17:46  <lcp> yeah, I honestly doubt the second half is recoverable by the looks of it tbh
2020-08-24T12:18:15  <lcp> because it's filled with broken characters, and no wrongly encoded characters
2020-08-24T12:19:08  <lcp> somebody forcefully encoded it as something else which isn't recoverable imo
2020-08-24T12:21:04  <lcp> although maybe 🤔
2020-08-24T12:23:25  <lcp> no, it's �, so doesn't seem recoverable
2020-08-24T12:24:15  <cboltz> IIRC it could be double-encoded utf8, but that's really IIRC - I looked at it several months ago
2020-08-24T12:25:04  <lcp> this whole database is so unfortunate
2020-08-24T12:25:19  <cboltz> no objections on that ;-)
2020-08-24T12:26:08  <cboltz> when we moved the forums from Provo to Nuremberg, we decided not to touch the charset "fun" and to leave those old posts with broken charset
2020-08-24T12:26:17  <cboltz> maybe you should just do the same?
2020-08-24T12:26:21  <lcp> I'm not giving up yet, but I'm not that far from it
2020-08-24T12:26:54  <lcp> mainly because fixing stuff on importer side was easy, now I just have to deal with latin-1 stuff inside the db itself
2020-08-24T12:27:29  <lcp> also yeah, we will need 2 days for the importer script, it's taking its time
2020-08-24T12:28:09  <lcp> we will probably have a touch slimmed down db after removal of old subforums, but that's not a lot
2020-08-24T12:30:04  <lcp> (also it's not possible to import this db without removing those subforums anyway, because it squishes subsubsubforums into subsubforums, and the names have to be different, which isn't the case for the TO BE REMOVED subforums)
2020-08-24T14:11:38  <lcp> 3 days, not 2 days
2020-08-24T14:11:52  <lcp> that's a lot of things to import
2020-08-24T14:40:41  <lcp> the import of posts will take 26 hours alone
2020-08-24T15:03:25  *** okurz_ is now known as okurz
2020-08-24T17:36:29  <malcolmlewis> lcp, if they are ready to delete, I say go for it :)
2020-08-24T17:40:34  <lcp> malcolmlewis: I mean https://forums.opensuse.org/forumdisplay.php/946-READY-TO-DELETE and its subforums
2020-08-24T17:41:08  <pjessen> still no vpn, huh?
2020-08-24T17:42:37  <cboltz> confirmed, my vpn connection is also broken (the certificate gets VERIFY OK: , followed by a "Connection reset")
2020-08-24T17:42:43  <lcp> what happened to the vpn
2020-08-24T17:44:20  <cboltz> no idea, and I don't have a way to check without, well, vpn
2020-08-24T17:47:42  <lcp> I didn't play with freeipa, so not me this time
2020-08-24T17:48:05  <cboltz> ;-)
2020-08-24T17:49:48  <cboltz> klein: can you please check/fix the vpn?
2020-08-24T17:50:54  <lcp> I will probably be attempting another freeipa upgrade soonish
2020-08-24T17:51:38  <cboltz> pjessen: FYI: provo-mirror is still outdated (including /repositories). I'm basically running   while true ; do rsync ... ; done   since some days, but it looks like there's a big backlog, and I wonder if OBS builds faster than rsync can catch up ;-)
2020-08-24T17:51:58  <cboltz> so when the vpn is back, please check if the rsync push really works as expected
2020-08-24T18:12:43  <pjessen> cboltz: repositories should be pushed out as they change, maybe leave those till last
2020-08-24T18:13:00  <pjessen> sorry, didn't see your last msg
2020-08-24T18:16:22  <pjessen> last I checked on the push, it was fine
2020-08-24T18:17:11  <cboltz> given how many different repos my manually rsync already updated, I slightly ;-) doubt it's still fine
2020-08-24T18:17:55  <cboltz> for bonus fun, it seems some files let rsync "explode" and abort the sync
2020-08-24T18:18:54  <cboltz> I've seen   inflate returned -3 (0 bytes)    a few times
2020-08-24T18:19:08  <cboltz> and I've also seen some   rsync: read error: Connection reset by peer (104)
2020-08-24T18:20:24  <pjessen> I can check the push, I think we keep the logs per server. not sure for how long.
2020-08-24T18:21:43  <pjessen> does anyone know whats happening to the vpn?  this morning, mine was still okay, but DNS was down. then I ressytarted the vpn .....
2020-08-24T18:26:03  <pjessen> cboltz: inflate - should we be running with compression?
2020-08-24T18:27:22  <cboltz> the rsync cronjobs on provo-mirror (and also my manually started rsync) use   rsync -v -H --compress -ar0 --stats --timeout=21600 --no-motd --fuzzy --exclude=... --delete
2020-08-24T18:28:09  <pjessen> I wonder if that compress isn't superfluous - the rpms are already compressed
2020-08-24T18:28:20  <cboltz> good question
2020-08-24T18:28:29  <cboltz> the rpm payload is indeed compressed
2020-08-24T18:28:39  <cboltz> but I'm not sure about the header and especially the changelog
2020-08-24T18:29:00  <pjessen> hmm, dunno
2020-08-24T18:30:30  <cboltz> just tested:   strings apparmor-utils-lang-2.13.4-703.1.noarch.rpm   gives me the changelog in plaintext, which means it isn't compressed
2020-08-24T18:31:21  <cboltz> interestingly I first get a list of authors, and then the changelog content without the authors - no idea what the reason for this technical detail is
2020-08-24T18:34:39  <cboltz> apparmor-utils-lang-*.rpm is 84 kB, and   rpm -q --changelog apparmor-utils-lang | wc -c   says ~55 kB, so compression could make sense