2022-06-27T07:57:33  <a-865k> unable to login on bugzilla.o.o in over 36 hours, while all other o.o. logins attempted work normally
2022-06-27T08:12:05  <a-865k> https://progress.opensuse.org/issues/113069
2022-06-27T10:50:25  <SasiOlin[m]> cboltz: I'm curious if you have ever seen an issue like this, I am trying to use mysqldump to dump the database with the correct encoding, but I have only had success doing that with utf8 data. the interesting part is that when I use mysql command itself with --default-character-set switch, that I'm using for mysqldump, I can see the data can be read with the correct encoding (using a select statement to output it to cli), but I
2022-06-27T10:50:26  <SasiOlin[m]> always get wrongly encoded data with mysqldump no matter what I do
2022-06-27T11:17:39  <cboltz> I think that's expected ;-)
2022-06-27T11:18:13  <cboltz> when using mysql + a SELECT query, you specified a charset for the database connection (either explicit, or by using the default charset)
2022-06-27T11:19:01  <SasiOlin[m]> I am using --default-character-set for both
2022-06-27T11:19:49  <cboltz> AFAIK mysqldump dumps the database "as is" (think "binary"), and includes a statement about the used charset so that re-importing uses the same charset
2022-06-27T11:20:10  <cboltz> (but when you open the dump in an editor, you'll see it in the editor's default charset)
2022-06-27T11:21:18  <SasiOlin[m]> yeah, I have also been playing with set fileencoding and set encoding in vi with no avail, that being said I did find it works just fine if the file is short enough to include latin1 encoded characters early on in the file for some reason
2022-06-27T11:21:46  <SasiOlin[m]> but setting fileencoding and encoding in vi breaks the display even more for some reason
2022-06-27T11:24:07  <cboltz> sounds like charset detection in vim works if latin1 appears early enough (check "set encoding"), but IIRC I also didn't have too much success with things like "set encoding=...")
2022-06-27T11:24:31  <SasiOlin[m]> yeah, that doesn't work
2022-06-27T11:24:59  <cboltz> does the dump have sql statements that say it's latin1, or does it say utf8?
2022-06-27T11:25:11  <cboltz> (or, to make things simpler: where can I look at the dump?)
2022-06-27T11:25:58  <SasiOlin[m]> that's a bit complicated, half the data is utf8 encoded and half is latin1 encoded
2022-06-27T11:26:04  <SasiOlin[m]> but the table definition says latin1
2022-06-27T11:26:45  <SasiOlin[m]> I've been getting through the dumps on jekyll.infra.opensuse.org:/tmp, so they should be easy enough to find
2022-06-27T11:28:14  <SasiOlin[m]> you can find a niceish example in posts.sql if you look for `2133578` for example
2022-06-27T11:30:25  <SasiOlin[m]> because those files don't get interpreted with the correct encoding, they are also equally broken when piping them into mysql later down the line, which is annoying
2022-06-27T11:39:31  <cboltz> ok, so I grepped 2133578 into its own file (/tmp/grepresult) to make opening it a bit ;-) faster
2022-06-27T11:40:30  <cboltz> one interesting thing is that vi says on startup   "grepresult" [converted] 1L, 1043927B
2022-06-27T11:40:58  <cboltz> no idea what it converts, I only know that it ends up with encoding set to utf-8
2022-06-27T11:41:39  <cboltz> next, I looked at the raw file:   dd if=/tmp/grepresult bs=1 skip=856310 count=10 | hexdump -C
2022-06-27T11:41:52  <cboltz> that results in   00000000  6d c3 b6 67 6c 69 63 68  65 72                    |m..glicher|
2022-06-27T11:42:49  <cboltz> which means that 2133578 is utf-8 (two bytes for the "�")
2022-06-27T11:45:44  <cboltz> I wouldn't be too surprised if adding   /*!40101 SET NAMES utf8 */   in posts.sql fixes the import
2022-06-27T11:50:35  <SasiOlin[m]> yeah, I don't really understand why it's so broken tbh
2022-06-27T11:51:55  <cboltz> try   ( echo ' /*!40101 SET NAMES utf8 */  ' ; cat posts.sql ) > posts_labeled_utf-8.sql
2022-06-27T11:52:08  <cboltz> with some luck, mysql will happily (and correctly) import that
2022-06-27T11:52:34  <cboltz> (note: I avoid vi here because I'm not sure what the  [converted]  does to the file)
2022-06-27T11:52:35  <SasiOlin[m]> yeah, let me try
2022-06-27T11:52:47  <SasiOlin[m]> sure
2022-06-27T11:57:03  *** teepee_ is now known as teepee
2022-06-27T11:59:20  <cboltz> I have to leave
2022-06-27T11:59:22  <cboltz> if you find more breakage, ask me tonight ;-)
2022-06-27T12:01:10  <SasiOlin[m]> sure
2022-06-27T12:56:34  <malcolmlewis> Anyone here with power to add a new package into openSUSE:infrastructure if requested?
2022-06-27T13:49:26  <malcolmlewis> SasiOlin[m], if the import becomes too much of a headache, we are happy just to have a read-only archive somewhere?
2022-06-27T13:49:58  <malcolmlewis> SasiOlin[m], is it possible just to import the structure as a test to see what it look like?
2022-06-27T13:50:27  <malcolmlewis> Knurpht-Gertjan[, ^^
2022-06-27T13:53:46  <SasiOlin[m]> I actually have a db with a faulty import where every user is system
2022-06-27T13:54:18  <SasiOlin[m]> that's not particularly usable, but I made a mistake and one import takes over 2 days, so I went with it for now
2022-06-27T13:55:31  <SasiOlin[m]> I also don't really wanna make it public because I would have to go through all of the old admin categories and hide them from view
2022-06-27T13:55:43  <SasiOlin[m]> might as well I guess
2022-06-27T13:57:19  <malcolmlewis> SasiOlin[m] ^^ as in?
2022-06-27T13:58:04  <SasiOlin[m]> I will hide all of the admin/mod categories from public view and upload the database to discourse.opensuse.org
2022-06-27T13:58:21  <malcolmlewis> SasiOlin[m], ahh ok :)
2022-06-27T14:01:49  <SasiOlin[m]> as for the read only archive, I would probably want to avoid it as much as possible, but for stuff like blogs/cms, there isn't really avoiding it
2022-06-27T14:04:48  <Knurpht-Gertjan[> <SasiOlin[m]> "I actually have a db with a..." <- Yeah, it's always good to get some idea with real data.
2022-06-27T15:43:59  <kl_eisbaer> malcolmlewis: > Anyone here with power to add a new package into openSUSE:infrastructure if requested? - There is currently no request pending in the openSUSE:infrastructure repository. What exactly are you looking for?
2022-06-27T16:00:43  <malcolmlewis> lkocman[m], you mean me?
2022-06-27T16:02:06  <lkocman[m]> whoever wants to submit anything to opensuse infra repo :-)
2022-06-27T16:08:20  <malcolmlewis> lkocman[m], hmmm, so what to call the ignite container? containers-ignite
2022-06-27T16:25:26  <kl_eisbaer> malcolmlewis: as we already have some container-* in this repo: yes, something like container-ignite should be ok.
2022-06-27T16:27:00  <kl_eisbaer> Btw: latest state from the upgrade runs inside openSUSE Infra:
2022-06-27T16:27:16  <lkocman[m]> it's called fuel-ignition
2022-06-27T16:27:18  <kl_eisbaer>       6     openSUSE Tumbleweed
2022-06-27T16:27:18  <kl_eisbaer>      17     openSUSE Leap 15.3
2022-06-27T16:27:18  <kl_eisbaer>      52     openSUSE Leap 15.4
2022-06-27T16:27:18  <kl_eisbaer>       2     SUSE Linux Enterprise Server 12 SP5
2022-06-27T16:27:18  <kl_eisbaer>       1     CentOS Stream 8
2022-06-27T16:27:19  <kl_eisbaer>       1     openSUSE Leap 15.2
2022-06-27T16:28:03  <lkocman[m]> we could simply call it fuel-ignition, or what's the naming convention for containarized apps if we have one :-)
2022-06-27T16:28:09  <kl_eisbaer> So we are on a good progress towards "using openSUSE everywhere" and "especially openSUSE Tumbleweed + latest Leap" :-)
2022-06-27T16:28:43  <malcolmlewis> lkocman[m], so container-fuel-ignition (so folks know it's a container)
2022-06-27T16:29:21  <kl_eisbaer> lkocman[m]: well, at the moment, the naming scheme is simply container-[enter_your_name_here]. For Containers depending on other Containers, we try to have the container-[dep_container]-[your_name]
2022-06-27T16:29:32  <lkocman[m]> sounds good
2022-06-27T16:29:46  <kl_eisbaer> malcolmlewis: +1
2022-06-27T16:31:39  <kl_eisbaer> Now I'm waiting for our Wiki/Elections-Master (cboltz, klingeling ;-) and anikitin to upgrade the remaining 15.3 to 15.4. ...and maybe another machine from DimStar, which could go to 15.4. After that, we can announce that the openSUSE infrastructure is done with the migration to Leap 15.4
2022-06-27T16:32:29  <lkocman[m]> please make sure to raise any high priority blockers to me, if it's from SLES I can help to push to get it fixed
2022-06-27T16:32:50  <lkocman[m]> this week is not ideal for escalations though :-) (hackweek)
2022-06-27T16:34:23  <DimStar> kl_eisbaer: what machine of mine? collab server i guess?
2022-06-27T16:35:56  <kl_eisbaer> DimStar: correct. IMHO the machine should be able to be upgraded. But as you are actively maintaining it (at least you log in from time to time ;-) - I want to leave it up to you, when time permits.
2022-06-27T16:35:56  <kl_eisbaer> #
2022-06-27T16:36:39  <DimStar> kl_eisbaer: logging in is mostly to kick it back into submission
2022-06-27T16:36:54  <kl_eisbaer> For most of the usual "I wanted the service, but not the work coming along with maintaining it", I already did the usual "zypper clean -a; zypper --releasever 15.4 ref; $fixing_repos; zypper --releasever 15.4 dup"
2022-06-27T16:37:15  <DimStar> but it should be ok to upgrade that one...
2022-06-27T16:37:39  <kl_eisbaer> Even if this is the only thing you are doing: most of the other machines only see my login for years when I check if the last kernel update was installed properly ;-)
2022-06-27T16:38:30  <kl_eisbaer> But with just 17(+1) machine left on 15.3, we are already faster than the last time.
2022-06-27T16:38:34  <DimStar> we shouuld use suma :p
2022-06-27T16:38:50  <kl_eisbaer> Nope: uyuni :-)
2022-06-27T16:39:10  <Eighth_Doctor> does anyone know what's going on with MirrorCache? I'm getting errors like this regularly: https://paste.centos.org/view/ac142c0f
2022-06-27T16:39:15  <DimStar> as long as it collects patch levels intoba central mgmtui
2022-06-27T16:39:18  <kl_eisbaer> But that would currently be just another machine to maintain, as nobody wants to do the work anyway ;-)
2022-06-27T16:40:25  <kl_eisbaer> Eighth_Doctor: ping anikitin, please. But maybe the HaProxy in Provo has again some Apparmor problems, if your errors are from today.
2022-06-27T16:40:45  <Eighth_Doctor> yes
2022-06-27T16:40:50  <Eighth_Doctor> Friday and today
2022-06-27T16:41:14  <kl_eisbaer> Maybe it's time to give the mirrorbrain-us machine a public IP. But that's up to anikitin to decide.
2022-06-27T16:49:18  <anikitin> Eighth_Doctor, 'connection refused' looks like haproxy problem. on Friday there was an outage with it, I can check if today it was rebooted. Can you reproduce it with curl ?
2022-06-27T16:51:31  <Eighth_Doctor> testing again now
2022-06-27T16:51:32  <anikitin> https://paste.opensuse.org/view/raw/32669358
2022-06-27T16:54:59  <Eighth_Doctor> seems to be okay atm
2022-06-27T17:44:51  *** teepee_ is now known as teepee
2022-06-27T19:11:59  <SasiOlin[m]> Conan Kudo: would you have a look at https://progress.opensuse.org/issues/113090
2022-06-27T19:12:42  <Eighth_Doctor> I can't right now, as I'm on my work computer in the office
2022-06-27T19:12:53  <Eighth_Doctor> but as I haven't touched anything in there, I don't know why it would 502
2022-06-27T19:16:36  <SasiOlin[m]> me neither, I am assuming an update broke it
2022-06-27T19:16:54  <SasiOlin[m]> since lars came back from his break, I am assuming he did an update >:D
2022-06-27T19:33:07  <SasiOlin[m]> cboltz: so, I was testing the wrong thing entirely, I was testing the exceptions, where the encoding was utf8 to begin with, instead of the thing I was actually supposed to test were latin1 bits
2022-06-27T19:34:26  <SasiOlin[m]> `2205557` is an example of a wrongly encoded latin1 post
2022-06-27T19:34:34  <cboltz> oh, nice ;-)
2022-06-27T19:35:07  <cboltz> yeah, it's utf-8 (at least the part I looked at)
2022-06-27T19:37:10  <SasiOlin[m]> the thing is, those easly utf8 posts will become broken, because there really is no good way to filter them out
2022-06-27T19:37:32  <SasiOlin[m]> or I just haven't found one
2022-06-27T19:38:31  <SasiOlin[m]> the date of switch to utf8 only gets us so far, to turn posts into proper encoding, some earlier posts were utf8 and there isn't much I can do to fix it
2022-06-27T19:38:43  <cboltz> in theory you could (for each text/post)   echo "$text" | file -   and it will tell you the charset - but that will take "a while"[tm]
2022-06-27T19:39:11  <cboltz> in practise it's probably good enough if you can fix 99% of the posts - the end result will still be much better than what we have now
2022-06-27T19:40:11  <SasiOlin[m]> actually I'm fairly sure that doesn't work, latin1 dump is also really broken
2022-06-27T19:40:18  <SasiOlin[m]> I don't exactly understand how
2022-06-27T19:40:38  <SasiOlin[m]> check out onepost.sql I think it was
2022-06-27T19:43:42  <SasiOlin[m]> the byte is correct for latin1 encoding, but it isn't displayed correctly
2022-06-27T19:44:11  <SasiOlin[m]> latin1 or windows1252, mysql's latin1 is windows1252 and not the iso-8859-1 that usually gets called latin1
2022-06-27T19:44:26  <SasiOlin[m]> it doesn't matter in 99% of cases tho
2022-06-27T19:46:19  <cboltz> the fc bytes are indeed � in latin1, and e4 is �, so that looks correct
2022-06-27T19:46:39  <cboltz> what / where do you mean with "isn't displayed correctly"?
2022-06-27T19:57:48  <SasiOlin[m]> cat, less, vi all don't display them correctly
2022-06-27T19:58:21  <cboltz> well, today's terminals expect everything to be utf-8
2022-06-27T19:58:40  <SasiOlin[m]> but the worst part is that importing them back to a mysql database, they don't get interpreted as latin1
2022-06-27T19:58:42  <cboltz> recode latin1..utf8 onepost.sql   to actually make it utf-8
2022-06-27T19:59:36  <cboltz> onepost.sql doesn't include a charset hint, which means the default gets used - guess what this is nowadays ;-)
2022-06-27T20:01:34  <cboltz> (oh, and vi displays [converted] again - which is useful and makes it readable, but you don't want this magic if you are looking for charset problems)
2022-06-27T20:02:41  <cboltz> for importing, try   ( echo '/*!40101 SET NAMES latin1 */; ' ; cat onepost.sql ) | mysql ......
2022-06-27T20:02:59  <SasiOlin[m]> it doesn't really matter if it gets utf8 content over a pipe I would assume
2022-06-27T20:03:25  <SasiOlin[m]> I should test
2022-06-27T20:03:48  <cboltz> the pipe doesn't change the content, the trick is the "SET NAMES" part to tell mysql about the encoding
2022-06-27T20:04:11  <SasiOlin[m]> right
2022-06-27T22:09:25  *** teepee_ is now known as teepee
2022-06-27T22:29:27  <mikeru> hey there, I'd like to volunteer into fixing the #support:opensuse.org  <--> irc.libera.chat/#opensuse bridge