2021-05-06T08:11:33  <pjessen> i guess the switch upgrade went fine - I got a "whoops" page, but not for very long.
2021-05-06T13:37:13  <crashmaster18> Hi Heros. Hope everyone is safe/healthy.  I have a mirror brain question...
2021-05-06T13:37:32  <crashmaster18> For past several days I've been having troubles updating a system from leap 15.1 to 2.
2021-05-06T13:37:37  <crashmaster18> zypper.log shows;
2021-05-06T13:37:39  <crashmaster18> 2021-05-06 08:10:07 <5> nagios(24497) [zypp] Exception.cc(log):166 Error message: The requested URL returned error: 416 Requested range not satisfiable
2021-05-06T13:37:40  <crashmaster18> 2021-05-06 08:10:07 <5> nagios(24497) [zypp] Exception.cc(log):166 MediaMultiCurl.cc(doGetFileCopy):1448 RETHROW:  Download (curl) error for 'http://mirrors.rit.edu/opensuse/update/leap/15.1/oss/x86_64/apache2-mod_php7-7.2.5-lp151.6.36.7_lp151.6.39.1.x86_64.drpm':
2021-05-06T13:38:28  <crashmaster18> Of course, I can manually download the file.
2021-05-06T13:38:51  <crashmaster18> But this happens for several packages - I cancel out...
2021-05-06T13:41:34  <crashmaster18> I had a similar problem on another leap 15.1 system, same local mirror issues. That one I just nuked and paved with a fresh 15.2 iso. This one I would like to try and troubleshoot the issue, if possible...
2021-05-06T13:41:42  <crashmaster18> I was entering the issue here, because I was wondering what validation mirrorbrain has for mirror content and if that validation is reflected on the status.opensuse.org page?
2021-05-06T13:47:18  <crashmaster18> I'm wondering if this (415 requested range not satisfiable) is related to resume options in curl. Maybe some mirrors aren't fully supporting it?
2021-05-06T13:47:50  <crashmaster18> correction: 416 Requested Range ...
2021-05-06T15:30:37  <pjessen> crashmaster18: requested range is a response to a partial get
2021-05-06T15:41:58  * Lars[m]1 < https://matrix.org/_matrix/media/r0/download/matrix.org/SzgJxMqSGNVtcBHkiQzgZEAE/message.txt >
2021-05-06T15:49:08  <pjessen>  that's very cryptic :-)
2021-05-06T15:52:48  <Lars[m]1> ?
2021-05-06T15:53:01  <pjessen> I'm pretty certain ranged requests are standard, I think you would need to somehow disable it, intentionally
2021-05-06T15:53:16  <pjessen> Lars[m]1: I only got a URL, no text
2021-05-06T15:53:56  <Lars[m]1> Yes. That's why we don't hear problems with this much often.
2021-05-06T15:54:07  <Lars[m]1> But we don't test for it.
2021-05-06T15:54:10  <pjessen> true
2021-05-06T15:54:56  <pjessen> but zypper tried to segmented downloads by default afair.  256K segments spread over multiple mirrors.
2021-05-06T15:55:00  <pjessen> tries
2021-05-06T15:55:24  <Lars[m]1> If you just get URL, I'm sorry. I'm currently traveling and use Matrix during that time. Here, everything looks ok. Maybe something for lcp to look after?
2021-05-06T15:56:01  <pjessen> now I'm getting you just fine, it was only that first reply
2021-05-06T15:56:42  <pjessen> funny - on the URL posted by crashmaster18, it is a 15.1 drpm ?  for an update to 15.2 that doesn't sound right
2021-05-06T15:57:11  <Lars[m]1> I just wanted to point out that it might be better to test with mirrorcache . This project is so "young" that I hope we get fixes in, if needed. I have no real how for mirrorbrain, I have to admit.
2021-05-06T15:57:53  <Lars[m]1> s/no real how/bad feeling/ - sorry
2021-05-06T15:58:38  <pjessen> I'm not so keen on mirrorcache, I would have hoped the effort could have gone into mirrorbrain instead, mirrorcache is a patchwork.
2021-05-06T15:59:16  <Lars[m]1> Feel free to improve mirrorbrain :-)
2021-05-06T15:59:28  <pjessen> Haha, I know, I know :-)
2021-05-06T15:59:50  <Lars[m]1> Mirrorbrain is C++, Python, Perl and Bash
2021-05-06T16:00:11  <Lars[m]1> MirrorCache is Perl and little Bash
2021-05-06T16:00:33  <Lars[m]1> So I see MirrorBrain as patchwork ;-)
2021-05-06T16:01:17  <Lars[m]1> But at the end, only results matter
2021-05-06T16:01:27  <pjessen> well, I meant "patchwork" as patching some minor flaws in mirrorbrain, but not really improving on the overall.
2021-05-06T16:02:34  <pjessen> results are okay, but ease of maintainence is what we really need. some more long term thinking.
2021-05-06T16:03:23  <Lars[m]1> I'm sure we could get MirrorBrain up to date as well - there's just nobody left who likes to spend his time. So I take what I can get... :-)
2021-05-06T16:05:30  <Lars[m]1> ....and if we can add our needs and feature requests as early as possible to mirrorcache and get them implemented....
2021-05-06T16:06:21  <Lars[m]1> ...we might get a nice piece of software for the job
2021-05-06T16:07:49  <pjessen> Lars[m]1: maybe - I would just worry about the future maintenance.  With mirrorcache we get more complexity, but mostly the same function.
2021-05-06T16:08:01  <Lars[m]1> And in 15 years, when all initial requests are implemented and we start with patchwork again, it's time for the next project
2021-05-06T16:08:31  <Lars[m]1> Which complexity?
2021-05-06T16:08:59  <Lars[m]1> Using a webui instead of command-line tools and self-written scripts?
2021-05-06T16:09:36  <pjessen> mirrorbrain + mirrorcache.  afaiu, mirrorache does not replace mirrorbrain, only add better support for  ipv6 and https
2021-05-06T16:19:16  <Lars[m]1> -> talk with Andrii about this, I suggest. I see it as replacement.
2021-05-06T16:22:59  <pjessen> oh, I have discussed it with andrii, a couple of times. unless I am completely wrong, mirrorcache does not work without mirrorbrain.
2021-05-06T16:35:11  <pjessen> otoh, if mirrorcache replaces mirrorbrain, I'm totally for it.
2021-05-06T17:38:57  <crashmaster18> I was disconnected from webchat and didn't notice so I'm re-sending my last just in case it failed: Ah, and if mirrorcache does not work without mirrorbrain, there's little benefit in going through the work of adding it. @pjessen Yes, I'm sorry, this local mirror had dozens of those messages in the zypper log, I pulled an old one.  I'm also aware
2021-05-06T17:38:58  <crashmaster18> it's a partial get, I'm wondering if there's a way to test for curl "resume" which I think is the root cause. If I force the mirror to Provo or Germany, of course, there are no issues at all....
2021-05-06T17:40:54  <crashmaster18> It's a very active project though (MirrorCache) https://github.com/openSUSE/MirrorCache
2021-05-06T17:41:28  <openSUSEBot> <LCP> yeah, mirrorbrain had a lot of commits when it started out too
2021-05-06T17:41:39  <openSUSEBot> <LCP> we all know what happened after a few years 😉
2021-05-06T17:44:14  <crashmaster18> lcp I know - but what's the alternative? Perhaps mirrorcache could give us time to come up with a better plan...
2021-05-06T17:45:35  <openSUSEBot> <LCP> I mean, I voiced my opinion on merging the effort with mirrormanager, which was way better maintained than mirrorbrain and codebase wasn't as much a mess, but there doesn't seem to be any interest either, so I guess we will just keep rewriting things until the end of times
2021-05-06T17:47:02  <crashmaster18> lcp Well, mirrormanager wasn't "invented here" - so of course we can't use it. ;)
2021-05-06T17:48:20  <lcp> it seems that there is an unreasonable amount of xenophobia, the fear of unknown, at SUSE yeah :P
2021-05-06T17:48:48  <lcp> I guess that will keep us stuck in the past for a while longer
2021-05-06T17:50:04  <cboltz> NIH isn't a real problem for the openSUSE infrastructure - hey, we even run some things on a Fedora VM ;-)
2021-05-06T17:52:05  <lcp> that may as well be a relic from the times when that xenophobia wasn't that obvious either, since that was set up by SUSE too :P
2021-05-06T18:00:15  <crashmaster18> lcp Do you know if SUSE is using mirrorbrain for SLE? If so, that might explain some similar issues we see there from time to time.  -- As a user of opensuse, I don't care what we use so long as we can improve things. It does seem like the past few years in the northeast usa that mirror performance is decreasing. I don't know if that's because of
2021-05-06T18:00:16  <crashmaster18> a general decrease in the number of mirrors, overwhelmed mirrors, or whatever. It has gotten to the point where I think I need to contribute a mirror on a reasonably priced hosting provider. ;) -- Whatever we choose, I think we need to make sure we can instrument things so that admins/hereos can see issues on the mirrors, if not on
2021-05-06T18:00:16  <crashmaster18> status.opensuse.org somwhere.
2021-05-06T18:01:37  <lcp> few NA mirrors closed because they were getting no traffic though, clearly there is no problem in that case /s
2021-05-06T18:02:40  <lcp> yeah, I don't understand why that's happening exactly either, that would require way more research, but clearly we need to improve things
2021-05-06T18:08:40  <pjessen> I'm a little late to this, sorry.
2021-05-06T18:10:01  <pjessen> I don't NIH is a problem at all. however, all we have is a slightly aging mirroring infrastructure with a few minor issues.
2021-05-06T18:10:51  <cboltz> one problem with download.o.o is that it is (only) in Nuremberg, so if you use download.o.o, each request first goes to germany (which takes some time), and then you get (hopefully) redirected to an US mirror
2021-05-06T18:11:59  <pjessen> I agree, but that has been a problem since the very beginning, and the internet connectivity has only improved, by an order of magnitude at least.
2021-05-06T18:12:43  <pjessen> IF that is a problem, the proper way to solve it is with multiple mirrorbrains on anycast DNS.
2021-05-06T18:16:05  <pjessen> Surely we are not the only organisation with an issue like this?  how does Fedora deal with e.g. Brazilian or Argentinian users?
2021-05-06T18:16:36  <crashmaster18> mirrormanager I believe...
2021-05-06T18:16:55  <pjessen> is mirrormanager geographically distributed?
2021-05-06T18:17:16  <crashmaster18> I suspect you know more about it than I do. lol. lcp just turned me on to it.
2021-05-06T18:18:01  <pjessen> crashmaster18: nah, I don't know much - it is just the same problem usually requires the same solution .
2021-05-06T18:18:21  <lcp> no, there is a single primary mirrormanager server just like we have it
2021-05-06T18:18:31  <lcp> and of course it redirects to other mirrors
2021-05-06T18:19:15  <pjessen> lcp: that is what I thought too. so from the user perspective, it's no different.
2021-05-06T18:19:21  <crashmaster18> pjessen it seems we need a better way to measure performance of the mirrors, even if it's periodically. Perhaps with openqa we could test them monthly or something with latest leap and tw. idk....
2021-05-06T18:19:57  <lcp> openqa is in nbg so it's gonna seem perfectly fine >:D
2021-05-06T18:20:15  <lcp> we could use github actions to do those tests for example though, since that's us based
2021-05-06T18:21:12  <pjessen> crashmaster18: "performance" of a mirror is a moving target - that is, it depends on from where.  downloading openSUSE in the Falklands _will_ be slower than from Zurich.
2021-05-06T18:23:17  <crashmaster18> Oh I'm not so worried about "slow". I'm more worried about missing/corrupted packages or broken downloads. I just tested a fresh vm and am having the exact same issue with the same mirror. Switch to Germany, all is fine.
2021-05-06T18:23:38  <pjessen> I would prefer to see someone actually outline which problems we really have with the mirroring infrastructure, apart from our lack of support for IPv6 and https.
2021-05-06T18:24:34  <pjessen> crashmaster18: for Leap, that is something to report as an issue.  MIrrors are all scanned continuously, you should never be directed to a broken mirror.
2021-05-06T18:24:35  <crashmaster18> It has happened to us at work here 17 times since January 1. The majority of our downloads are through uyuni, then we have some systems outside uyuni where it happens. https would be nice......very nice.....
2021-05-06T18:26:33  <crashmaster18> Right, but are they using the same method as zypper is to pull partial updates? I don't think the scans are doing that. So they look fine, but then sometimes zypper breaks when the curl call isn't supported for partial downloads. I really hate to complain about these things. I know it's an open source project. I think I need to setup that mirror
2021-05-06T18:26:33  <crashmaster18> later this year and see for myself if that just solves the issue....
2021-05-06T18:28:03  <pjessen> true, the scans just check for availability, but a mirror would have to explicitly disable a ranged download, it is so odd as to be very unlikely.
2021-05-06T18:29:10  <pjessen> crashmaster18: hey, complaining is absolutely fine - we just need the information to be able to do something.
2021-05-06T18:31:18  <crashmaster18> lcp yes, but I'm in IT for a long time now. Complaining doesn't help if we can't figure out the details of the actual problem(s). I just see mirror reliability becoming an issue where I am  and I am happy to contribute help (or to test alternates to mirrorbrain should we go in that direction!) If the alternatives will allow us to use https....we
2021-05-06T18:31:18  <crashmaster18> need to do this IMO. :)
2021-05-06T18:31:48  <crashmaster18> pjessen oops I @ the wrong person
2021-05-06T18:32:22  <crashmaster18> In anycase, I monitor the mailing lists, if mirrors come up for trying alternatives, I'll volunteer...
2021-05-06T18:37:20  <pjessen> crashmaster18: basically, there are two possible approaches - if one mirror is misbehaving (and it does happen), you open a ticket. if you feel you are overall seeing a poor service, you open a ticket (if nothing else to vent).
2021-05-06T18:38:33  <pjessen> we cant always do much about them - if it is one mirror, we can, but improving mirroring availability in general is impossible.
2021-05-06T18:42:19  <crashmaster18> pjessen Understood. If a new mirror software stack helps us automate out bad apples easier than mirrorbrain, maybe we should look at it. We automate nearly all of our updates. Seems like it would be a cool feature "opt in" for zypper / libzypp to report possible mirror problems to status.opensuse.org.  :)
2021-05-06T20:24:36  <bmwiedemann> cboltz: I found that gitlab.i.o.o lacks the 2nd intermediate crt "Heroes internal CA Root CA" in /etc/nginx/ssl/*chain.pem
2021-05-06T20:25:19  <bmwiedemann> could also add the trust root just in case
2021-05-06T20:31:55  <cboltz> nice find - I guess you have enough permissions to fix it? ;-)
2021-05-06T21:53:47  <Lars[m]1> About mirrormanager: IMHO they only check the existence of repomd.xml to enable redirecting. This makes the scans way faster. But Fedora has no OBS - and not ~20TB data in it.
2021-05-06T21:54:58  <Lars[m]1> SUSE itself uses a CDN (Verizon), who have own servers around the world in each country and a Geo based DNS.
2021-05-06T21:56:29  <Lars[m]1> Issues with MirrorBrain are tracked in Github.
2021-05-06T21:56:33  <Lars[m]1> https://github.com/poeml/mirrorbrain/issues
2021-05-06T21:57:07  <Lars[m]1> As upstream is not reacting any more (since years), we forked the repo.
2021-05-06T21:57:32  <Lars[m]1> https://github.com/openSUSE/mirrorbrain/issues
2021-05-06T21:58:05  <Lars[m]1> But you might notice, that there is not much movement in any of the repos.
2021-05-06T22:00:58  <Lars[m]1> NIH is IMHO a bit unfair: MirrorBrain was the first open source redirector, developed at SUSE. MirrorManager and others came up independent of it. All are specialized for their environment. Other organizations (like LibreOffice or KDE) used MirrorBrain (at least in the past) as well, because of all the features and the speed.
2021-05-06T22:03:11  <Lars[m]1> MirrorCache (which *is* independent from MirrorBrain, it currently is just integrated as "normal mirror" in MirrorBrain for testing) should solve especially the following issues:
2021-05-06T22:03:48  <Lars[m]1> * https
2021-05-06T22:03:48  <Lars[m]1> * IPv6
2021-05-06T22:03:48  <Lars[m]1> * No local storage
2021-05-06T22:04:08  <Lars[m]1> * Geo based distribution
2021-05-06T22:05:14  <Lars[m]1> http and https redirects are long standing issues in MirrorBrain. Not unsolvable, but nobody works on it.
2021-05-06T22:05:25  <Lars[m]1> Same for IPv6
2021-05-06T22:07:38  <openSUSEBot> <LCP> https and ipv6 seem like things that need to be primarily fixed in the master redirector and not the clients
2021-05-06T22:07:59  <Lars[m]1> No local storage is kind of possible with MirrorBrain as well, but it requires zero byte files ( "touch foo.rpm") on the master. IMHO not the right approach. MirrorCache (imho like mirrormanager) is database driven: you define one or more master mirrors and these are scanned.
2021-05-06T22:09:14  <Lars[m]1> Geo based distribution of the redirector is also possible with MirrorBrain, but - because of the local storage problem - was never followed up at openSUSE.
2021-05-06T22:09:56  <Lars[m]1> In the end: it stands and falls with the manpower.
2021-05-06T22:10:36  <cboltz> isn't that last line true for all projects?
2021-05-06T22:10:38  <Lars[m]1> If there are issues affecting production, you start searching for someone to fix them.
2021-05-06T22:10:46  <cboltz> (and thanks for all the details!)
2021-05-06T22:11:00  <Lars[m]1> If nobody is fixing, you start looking for alternatives...
2021-05-06T22:11:56  <Lars[m]1> Yes, to both: ipv6 and https needs "just" to be fixed on the server side.
2021-05-06T22:12:18  <Lars[m]1> And that's true for all projects
2021-05-06T22:12:53  <Lars[m]1> Think about the problem with the redmine plugins for example
2021-05-06T22:13:18  <Lars[m]1> All developed to solve single problems by single persons.
2021-05-06T22:13:38  <Lars[m]1> Now they make problems. 10 years later.
2021-05-06T22:14:48  <Lars[m]1> Even Microsoft learned that it's hard work to keep something supported for such a long time... And SUSE (and RH) earn good money by proving such long term support contracts.
2021-05-06T22:15:48  <Lars[m]1> What I start to learn from this: as admin of a community project, you need to learn to "let go".
2021-05-06T22:16:13  <Lars[m]1> Something that I (at least) still need to learn.
2021-05-06T22:16:18  <openSUSEBot> <LCP> huh, I wonder who wanted openSUSE to use solutions that are used by more than one project and got repeatedly rejected because it came from fedora >:D
2021-05-06T22:16:34  <Lars[m]1> community.o.o anyone? ;-)
2021-05-06T22:16:59  <openSUSEBot> <LCP> also related to community.i.o.o ;)
2021-05-06T22:17:59  <openSUSEBot> <LCP> actually, we should be able to drop community.i.o.o, but nobody answered me how we would want to actually deploy it and I don't trust myself alone to decide
2021-05-06T22:18:16  <Lars[m]1> Hm: at least I can not remember that I rejected a solution because it came from any other project. With this, we should shut down our apache, nginx, haproxy, ... stuff immediately
2021-05-06T22:18:20  <openSUSEBot> <LCP> (that being doc.o.o main page that's done with jekyll)
2021-05-06T22:18:59  <cboltz> sadly some people still see other distros as "enemies" - which they clearly are not
2021-05-06T22:19:17  <Lars[m]1> LCP: welcome in the openSUSE universe: those, who do, decide...
2021-05-06T22:20:24  <Lars[m]1> I worked with all the Education centric distributions for a long time. None of them felt the "others" are enemies.
2021-05-06T22:21:03  <Lars[m]1> I think that especially open source allows collaboration.
2021-05-06T22:21:49  <Lars[m]1> It allows to "steal" good ideas - while on the other side allows individuality.
2021-05-06T22:22:07  <cboltz> that's also the impression I have from lots of people - and still, you find a few that didn't understand that and still think of "we vs. them"
2021-05-06T22:22:38  <Lars[m]1> Yeah, this "we vs them" is human nature.
2021-05-06T22:23:34  <openSUSEBot> <LCP> I mean, your response on the list to the idea of using noggin sounded incredibly negative, or it's how I perceived it
2021-05-06T22:23:41  <Lars[m]1> As long as it motivates people to get better, it's ok.
2021-05-06T22:24:06  <Lars[m]1> But if people start to see others as enemies, it's the wrong way
2021-05-06T22:24:52  <Lars[m]1> LCP sorry if it sounded that way. That was not my intention.
2021-05-06T22:25:21  <Lars[m]1> I thing we should choose the right tool for a job.
2021-05-06T22:26:23  <Lars[m]1> History told me that we should choose important tools wisely, as they tend to stay looooong with us.
2021-05-06T22:27:46  <Lars[m]1> So I'm often playing advocatus diaboly, to stress suggestions - trying to see if we covered all areas.
2021-05-06T22:28:32  <cboltz> hey, I thought that's my job ;-)
2021-05-06T22:30:34  <openSUSEBot> <LCP> we need to schedule replacement of the old freeipa with freeipa2 on that matter
2021-05-06T22:30:49  <openSUSEBot> <LCP> it can go horribly wrong >:D
2021-05-06T22:31:14  <openSUSEBot> <LCP> and for redundancy I hope we can do 2 freeipas after we replace the old one
2021-05-06T22:32:04  <openSUSEBot> <LCP> (I wish we had a second location for it, but alas we don't really have that option afaik)
2021-05-06T22:32:25  <Lars[m]1> Well: we already separated DNS. What's left is LDAP...
2021-05-06T22:32:48  <openSUSEBot> <LCP> indeed
2021-05-06T22:32:50  <Lars[m]1> and regarding other locations....
2021-05-06T22:33:03  <Lars[m]1> We have Nuremberg and Provo
2021-05-06T22:33:21  <openSUSEBot> <LCP> do we have the ability to deploy vms in provo?
2021-05-06T22:33:23  <cboltz> wait - AFAIK some internal zones like infra.o.o still live in FreeIPA, and are not served by chip [anymore]
2021-05-06T22:33:32  <Lars[m]1> And we have a 2nd, independent DC in Nuremberg
2021-05-06T22:34:04  <cboltz> IMHO we should move _all_ zones to chip
2021-05-06T22:35:01  <Lars[m]1> No objections from my side. The zones are already on chip.
2021-05-06T22:35:28  <Lars[m]1> All what's left to do is to tell chip that he is also master for infra.o.o
2021-05-06T22:36:00  <Lars[m]1> And after that, you might want to adjust anna/elsa
2021-05-06T22:36:05  <cboltz> last time I checked (which was admittedly some weeks ago) chip refused to answer queries for *.infra.o.o
2021-05-06T22:36:30  <cboltz> which is also why anna/elsa now only query FreeIPA for *.infra.o.o
2021-05-06T22:36:55  <cboltz> (also trying chip resulted in some funny[tm] issues)
2021-05-06T22:38:49  <Lars[m]1> it's "just a setting" - as usual ;-)
2021-05-06T22:39:09  <cboltz> yeah, if you know which setting ;-)
2021-05-06T22:39:14  <Lars[m]1> I think we should rethink the whole setup anyway
2021-05-06T22:39:30  <cboltz> so far, I only know how to edit a zone in powerdns, but not more
2021-05-06T22:39:51  <openSUSEBot> <LCP> how much control do we have in provo actually?
2021-05-06T22:40:09  <Lars[m]1> That's (beside time) what stopped me from finishing that "simple" DNS thing
2021-05-06T22:40:13  <openSUSEBot> <LCP> could we set up a freeipa there for the sake of having a backup?
2021-05-06T22:40:27  <Lars[m]1> We have the same control as in Nuremberg
2021-05-06T22:40:35  <openSUSEBot> <LCP> oh, nice
2021-05-06T22:40:37  <Lars[m]1> Of course
2021-05-06T22:40:50  <openSUSEBot> <LCP> I guess I would like that then, seems like the optimal way to do this
2021-05-06T22:41:15  <openSUSEBot> <LCP> I just need to finish the migration of the current one
2021-05-06T22:41:26  <Lars[m]1> Like we have a narwal, a mirrordb, a haproxy, a DNS there already
2021-05-06T22:41:32  <openSUSEBot> <LCP> which means switching over, which does make me a touch nervous
2021-05-06T22:41:48  <Lars[m]1> I also started with one MariaDB node already
2021-05-06T22:42:18  <openSUSEBot> <LCP> that seems like a nice redundancy setup >:D
2021-05-06T22:42:28  <Lars[m]1> LCP: we could also first simply setup a plain 389ds
2021-05-06T22:42:30  <openSUSEBot> <LCP> as far from nbg as it gets
2021-05-06T22:42:43  <openSUSEBot> <LCP> indeed we could
2021-05-06T22:43:05  <Lars[m]1> And jdsn prepared a sponsored new machine for the other DC in Nuremberg
2021-05-06T22:43:33  <openSUSEBot> <LCP> but I we would miss out on being able to do openSUSE accounts at all
2021-05-06T22:43:39  <Lars[m]1> Which has ~80-90TB disk capacity and 256G RAM
2021-05-06T22:43:45  <openSUSEBot> <LCP> and I will fight for it until the day I die >:D
2021-05-06T22:44:48  <Lars[m]1> I think we should focus on bugzilla and OBS first, so they support more than one auth systems
2021-05-06T22:44:50  <openSUSEBot> <LCP> mainly because I can tell SUSE IT has done their system with 0 idea what community means
2021-05-06T22:45:05  <openSUSEBot> <LCP> Bugzilla is done though
2021-05-06T22:45:23  <Lars[m]1> Once this is done, the separation is not that far.
2021-05-06T22:45:30  <openSUSEBot> <LCP> except for the part where we were unable to get in touch with SUSE Bugzilla maintainers to actually implement it there
2021-05-06T22:45:59  <openSUSEBot> <LCP> which is frustrating
2021-05-06T22:46:37  <Lars[m]1> If SUSE can still maintain "their" user data and agree to support another with provider for the two systems, you won :-)
2021-05-06T22:46:38  <openSUSEBot> <LCP> we do actually really want to do this, but clearly cooperation is not a word in IT's dictionary ;)
2021-05-06T22:47:14  <Lars[m]1> s/with provider/auth provider/
2021-05-06T22:47:52  <Lars[m]1> Well, IMHO we all know some of the current bugzilla maintainer very well.
2021-05-06T22:48:10  <openSUSEBot> <LCP> I guess bmwiedermann?
2021-05-06T22:48:22  <Lars[m]1> I think, this topic is worth to get board attention.
2021-05-06T22:48:33  <openSUSEBot> <LCP> hm?
2021-05-06T22:48:39  <Lars[m]1> lcp: jip :-)
2021-05-06T22:49:47  <openSUSEBot> <LCP> ehhhhhh, I'm not happy to work on OBS side of the deal because it's very complex and has to change some workflows to actually work at all and I can guarantee you people won't be too happy
2021-05-06T22:50:00  <openSUSEBot> <LCP> it's fun >:D
2021-05-06T22:50:10  <Lars[m]1> I think that we will easily get help from Bernhard. That's not the problem.
2021-05-06T22:50:27  <Lars[m]1> But he needs approval from his manager.
2021-05-06T22:50:41  <openSUSEBot> <LCP> I'm very familiar with how to do it in obs now, but it's a bunch of work
2021-05-06T22:50:48  <openSUSEBot> <LCP> nice
2021-05-06T22:50:59  <openSUSEBot> <LCP> managers, love the sound of that >:D
2021-05-06T22:51:41  <Lars[m]1> To get this, I suggest to ask the board and - via gp - the upper SUSE management for collaboration support :-)
2021-05-06T22:52:12  <Lars[m]1> They claim that they love openSUSE.
2021-05-06T22:52:44  <Lars[m]1> Let's ask them what this means to them by asking for a dedicated community account system...
2021-05-06T22:52:47  <openSUSEBot> <LCP> after being on the board for 3ish months I have my doubts
2021-05-06T22:53:15  <openSUSEBot> <LCP> wrt that love >:D
2021-05-06T22:53:40  <Lars[m]1> I think if you (or gp) explains it in "manager speach" ( No technical stuff, please ;-) they will listen and agree.
2021-05-06T22:54:28  <Lars[m]1> We already got the opensuse.org DNS
2021-05-06T22:54:29  <cboltz> you mean like they made SLE bugs public whenever possible? *g,d&r*
2021-05-06T22:54:53  <openSUSEBot> <LCP> I think we can add some legal stuff to scare them into it too >:D
2021-05-06T22:54:55  <Lars[m]1> We will hopefully get DNSSec (hint: registrar access) soon
2021-05-06T22:55:15  <cboltz> :-)
2021-05-06T22:55:43  <Lars[m]1> It's just a matter of being friendly, pedantic and persistant
2021-05-06T22:56:05  <openSUSEBot> <LCP> yeah, I'm only lacking the friendly bit
2021-05-06T22:56:30  <Lars[m]1> Yeah, me.too :-)
2021-05-06T22:56:45  <Lars[m]1> That's why I bring gp into the game
2021-05-06T22:56:47  <cboltz> count me in ;-)
2021-05-06T22:58:26  <Lars[m]1> Lcp: as you are on the board, can I leave that task of convincing Gerhard that bugzilla and obs should support more than just one auth provider to you? :-)
2021-05-06T22:59:13  <openSUSEBot> <LCP> oh, I'm no longer on the board since like 4 months
2021-05-06T22:59:22  <openSUSEBot> <LCP> I resigned after 3ish months >:D
2021-05-06T22:59:47  <openSUSEBot> <LCP> but I do have ways to relay this property
2021-05-06T22:59:51  <openSUSEBot> <LCP> properly*
2021-05-06T22:59:59  <Lars[m]1> ups? Sorry to hear (and not noticing)
2021-05-06T23:00:21  <Lars[m]1> At least you had my vote :-)
2021-05-06T23:01:01  <openSUSEBot> <LCP> thank you, I just need to get my life in order before I can commit further ;)
2021-05-06T23:01:31  <Lars[m]1> I have to admit that the technical part on OBS is not that easy as well. But technical problems can be solved.
2021-05-06T23:02:22  <Lars[m]1> We have already tickets open for this - but at the moment, there are other priorities...
2021-05-06T23:02:23  <openSUSEBot> <LCP> yeah, I do have a solid idea how to do that well, but it's both a backend and a frontend job to do that
2021-05-06T23:02:56  <Lars[m]1> but If there would be a  request from upper management ....
2021-05-06T23:02:57  <openSUSEBot> <LCP> also it's not really doing it my preferred way but any way will have to do ;)
2021-05-06T23:03:14  <openSUSEBot> <LCP> fair enough
2021-05-06T23:04:14  <Lars[m]1> That's why I would love to see this addressed via the board. It gives other opportunities, if we convinced SUSE management.
2021-05-06T23:06:16  <Lars[m]1> And implementation in obs and bugzilla is currently more important for me than the final auth solution.
2021-05-06T23:07:33  <Lars[m]1> Just think about enabling Multi Factor Authentication for openSUSE accounts or the auth-tokens
2021-05-06T23:09:04  <openSUSEBot> <LCP> noggin has support for 2fa which is great
2021-05-06T23:09:34  <Lars[m]1> Finally: finding a tool around LDAP and kerberos, that supports openID, saml and all the other nice gimmicks can be run in parallel
2021-05-06T23:10:13  <openSUSEBot> <LCP> so ipsilon? :P
2021-05-06T23:10:16  <Lars[m]1> ...and if noggin runs only on Fedora for now - who cares.
2021-05-06T23:10:41  <openSUSEBot> <LCP> uh, pretty much because openSUSE python packaging is really in poor shape
2021-05-06T23:10:46  <Lars[m]1> I just want an open source tool. So we can adjust it on our own, if needed.
2021-05-06T23:10:50  <openSUSEBot> <LCP> but you could have guessed that
2021-05-06T23:11:04  <Lars[m]1> Jip.
2021-05-06T23:11:50  <Lars[m]1> And yes: starting to collaborate with Fedora was not ment as bad joke or something like this
2021-05-06T23:12:34  <openSUSEBot> <LCP> fair enough
2021-05-06T23:12:45  <Lars[m]1> I have a little fear that "one community account" might be too interesting for hackers
2021-05-06T23:13:05  <openSUSEBot> <LCP> (I was planning to integrate Release-Monitoring with obs for that matter)
2021-05-06T23:13:08  <Lars[m]1> But in the other side, there would also be many nice benefits
2021-05-06T23:13:51  <openSUSEBot> <LCP> well, with pagure that doesn't matter  since it will have federation support in the future so you will be able to contribute to code on any pagure instance from your preferred pagure instance
2021-05-06T23:14:07  <Lars[m]1> That's something you should explain more in detail
2021-05-06T23:14:08  <openSUSEBot> <LCP> (that's currently being tested)
2021-05-06T23:14:31  <Lars[m]1> But maybe via email or during oSC.
2021-05-06T23:14:38  <openSUSEBot> <LCP> indeed
2021-05-06T23:14:41  <Lars[m]1> I'm currently too tired...
2021-05-06T23:15:04  <openSUSEBot> <LCP> it's pretty late, ngl
2021-05-06T23:15:30  <Lars[m]1> ...and I'm the other side, I can note down what we currently have as opportunities in our infrastructure.
2021-05-06T23:16:20  <Lars[m]1> And we can sit down in a nice bar with a beer and update us on all the topics...
2021-05-06T23:16:36  <openSUSEBot> <LCP> we need to get rid of the old cruft as the priority tbh
2021-05-06T23:16:52  <openSUSEBot> <LCP> the rest is just nice additions :P
2021-05-06T23:16:56  <Lars[m]1> Until the morning starts and we need to go for breakfast before we rejoin the hackaton at oSC
2021-05-06T23:17:26  <Lars[m]1>  /me is dreaming already ... :-)
2021-05-06T23:17:39  <cboltz> I like that dream ;-)
2021-05-06T23:17:47  <cboltz> well, except s/beer/wine/ please
2021-05-06T23:18:03  <Lars[m]1> I was waiting for.exactly.that.
2021-05-06T23:18:05  <openSUSEBot> <LCP> I will take mint tea, or any tea without caffeine
2021-05-06T23:18:15  <cboltz> ;-)
2021-05-06T23:19:07  <Lars[m]1> I will leave you alone now - following up with my dream... Good 🌙 CU!
2021-05-06T23:19:10  <openSUSEBot> <LCP> no alcohol for me :P
2021-05-06T23:19:29  <cboltz> good night!
2021-05-06T23:19:59  <openSUSEBot> <LCP> night!