2017-11-07T00:14:53 *** Son_Goku has quit IRC 2017-11-07T00:36:18 *** Son_Goku has joined #opensuse-admin 2017-11-07T00:36:26 *** okurz[m] has joined #opensuse-admin 2017-11-07T00:50:20 *** cboltz has quit IRC 2017-11-07T00:53:05 *** Son_Goku has quit IRC 2017-11-07T01:10:08 *** Son_Goku has joined #opensuse-admin 2017-11-07T01:19:01 *** Son_Goku has quit IRC 2017-11-07T01:31:52 *** Son_Goku has joined #opensuse-admin 2017-11-07T02:04:26 *** Son_Goku has quit IRC 2017-11-07T02:20:43 *** Son_Goku has joined #opensuse-admin 2017-11-07T02:31:00 *** Son_Goku has quit IRC 2017-11-07T02:31:14 *** plinnell has quit IRC 2017-11-07T03:04:21 *** okurz has quit IRC 2017-11-07T03:05:09 *** okurz has joined #opensuse-admin 2017-11-07T04:04:19 *** okurz has quit IRC 2017-11-07T04:04:46 *** okurz_ has joined #opensuse-admin 2017-11-07T04:04:53 *** okurz_ is now known as okurz 2017-11-07T04:23:44 *** nicolasbock has quit IRC 2017-11-07T05:05:07 *** plinnell has joined #opensuse-admin 2017-11-07T05:05:07 *** plinnell has joined #opensuse-admin 2017-11-07T05:39:04 *** tampakrap has quit IRC 2017-11-07T05:39:08 *** skiarxon has quit IRC 2017-11-07T05:39:28 *** differentreality has quit IRC 2017-11-07T06:43:15 *** maxlin has joined #opensuse-admin 2017-11-07T06:44:07 hi, https://progress.opensuse.org/ is unreachable , please somebody can take a look? 2017-11-07T07:00:45 *** Son_Goku has joined #opensuse-admin 2017-11-07T07:12:53 *** asmorodskyi has joined #opensuse-admin 2017-11-07T07:16:04 *** Son_Goku has quit IRC 2017-11-07T07:21:26 *** asmorodskyi_ has joined #opensuse-admin 2017-11-07T07:21:52 *** asmorodskyi has quit IRC 2017-11-07T07:26:12 *** tigerfoot has quit IRC 2017-11-07T07:34:11 *** mgriessmeier has joined #opensuse-admin 2017-11-07T07:40:46 *** malcolmlewis has quit IRC 2017-11-07T07:46:21 *** asmorodskyi_ has quit IRC 2017-11-07T07:54:26 *** malcolmlewis has joined #opensuse-admin 2017-11-07T08:15:33 *** matthias_bgg has joined #opensuse-admin 2017-11-07T08:17:01 *** kl_eisbaer has joined #opensuse-admin 2017-11-07T08:18:19 *** tigerfoot has joined #opensuse-admin 2017-11-07T08:19:12 *** asmorodskyi_ has joined #opensuse-admin 2017-11-07T08:44:16 *** mcaj has joined #opensuse-admin 2017-11-07T09:14:11 *** ancorgs has joined #opensuse-admin 2017-11-07T09:25:41 *** matthias_bgg has quit IRC 2017-11-07T09:34:27 *** matthias_bgg has joined #opensuse-admin 2017-11-07T09:58:36 *** Son_Goku has joined #opensuse-admin 2017-11-07T10:02:44 *** tampakrap has joined #opensuse-admin 2017-11-07T10:02:47 *** differentreality has joined #opensuse-admin 2017-11-07T10:02:47 *** differentreality has joined #opensuse-admin 2017-11-07T10:02:52 *** skiarxon has joined #opensuse-admin 2017-11-07T10:07:01 *** Son_Goku has quit IRC 2017-11-07T10:29:11 tampakrap: ping 2017-11-07T10:29:22 tampakrap: can you create a VPN setup for jdsn, please? 2017-11-07T10:29:44 tampakrap: I would do it on my own, but I don't have the secret for the CA that you are using ;-) 2017-11-07T10:29:52 sure 2017-11-07T10:31:03 thanks 2017-11-07T10:31:29 tampakrap: maybe you can also document the "HowTO create VPN setup" a bit in our wiki? 2017-11-07T10:31:55 yep 2017-11-07T10:32:36 tampakrap: thanks 2017-11-07T10:32:39 *** malcolmlewis has quit IRC 2017-11-07T10:32:57 tampakrap: JFYI: jdsn will migrate svn.opensuse.org to 42.3 and into the heroes network 2017-11-07T10:33:40 but kernel.o.o will stay at the suse-dmz network right? 2017-11-07T10:34:30 jdsn already has openvpn account :) 2017-11-07T10:34:33 jdsn: query 2017-11-07T10:46:39 *** malcolmlewis has joined #opensuse-admin 2017-11-07T10:46:39 *** malcolmlewis has joined #opensuse-admin 2017-11-07T11:05:36 *** tampakrap_ has joined #opensuse-admin 2017-11-07T11:06:27 *** tampakra- has joined #opensuse-admin 2017-11-07T11:10:05 *** tampakrap has quit IRC 2017-11-07T11:10:05 *** tampakrap_ has quit IRC 2017-11-07T11:10:05 *** tampakra- has quit IRC 2017-11-07T11:10:52 *** tampakrap has joined #opensuse-admin 2017-11-07T11:11:20 *** tampakrap_ has joined #opensuse-admin 2017-11-07T11:11:54 *** tampakra- has joined #opensuse-admin 2017-11-07T11:12:07 *** Son_Goku has joined #opensuse-admin 2017-11-07T11:14:02 *** differentreality has quit IRC 2017-11-07T11:14:02 *** skiarxon has quit IRC 2017-11-07T11:25:46 *** Son_Goku has quit IRC 2017-11-07T11:31:09 *** cboltz has joined #opensuse-admin 2017-11-07T11:32:08 *** Son_Goku has joined #opensuse-admin 2017-11-07T11:40:52 *** Son_Goku has quit IRC 2017-11-07T11:41:59 *** tampakrap has left #opensuse-admin 2017-11-07T11:43:10 *** tampakrap_ has left #opensuse-admin 2017-11-07T11:44:04 *** tampakra- has quit IRC 2017-11-07T11:45:47 *** tampakrap has joined #opensuse-admin 2017-11-07T12:02:58 *** nicolasbock has joined #opensuse-admin 2017-11-07T12:15:27 *** Son_Goku has joined #opensuse-admin 2017-11-07T12:35:46 *** Son_Goku has quit IRC 2017-11-07T12:46:13 *** Son_Goku has joined #opensuse-admin 2017-11-07T12:59:03 *** cboltz has quit IRC 2017-11-07T13:01:46 *** Son_Goku has quit IRC 2017-11-07T13:08:17 *** Son_Goku has joined #opensuse-admin 2017-11-07T14:58:28 *** katnip has joined #opensuse-admin 2017-11-07T14:59:34 hello, is there someone i can talk with about two issues before hack week begins? 2017-11-07T15:21:46 *** sven15 has joined #opensuse-admin 2017-11-07T15:22:00 katnip: shoot 2017-11-07T15:22:59 Anyway, i'll state the 'issues' i have here, maybe someone can help, i have already sent a ticket to admin@opensuse.org to no reply yet, i emailed Robert Brown about them and he suggested i come in here to see if i can get help, i was recently accepted as an openSUSE member. However, two things are at mind 2017-11-07T15:23:55 my email that katnip@opensuse.org is forwarded to the wrong address now, it should be forwarded to linuxmajic@gmail.com 2017-11-07T15:24:00 you have a ticket number? 2017-11-07T15:24:18 second, when might i be able to get the cloak? 2017-11-07T15:24:31 one second, ill look and see if i do 2017-11-07T15:25:14 your mail is fixed, it takes a few minutes to propagate 2017-11-07T15:26:13 [openSUSE Tracker] 2017-11-07T15:26:13 Issue #27100 has been reported by linuxmajic@gmail.com. 2017-11-07T15:26:29 thank you 2017-11-07T15:27:26 reassigned to the appropriate person to set up your cloak as well 2017-11-07T15:28:05 thank you very much :) 2017-11-07T15:28:24 welcome! 2017-11-07T16:05:49 *** orangecms has joined #opensuse-admin 2017-11-07T16:09:46 *** sven15 has quit IRC 2017-11-07T16:12:15 *** matthias_bgg has quit IRC 2017-11-07T16:17:43 *** orangecms has joined #opensuse-admin 2017-11-07T16:22:41 *** mcaj has quit IRC 2017-11-07T16:24:43 *** Son_Goku has quit IRC 2017-11-07T16:41:51 *** asmorodskyi_ has quit IRC 2017-11-07T17:06:52 *** orangecms has quit IRC 2017-11-07T17:23:12 *** Ada_Lovelace has joined #opensuse-admin 2017-11-07T17:23:57 tampakrap: still around ? 2017-11-07T17:24:11 tampakrap: I want to know a bit more about the issue with kernel.o.o ? 2017-11-07T17:24:12 *** cboltz has joined #opensuse-admin 2017-11-07T17:24:12 *** cboltz has joined #opensuse-admin 2017-11-07T17:24:41 tampakrap: what's the problem with this part? Why can't it go to the openSUSE network ? 2017-11-07T17:25:06 the problem is: neither me nor jdsn know about this kernel.o.o stuff, which is running on the machine 2017-11-07T17:25:44 ...and as jdsn is listed as official maintainer for the machine, we thought that this should not be a problem (as nobody informed him) 2017-11-07T17:25:53 kl_eisbaer: I was just asking, I didn't state a problem :) 2017-11-07T17:25:58 ah okay 2017-11-07T17:26:14 my understanding was that someone from the suse kernel team is responsible for that service and is co-maintaining the machine 2017-11-07T17:45:53 tampakrap: that's the funny thing that jdsn could not answer so far ;-) 2017-11-07T17:46:06 tampakrap: ...and to my understanding, he was/is the only maintainer of the machine 2017-11-07T17:46:07 *** orangecms has joined #opensuse-admin 2017-11-07T17:46:22 but we will definitively find out once we start migrating the service :-) 2017-11-07T17:46:39 lol okay 2017-11-07T17:46:40 tampakrap: JFYI: I setup a new VM "svn2" for this with openSUSE 42.3 2017-11-07T17:47:16 once jdsn setup the svn and maybe stuff from the kernel guys there, we can remount the two other devices on the new system and kill the old one 2017-11-07T17:48:09 might be that the new machine needs some special handling in the firewall as it looks like it's trying to contact git repos from all over the world - but I was just looking for a minute at the stuff, so could not tell you more at the moment 2017-11-07T17:48:39 but at least we could try to migrate it and see if/where there is a problem 2017-11-07T17:48:59 * kl_eisbaer is counting 40 VMs meanwhile ... 2017-11-07T17:49:09 probably it was trying to contact git.kernel.org which is a round-robin dns? 2017-11-07T17:49:29 tampakrap: yes, that was what I figured out 2017-11-07T17:49:56 tampakrap: but I don't think that this should be a big problem, as the connection is going out to the world and not the other way around 2017-11-07T17:50:10 yep I agree 2017-11-07T17:50:28 tampakrap: the more funny question would be who is responsible for this stuff - and who decided to put it on the svn machine 2017-11-07T17:50:42 ^^ => something to find out until Thursday :-) 2017-11-07T17:51:06 tampakrap: do you know when we have our meeting? 2017-11-07T17:51:11 19:00 or 20:00 today ? 2017-11-07T17:51:29 I assume it was on the svn machine because all control management stuff went to the same box back then 2017-11-07T17:51:33 kl_eisbaer: 20:00 CET 2017-11-07T17:51:40 tampakrap: puh, ok. 2017-11-07T17:51:49 moving my ass* out of the office 2017-11-07T17:51:56 ...CU there 2017-11-07T17:51:59 so I'm going home as well to be in time 2017-11-07T17:52:02 cya lazer 2017-11-07T17:52:05 *** kl_eisbaer has quit IRC 2017-11-07T18:09:56 *** fvogt has joined #opensuse-admin 2017-11-07T18:14:43 cboltz, are you around? 2017-11-07T18:14:58 yes ;-) 2017-11-07T18:15:35 are you in a position that you can work with me on a cloak? :) 2017-11-07T18:16:38 yes 2017-11-07T18:16:47 good :) 2017-11-07T18:16:59 give me a few minutes to check if my contact at freenode is available 2017-11-07T18:17:06 ok np 2017-11-07T18:29:45 *** orangecms has quit IRC 2017-11-07T18:41:02 katnip: no response yet - I'll tell you here or in the ticket when the cloak setup is done 2017-11-07T18:41:37 okay thank you 2017-11-07T18:42:19 *** tigerfoot has quit IRC 2017-11-07T18:56:27 *** ancorgs has quit IRC 2017-11-07T18:59:23 *** pjessen has joined #opensuse-admin 2017-11-07T18:59:40 I will join in about 10mins. 2017-11-07T19:01:45 I will also be a few mins late, I'm on the phone 2017-11-07T19:02:30 *** lars__ has joined #opensuse-admin 2017-11-07T19:03:42 Huhu? 2017-11-07T19:03:51 hi lars__ 2017-11-07T19:03:53 Hi 2017-11-07T19:04:06 Hi, isn't meeting time ? 2017-11-07T19:04:10 pjessen and tampakrap will both be a few minutes late, therefore we didn't start yet 2017-11-07T19:04:12 Good arrived at home? :-) 2017-11-07T19:04:18 *** kl_eisbaer has joined #opensuse-admin 2017-11-07T19:04:18 ah, ok. 2017-11-07T19:04:23 Ada_Lovelace: jip, thanks 2017-11-07T19:04:38 ...and now my messenger also arrived :-) 2017-11-07T19:04:44 but actually - we can start with the community Q&A ;-) 2017-11-07T19:04:55 okay I'm here as well 2017-11-07T19:05:04 so - welcome everybody to the monthly Heroes meeting! 2017-11-07T19:05:19 does someone from the community have questions? 2017-11-07T19:05:33 I'm moving the topics from last meeting 2017-11-07T19:06:29 added 2017-11-07T19:07:02 looks like nobody has questions, so let's move to the status reports 2017-11-07T19:07:38 who should start? 2017-11-07T19:08:11 what about you? ;-) 2017-11-07T19:08:17 :D 2017-11-07T19:08:40 Monitoring: we are now at 40 VMs and >900 monitored services 2017-11-07T19:08:41 You can tell us a lot. :) 2017-11-07T19:08:59 wow 2017-11-07T19:09:20 as I wrote in the news, not only the checks are interesting, also the information provided via the graphs are helpful 2017-11-07T19:09:39 ...so - as example - to see how many people are logged in via VPN all the time ;-) 2017-11-07T19:10:07 I plan to get more information also from our nginx and lighttpd instances, but this seems to be a bit tricky 2017-11-07T19:10:49 JFYI: I disabled notifications for the Updates check, as this turned out to be the service with most notifications ;-) 2017-11-07T19:11:17 ..and as we update on Thursday anyway, I think it is ok to disable notifications for this check 2017-11-07T19:12:03 I also wrote a short "run_zypper_up" script, which is in /root/bin/ on monitor.infra.opensuse.org 2017-11-07T19:12:14 I'm back 2017-11-07T19:12:17 would it be possible to run the update check _after_ the cronjob that auto-installs most updates? 2017-11-07T19:12:40 if you log in to this machine via SSH and have agent forwarding enabled, running this script helps to automate the maintenance on thursday 2017-11-07T19:12:55 cboltz: the update check *is* running after that, yes 2017-11-07T19:13:40 :-) 2017-11-07T19:13:47 Other topic: 2017-11-07T19:13:57 The Provo mirror supports now http2 protocol 2017-11-07T19:14:18 ...but we need to investigate if libcurl uses that protocol automatically or not 2017-11-07T19:15:15 I talked with the zypper maintainer, and he told me that it might be possible to add a configurable feature, it needed. So zypper could make use of the http2 protocol, too 2017-11-07T19:15:30 tcp fast open is enabled anyway already 2017-11-07T19:16:07 The Nuremberg mirror (aka download.opensuse.org) is getting closer to a state where I can open it up to the heroes: 2017-11-07T19:16:23 SUSE specific stuff has been migrated to another machine already 2017-11-07T19:16:47 so what's left is to have the machine re-installed with openSUSE and place it in the heroes network 2017-11-07T19:17:05 sounds good 2017-11-07T19:17:14 we might need a special network interface for the push from the OBS, but that should not be a big problem 2017-11-07T19:17:24 will that re-install happen with salt? ;-) 2017-11-07T19:17:34 I hope to get this done next week 2017-11-07T19:17:49 cboltz: problably not - as long as I do it 2017-11-07T19:18:12 Next mirror under our control: widehat, aka rsync.opensuse.org 2017-11-07T19:18:24 this host as currently a major problem: disk full 2017-11-07T19:18:51 as we can not extend the storage of this machine, I'm currently thinking to declare our mirror in Provo as new rsync.opensuse.org 2017-11-07T19:19:19 as this mirror in Provo is currently the only one who has everything that is also available on download.opensuse.org 2017-11-07T19:19:36 => in the end it's just about the DNS name 2017-11-07T19:19:40 Is this mirror working correctly now? 2017-11-07T19:20:00 so far no problem reports so far (other than one bug report about the used style sheet) 2017-11-07T19:20:06 I know about problems with the provo mirror in the past... 2017-11-07T19:20:11 it doesn't have ipv6 though, would that be a problem? 2017-11-07T19:20:17 Ada_Lovelace: I guess the problem in the past was just a missing redirector setup 2017-11-07T19:20:41 tampakrap: right, but latest rumor has it that MF-IT is able now to assign IPv6 addresses in Provo, too 2017-11-07T19:20:50 ah nice 2017-11-07T19:20:53 nice 2017-11-07T19:21:15 tampakrap: so let's wait for my request for IPv6 addresses, check of the rsync modules (and the "knapsack" stuff) to work 2017-11-07T19:21:24 and then switch over 2017-11-07T19:22:05 I still need to get some approval because of the additional bandwidth used, but I see currently not really any other chance to provide a fully equipped rsync.opensuse.org 2017-11-07T19:22:41 ...or we find a mirror as "sponsor" that hosts all the stuff, including the knapsack module, for us 2017-11-07T19:23:16 JFYI: the python-knapsack scripts fill up the 80g, 160g, 320g, ... modules with content based on the access logs 2017-11-07T19:23:43 what exactly does " we can not extend the storage" mean? All disks in NBG full, or just a restriction on this VM? 2017-11-07T19:23:59 The widehat machine is not running inside the Nuremberg office 2017-11-07T19:24:30 cboltz: widehats place is sponsored by QSC - and is an old machine with "just 7TB" capacity 2017-11-07T19:24:59 cboltz: does that answer your question? 2017-11-07T19:25:02 I know the new hosting boss at QSC. He changed from 1&1 to them after my election. 2017-11-07T19:25:05 yes, thanks 2017-11-07T19:25:08 Should I write him? 2017-11-07T19:25:25 Ada_Lovelace: in the past, our problem was that they want to end the sponsoring ... 2017-11-07T19:25:28 That's the ex 1&1 hosting boss. 2017-11-07T19:25:35 Oh... 2017-11-07T19:25:45 ...so we tried ot avoid any requests ... 2017-11-07T19:26:00 I guess it's more about bandwidth than the actual disk space? 2017-11-07T19:26:01 as long as the machine is up and running, everything is ok for us 2017-11-07T19:26:35 pjessen: in the past, widehat was indeed a bandwidth saver for the NUE office 2017-11-07T19:26:48 especially as we had a dark fiber to their data center 2017-11-07T19:27:09 so we could push stuff from download.o.o to widehat via that dark fiber 2017-11-07T19:27:31 ...and everybody else was downloading from widehat using the bandwidth from QSC 2017-11-07T19:27:39 yeah, I get the picture. 2017-11-07T19:27:56 the machine is an old system with a local RAID controller 2017-11-07T19:28:11 the maximum amount of disks is installed 2017-11-07T19:28:20 ...and it sadly supports only 1TB disks 2017-11-07T19:28:28 so extending it is not possible 2017-11-07T19:28:54 we might be able to replace the machine silently with a new one with bigger disks, but so far I don't see that happen 2017-11-07T19:29:12 *** asmorodskyi has joined #opensuse-admin 2017-11-07T19:29:45 how about a bigger raid controller? I have some spare that will take 2Tb disks. 2017-11-07T19:30:04 might be possible, but I need to check the hardware first 2017-11-07T19:30:23 You don't have access to the data center? 2017-11-07T19:30:34 Ada_Lovelace: I have access, yes. 2017-11-07T19:30:53 But I need to fire up the next VPN to get access to the machine for checking dmidecode ;-) 2017-11-07T19:31:40 => what about moving the discussion about widehat to "later", resp. mailing list, once I have the hardware details ? 2017-11-07T19:32:07 sounds like a good idea 2017-11-07T19:32:09 yep. 2017-11-07T19:32:16 Next topic: logs from #opensuse-admin 2017-11-07T19:32:36 It's true, I'm lazy ;-) 2017-11-07T19:32:56 ...and I'm not running my IRC client all the day, but our bot is there... 2017-11-07T19:33:23 ...so I - in my glory ;-) decided to let it log for me, but I guess this might be useful for others, too? 2017-11-07T19:33:31 => https://monitor.opensuse.org/heroes/ 2017-11-07T19:33:55 I could put the URL behind LDAP auth, but at the moment, I think it might be useful also for others. 2017-11-07T19:34:01 => what do you think? 2017-11-07T19:34:11 leave it public, it's a public channel after all 2017-11-07T19:34:18 It should be public... 2017-11-07T19:34:26 yes, keep it public 2017-11-07T19:34:41 fine with me ;-) 2017-11-07T19:34:47 but maybe add a note to /topic to make people aware that the channel is logged 2017-11-07T19:34:58 cboltz: up to you ;-) 2017-11-07T19:35:15 we might also put a link to the log in our wiki page ? 2017-11-07T19:35:23 But we should speak about the time how long it should be saved or when cleaned. 2017-11-07T19:35:27 *** asmorodskyi has quit IRC 2017-11-07T19:35:33 I doubt I have permissions to update /topic 2017-11-07T19:35:55 cboltz: IMHO /op should work in this channel for everyone 2017-11-07T19:36:00 +1 for the link to the wiki page 2017-11-07T19:36:12 Ada_Lovelace: any suggestions for the time frame? 2017-11-07T19:36:52 kl_eisbaer: no, /op gives me "you have to be operator in #opensuse-admin to do that" 2017-11-07T19:36:53 1 - 2 months (depends on the size of the disk) 2017-11-07T19:37:19 cboltz: sorry, I mean you should ask chanserv to become op 2017-11-07T19:37:41 cboltz: if you have a suggestion for the topic header, I can do that later for you ;-) 2017-11-07T19:37:53 Ada_Lovelace: the size does not really matter 2017-11-07T19:38:35 Ada_Lovelace: we are speaking about 472k at the moment 2017-11-07T19:38:45 I'd vote to keep the logs forever like we do with ML archives 2017-11-07T19:38:46 Then 3 months should be more than enough... It can be that anybody wants to refer to any task in the chat. 2017-11-07T19:39:09 deleting something from the internet doesn't work anyway ;-) 2017-11-07T19:39:26 cboltz: +1 from my side 2017-11-07T19:39:31 Ok 2017-11-07T19:39:41 restrict the search engines maybe? 2017-11-07T19:39:41 if we need to censor something, we can do it on the monitoring host anyway 2017-11-07T19:39:51 tampakrap: fine with me 2017-11-07T19:40:21 okay 2017-11-07T19:40:32 the bot is amazing, thanks a lot for it! 2017-11-07T19:40:54 tampakrap: you can even enhance it to a "meet bot" ;-) 2017-11-07T19:41:14 where is it running btw? on scar? 2017-11-07T19:41:24 tampakrap: ...or connect it to the rabbitmq queue and get informations pushed here once a package in openSUSE:infrastructure is built 2017-11-07T19:41:34 tampakrap: it's running on the monitor machine ;-) 2017-11-07T19:41:35 yeah that would be amazing 2017-11-07T19:41:55 or to send new merge requests and commits in the salt repository 2017-11-07T19:42:22 tampakrap: ...and all you need for this is netcat ;-) 2017-11-07T19:42:34 *** asmorodskyi has joined #opensuse-admin 2017-11-07T19:42:57 I've placed an example script in /root/bin/send_irc_message on the monitor machine 2017-11-07T19:42:57 2017-11-07T19:43:09 ...if you want to try it out. 2017-11-07T19:43:33 At the moment, the port is bond to localhost, but I can open it up to the whole infra.opensuse.org network, if this is needed 2017-11-07T19:44:27 Other topic: galera cluster ? 2017-11-07T19:44:28 PROBLEM: MySQL WSREP recv on galera2.infra.opensuse.org - CRIT wsrep_local_recv_queue_avg = 1.184783 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera2.infra.opensuse.org&service=MySQL%20WSREP%20recv 2017-11-07T19:44:44 ...no, that ^^ was not planned ;-) 2017-11-07T19:44:54 I already wondered ;-) 2017-11-07T19:44:56 hahahaha 2017-11-07T19:45:06 but as you can see, the cluster still needs some fine tuning 2017-11-07T19:45:30 I just took some default values and combined them with some "good practices" from our internal cluster at SUSE 2017-11-07T19:45:56 At the moment, I have 3 interesting things: 2017-11-07T19:46:36 * mysql-tuning-scripts is updated and contains now the latest mysqltuner.pl script, which is analizing a running mysql instance and (new) also gives some hints for galera cluster 2017-11-07T19:47:30 * I will update/change the VM definition for the machines, to allow more features from the hypervisor CPUs - sadly this requires a complete power cycle of the VM 2017-11-07T19:47:46 ^^^ that should give hopefully a bit more performance 2017-11-07T19:48:24 * I think we don't need to migrate so many databases as I initially thought: most of the DBs hosted at the old cluster are meanwhile obsolete 2017-11-07T19:48:51 The biggest DB will be the one from beans.o.o aka piwik 2017-11-07T19:49:32 I'm currently thinking to add the new cluster as slave for this DB to the old cluster, to avoid a big downtime if we migrate piwik 2017-11-07T19:49:53 but I haven't done that, so I need to do some more testing 2017-11-07T19:50:24 the other databases (incl. the wiki ones) should be "ready to migrate" in one or two weeks, IMHO 2017-11-07T19:51:02 if someone is interested and wants to become a "DB-Admin", I'm happy to hand over ;-) 2017-11-07T19:51:15 just tell me when you want to migrate the wiki DBs, so that we can make the wikis read-only during that time 2017-11-07T19:51:26 cboltz: will do, of course 2017-11-07T19:51:50 well, ideally I'd like to hand over creating database users and databases to salt ;-) 2017-11-07T19:51:59 I hope to have just minimal impact for all DBs other than the piwik one 2017-11-07T19:52:21 cboltz: but even than, you definitively want to have some DB-Admin who cares about your profile ;-) 2017-11-07T19:52:44 otherwise, your wikis would suddenly become very, very slow .... 2017-11-07T19:53:17 ...another topic: svn/kernel.opensuse.org 2017-11-07T19:53:21 I know salt can't do automated performance monitoring and tuning ;-) 2017-11-07T19:53:44 the current plan is to migrate the machine hosting the two services on Thursday to the new network 2017-11-07T19:54:26 ...that's all I have as "status report" so far 2017-11-07T19:55:23 one quick question 2017-11-07T19:55:33 the mysql servers are using the official opensuse packages? 2017-11-07T19:55:42 no 2017-11-07T19:55:55 okay 2017-11-07T19:56:02 they are using the ones from server:database, as the latest galera features are very fresh 2017-11-07T19:56:22 the ones for 42.3 don't support galera so good, (yet) 2017-11-07T19:56:53 with the latest packages from server:database, it's more or less just adding the galera packages and a configuration snipplet 2017-11-07T19:56:59 got it 2017-11-07T19:57:18 any other status reports ? 2017-11-07T19:57:36 no much from me, havent had time since the summer holidays 2017-11-07T19:57:41 about salt 2017-11-07T19:57:51 we support now gpg encrypted pillars 2017-11-07T19:57:55 pjessen: is the "sience" list online already ? 2017-11-07T19:58:05 it should be, yes. 2017-11-07T19:58:12 opensuse-science@o.o 2017-11-07T19:58:21 ok, thanks 2017-11-07T19:58:45 tampakrap: what does this mean ? 2017-11-07T19:59:10 this means that we can put passwords in pillars 2017-11-07T19:59:30 nice :-) 2017-11-07T19:59:42 there is a MR open that has the passwords for the keepalived config on anna/elsa encrypted 2017-11-07T20:00:03 it needs documentation though on how to use it and about the structure 2017-11-07T20:00:09 and then I'll proceed with that MR 2017-11-07T20:00:43 :D 2017-11-07T20:01:13 BTW: daffy (aka login2) is prepared for the 2nd daffy already - I setup the keepalived there already 2017-11-07T20:02:03 cool 2017-11-07T20:02:08 BTW2: something for cboltz (-: https://monitor.opensuse.org/pnp4nagios/index.php/graph?host=redmine.infra.opensuse.org&srv=Heroes+tickets&view=2 2017-11-07T20:02:46 yes, I already noticed that :-) 2017-11-07T20:02:52 ^^ should give you a quick overview about the current tickets on progress.opensuse.org 2017-11-07T20:03:24 I also defined some warning/critical levels - just to create some fun here in the channel ;-) 2017-11-07T20:03:30 hehe 2017-11-07T20:04:45 but while I was creating that check, I was wondering if we shouldn't define a maximum lifetime for a ticket 2017-11-07T20:05:25 I was just wondering if tickets that are older than a year are really interesting anyone any longer? 2017-11-07T20:05:26 We had the same in Bugzilla... 2017-11-07T20:06:09 Ada_Lovelace: ...and was there a solution ? 2017-11-07T20:06:11 Christian and I found some tickets from the past which were interesting for us. 2017-11-07T20:06:17 in bugzilla, things take time. 2017-11-07T20:06:29 Reviewing and pinging was the solution. 2017-11-07T20:07:24 maybe we should schedule a "progress" cleanup event ? 2017-11-07T20:07:28 right, let's handle tickets quickly instead of inventing an auto-close (which is more or less a motivation to be lazy IMHO) 2017-11-07T20:07:47 agree. 2017-11-07T20:07:49 yes, such a cleanup would make sense 2017-11-07T20:08:20 maybe a good transfer to the next topic: " offsite meeting? " :-) 2017-11-07T20:08:28 I would be available. 2017-11-07T20:09:05 kl_eisbaer: indeed, pinging people in the same room is much easier ;-) 2017-11-07T20:09:24 any suggestions ? 2017-11-07T20:09:41 ...I hope everyone knows that SUSE will have the yearly hackweek starting on Friday ? 2017-11-07T20:09:41 somewhere in Zurich? 2017-11-07T20:09:47 :-) 2017-11-07T20:09:55 pjessen: fine with me :-) 2017-11-07T20:09:58 I saw it today. 2017-11-07T20:10:14 pjessen: but I guess you need to organize the "where" in Zurich for us :-) 2017-11-07T20:10:29 zurich is fine for me, my brother lives there 2017-11-07T20:10:50 heya: so we have a Party location already :-) 2017-11-07T20:12:20 But wait... Are trains from Germany to Zurich going at the moment? There is a lot damageg, if I think back. 2017-11-07T20:12:37 *damaged* 2017-11-07T20:12:39 pjessen: can you try to organize something ? 2017-11-07T20:12:42 I think that tunnel was fixed already. 2017-11-07T20:12:54 Really? Then all is ok. 2017-11-07T20:13:14 I'll have to say no - at least not on this side of Christmas 2017-11-07T20:13:17 pjessen: I guess the main need is a reliable internet connection 2017-11-07T20:13:41 pjessen: next year should be not a big problem ;-) 2017-11-07T20:14:22 What time frame do we have in mind? Roughly. 2017-11-07T20:14:23 February should be the best for me because of semester holidays. ;-) 2017-11-07T20:14:43 Yeah, we have Sportferien in Feb too. 2017-11-07T20:15:04 fine by me 2017-11-07T20:15:09 as long as you avoid the carnival and FOSDEM weekends, February sounds good 2017-11-07T20:15:20 I guess 2-3 days (a weekend?) should really be enough 2017-11-07T20:15:44 We weekend after FOSDEM is good. 2017-11-07T20:16:01 Is this something we should continue on the mailing list later? 2017-11-07T20:16:18 pjessen: jip, good idea 2017-11-07T20:16:24 no, the weekend after FOSDEM is carnival 2017-11-07T20:16:27 perfect 2017-11-07T20:16:40 Then we can have a party. ;-) 2017-11-07T20:16:42 and I know ~40 people who would hate me if I refuse to drive the carnival float ;-) 2017-11-07T20:17:19 Let's speak about it on the mailinglist. 2017-11-07T20:17:20 So it looks like everybody is looking forward to a meeting in Zurich :-) 2017-11-07T20:17:34 Wow, I should have kept my mouht shut .... 2017-11-07T20:17:51 otherwise we could always schedule a meeting here in Nuremberg again. That should not really be a problem. 2017-11-07T20:18:04 * kl_eisbaer now waits for Theo to invite everyone to Prague ;-) 2017-11-07T20:18:11 prague yey! 2017-11-07T20:18:19 we have the conference as well here 2017-11-07T20:18:31 Yes. The oSC 2017-11-07T20:18:37 or actually, let's go to greece! 2017-11-07T20:18:38 I don't mind setting something up, but Nuernberg is within range for me too. 2017-11-07T20:18:53 Too cold there in Feb :-) 2017-11-07T20:19:18 pjessen: I just don't wanted to put pressure on you, that's why I offered NUE again ;-) 2017-11-07T20:19:56 pjessen: but we can of course include a survival training in the offsite meeting 2017-11-07T20:20:16 ^ => let's move the discussion to the mailing list ;-) 2017-11-07T20:20:21 thanks - I have a lot on my plate, to be honest. 2017-11-07T20:20:41 ok 2017-11-07T20:20:53 Next topic: enhance infra.opensuse.org domain ? 2017-11-07T20:21:05 that one's from me 2017-11-07T20:21:18 "enhance" ? 2017-11-07T20:21:38 in short: are there any objections, if I add the "service" names as aliases to our hosts with their special names? 2017-11-07T20:21:58 boosters.infra.opensuse.org would have an additional alias connect.infra.opensuse.org as example 2017-11-07T20:22:13 infra.opensuse.org isn't the right thing for real domainnames. 2017-11-07T20:22:24 That's something for hostnames. 2017-11-07T20:22:26 that would make it a bit easier - at least for me - to get to the "right" machine directly 2017-11-07T20:22:37 I would like to have them, and I created some as well already 2017-11-07T20:22:38 Sounds good, that's what we do with all services locally. A service can always move., 2017-11-07T20:22:40 Ada_Lovelace: ? 2017-11-07T20:23:12 You want to offer easy domainnames for users without infra in it. 2017-11-07T20:23:26 Ada_Lovelace: no, sorry 2017-11-07T20:23:28 I know such names only as hostnames. 2017-11-07T20:23:39 Ada_Lovelace: I just want to make my live as admin easier 2017-11-07T20:23:41 it's just a dns cname 2017-11-07T20:23:56 no, the point is to be able to do `ssh connect` instead of going to the machine list to find out where you need to ssh to check what is broken on connect 2017-11-07T20:24:08 If we have the cname additionally to this name, then ok. 2017-11-07T20:24:09 CNAMEs are cheap 2017-11-07T20:24:10 tampakrap: exactly 2017-11-07T20:24:15 are for free actually 2017-11-07T20:24:22 I don't like the situation with gitlab. 2017-11-07T20:24:27 totally desired imho 2017-11-07T20:24:48 gitlab moved from gitlab.o.o to gitlab.infra.o.o 2017-11-07T20:24:51 Ada_Lovelace: that's not related to my request, sorry 2017-11-07T20:24:59 ok 2017-11-07T20:25:05 I'm speaking about all the machines like scar.infra.opensuse.org 2017-11-07T20:25:24 not everyone knows what scar.infra.opensuse.org or mickey.infra.opensuse.org is doing 2017-11-07T20:25:58 adding service names is a good thing. 2017-11-07T20:26:02 but if they become aliases like "vpn.infra.opensuse.org" or "salt.infra.opensuse.org", most people might instantly know which services they can find on those machines 2017-11-07T20:26:40 Ada_Lovelace: does this make it a bit clearer to you? 2017-11-07T20:27:26 Yes 2017-11-07T20:27:33 still objections ? 2017-11-07T20:28:25 I like service names as additional names. 2017-11-07T20:28:47 ok - I take this as a "yes" from everyone ;-) 2017-11-07T20:28:51 thanks! 2017-11-07T20:29:10 Next topic: sponsoring offer from cPanal (see Doug's mail on the ML) 2017-11-07T20:29:24 typo, I'll fix it 2017-11-07T20:30:03 I've heard no news about this topic - anyone else? 2017-11-07T20:30:26 nothing 2017-11-07T20:30:45 anyone who wants to drive this ? 2017-11-07T20:31:07 I will ask Max tomorrow, I *think* he communicated something about htis already 2017-11-07T20:31:10 so AI for me 2017-11-07T20:31:20 thanks 2017-11-07T20:31:50 Next topic: transfer opensuse.cz domain (another mail from Doug) 2017-11-07T20:32:00 I object on this 2017-11-07T20:32:11 tampakrap: ok 2017-11-07T20:32:42 I told Petr (the original requestor) already that I don't like the idea that the official opensuse DNS will handle domains from other community teams 2017-11-07T20:32:54 their request is simple though, they want a redirect to the wiki 2017-11-07T20:33:11 but we will have to accept other domains in the future as well, and maintain them 2017-11-07T20:33:16 do we want to do that? 2017-11-07T20:33:38 Why not? 2017-11-07T20:33:44 Who is behind opensuse.cz ? 2017-11-07T20:33:48 if it's just a redirect, it's easy to answer 'yes' 2017-11-07T20:34:17 besides that, having control over opensuse.* domains can't hurt 2017-11-07T20:34:17 tampakrap: just to understand you right: Petr offered us to get the owner of the opensuse.cz domain? 2017-11-07T20:35:02 can "we" even act as the owner? 2017-11-07T20:35:10 correct, I said that I will take the topic to our meeting, but he went as well to doug, who sent the mail first to our ml 2017-11-07T20:35:47 from a technical standpoint, I see no issues - but the legal point needs to be clarified by the board IMHO 2017-11-07T20:36:08 actually it's already on the board's radar 2017-11-07T20:36:35 last I heard is that Richard waits for response from Ciaran who should be in the whois data 2017-11-07T20:37:04 okay mind responding to Petr and Doug then? 2017-11-07T20:37:11 the whois data will accept anything. 2017-11-07T20:37:27 pjessen: :-) 2017-11-07T20:37:32 *** solevi has joined #opensuse-admin 2017-11-07T20:37:47 tampakrap: I would say: go ahead 2017-11-07T20:37:47 pjessen: you are technically right, but legally Ciaran might have a different opinion ;-) 2017-11-07T20:38:04 ...as we are just the technical part of the story 2017-11-07T20:38:15 tampakrap: but what is with their webside? 2017-11-07T20:38:43 last thing they told me, they want to get rid of it 2017-11-07T20:38:47 I guess they want to leave that stuff as it is and just want to get the opensuse.cz domain (DNS) under openSUSE control? 2017-11-07T20:39:25 there is presuambly also an issue of cost? 2017-11-07T20:39:30 tampakrap: maybe put this (together with the Email question) in your answer email? 2017-11-07T20:39:39 ack 2017-11-07T20:40:22 means: is it ok for them that openSUSE takes over the domain, redirects anything opensuse.cz related to the CZ wiki and skip the email part? 2017-11-07T20:40:31 ...something like that 2017-11-07T20:41:08 ok for everyone ? 2017-11-07T20:41:26 yes 2017-11-07T20:41:29 So we would be just bne the DNS admns? 2017-11-07T20:42:00 yes 2017-11-07T20:42:20 pjessen: IMHO yes. That would be my understanding 2017-11-07T20:42:37 I have no issue with that. Especially as I don't do any DNS admin ... 2017-11-07T20:42:52 pjessen: not yet... ;-) 2017-11-07T20:43:16 I guess the next 2 topics were handled already: monitoring/mirror status ? 2017-11-07T20:43:33 yes. You told all. ;) 2017-11-07T20:43:39 and the salt status 2017-11-07T20:44:01 tampakrap: I'm happy to hear more ;-) 2017-11-07T20:45:16 * cboltz wonders how the 40 VMs match the 28 pillar/id/* files 2017-11-07T20:45:40 cboltz: now you know how many machines are administrated by me ;-) 2017-11-07T20:46:00 lol 2017-11-07T20:46:15 yep let's fix that 2017-11-07T20:46:18 at least the galera machines are currently completely unmanaged 2017-11-07T20:46:18 but seriously - why don't you use salt? 2017-11-07T20:46:43 cboltz: fear? not enough knowledge? using ansible ? 2017-11-07T20:46:51 cboltz: choose one and you might be right 2017-11-07T20:47:51 I can help you to fix #2, which might then also fix #1 ;-) 2017-11-07T20:48:08 cboltz: thanks - I will definitively come back to that :-) 2017-11-07T20:48:23 tampakrap: btw, I've one question 2017-11-07T20:48:33 shoot 2017-11-07T20:48:46 tampakrap: is the salt master in the heroes network only serving for the heroes machines ? 2017-11-07T20:48:59 yes 2017-11-07T20:49:06 I was just wondering as I was on the svn machine in the other network 2017-11-07T20:49:26 but that might be just a "flash back", as I saw there some repos pointing to "nowhere" 2017-11-07T20:49:39 we used to have a separate master for the suse-dmz but it is currently broken since the network split 2017-11-07T20:49:48 ok 2017-11-07T20:49:57 hopefully will be back during hackweek 2017-11-07T20:50:13 about the ntp stuff - I've two notes from what I saw so far... 2017-11-07T20:50:22 That's your hackweek project? :D 2017-11-07T20:50:26 1) any objections using chrony instead of ntpd ? 2017-11-07T20:50:46 Any reason why? 2017-11-07T20:50:58 2) any objections to use just "ntp1", "ntp2" instead of the full DNS name? 2017-11-07T20:50:59 *** asmorodskyi has quit IRC 2017-11-07T20:51:05 no objections, I saw the configs are quite similar, but I'll need to disable it in salt as well, as ntp is currently managed in salt 2017-11-07T20:51:24 chrony is a bit more secure, as some tests from secint showed 2017-11-07T20:51:44 for 2 it will work if the machines have properly set up the searchlist right? 2017-11-07T20:51:46 the chrony maintainers did not implement all features, so they are not fully RFC compatible 2017-11-07T20:51:52 tampakrap: sounds like kl_eisbaer found a nice task to practise salt ;-) 2017-11-07T20:52:05 but they implemented enough for having everything that a client needs 2017-11-07T20:52:27 I'm a long-time fan of ntp, but that's personal. 2017-11-07T20:52:59 one example: chrony only binds to localhost and no other interface per default 2017-11-07T20:53:15 pjessen: I was, too, but times are changing ;-) 2017-11-07T20:53:55 tampakrap: yes, with just the hostname (ntp1), they will always just try their local domain 2017-11-07T20:54:37 pjessen: I just found one more or less important feature that chrony handles different than ntpd 2017-11-07T20:55:19 the "tinker panic 0" (which is btw. missing in the salt profile), is translated there to something like "makestep 1.0 3" 2017-11-07T20:55:49 tampakrap: ...and IMHO "disable monitor" is also missing in the salt profile, but I'm not sure here 2017-11-07T20:56:11 I'll check 2017-11-07T20:56:27 cboltz: I've already some other quick and easy things to add to salt 2017-11-07T20:57:02 disable monitor is there 2017-11-07T20:57:02 but my last merge requests are long ago - so I probably forgot how to do it properly 2017-11-07T20:57:10 tampakrap: ah, sorry 2017-11-07T20:57:18 tinker panic 0 is not 2017-11-07T20:57:22 add it to all machines? 2017-11-07T20:57:33 tampakrap: I guess the tinker panic 0 was left out because the initial formula did not support it 2017-11-07T20:57:40 but all virtual machines should have it 2017-11-07T20:57:52 only virtual? no physical? 2017-11-07T20:58:03 because it allows the clock to be stepped 2017-11-07T20:58:04 ntp servers as well? 2017-11-07T20:58:24 yes: for virtual machines this might be important during live migration or when they are paused 2017-11-07T20:58:46 ack 2017-11-07T20:58:59 some docs also mention that the virtual machines might have problems right after boot, when their "virtual hw clock" is not synced with the hypervisor 2017-11-07T20:59:13 so it's definitively a setting you want to have for virtual machines 2017-11-07T21:00:07 will have to ask you about that chrony feature tomorrow. 2017-11-07T21:00:11 the funny part with the "restrict" settings: you can skip all of them with chrony, if you do not plan to use the VM as time server 2017-11-07T21:00:34 ah, better defaults? 2017-11-07T21:00:35 pjessen: https://chrony.tuxfamily.org/ 2017-11-07T21:00:44 pjessen: more secure defaults, yes 2017-11-07T21:02:00 but I would say: 2 hours is enough for a meeting ;-) 2017-11-07T21:02:20 yeah, I have to go. Goot meeting though. 2017-11-07T21:02:23 good 2017-11-07T21:02:31 pjessen: CU 2017-11-07T21:03:06 see ya all. 2017-11-07T21:03:07 ...and bye, bye to everyone else :-) 2017-11-07T21:03:09 bye 2017-11-07T21:03:14 *** pjessen has quit IRC 2017-11-07T21:03:20 bye 2017-11-07T21:03:21 bye 2017-11-07T21:03:44 yeah let's finish it 2017-11-07T21:04:06 *** Ada_Lovelace has quit IRC 2017-11-07T21:05:13 *** kl_eisbaer has left #opensuse-admin 2017-11-07T21:06:01 *** lars__ has left #opensuse-admin 2017-11-07T21:19:42 log uploaded, and ticket for the December meeting created 2017-11-07T21:20:29 lars was right, we didn't put the tinker panic 0 because it was not supported back then 2017-11-07T21:21:28 tampakrap: the link to the admin wiki in the gitlab menu is a good idea - but it should point to the right project/wiki on progress ;-) 2017-11-07T21:22:05 I don't follow 2017-11-07T21:22:38 https://gitlab.infra.opensuse.org/infra/salt 2017-11-07T21:22:46 there's a "Wiki" link in the left column 2017-11-07T21:23:11 but it points to progress.o.o/opensuse-admin instead of progress.o.o/opensuse-admin-wiki 2017-11-07T21:23:18 ah okay 2017-11-07T21:23:39 it is under settings -> integrations 2017-11-07T21:23:41 I'll fix it now 2017-11-07T21:24:52 fixed 2017-11-07T21:25:08 thanks! 2017-11-07T21:25:25 can I update gitlab or you're busy with it? 2017-11-07T21:26:01 go ahead ;-) 2017-11-07T21:26:56 false alarm, still building 2017-11-07T21:27:10 ;-) 2017-11-07T21:34:28 RECOVERY: MySQL WSREP recv on galera2.infra.opensuse.org - OK wsrep_local_recv_queue_avg = 0.000000 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera2.infra.opensuse.org&service=MySQL%20WSREP%20recv 2017-11-07T21:38:24 *** matthias_bgg has joined #opensuse-admin 2017-11-07T21:43:11 *** katnip has quit IRC 2017-11-07T21:43:11 *** katnip has joined #opensuse-admin 2017-11-07T21:44:10 thank you cboltz :) 2017-11-07T21:44:21 you are welcome ;-) 2017-11-07T21:46:39 *** fvogt has quit IRC 2017-11-07T21:56:47 *** malcolmlewis has quit IRC 2017-11-07T22:09:40 *** malcolmlewis has joined #opensuse-admin 2017-11-07T23:06:27 *** solevi has quit IRC 2017-11-07T23:13:27 *** matthias_bgg has quit IRC 2017-11-07T23:25:13 *** Son_Goku has joined #opensuse-admin 2017-11-07T23:28:23 *** Fraser_Bell has joined #opensuse-admin 2017-11-07T23:28:23 *** Fraser_Bell has joined #opensuse-admin 2017-11-07T23:44:39 *** cboltz has quit IRC