2017-10-31T00:44:12 *** tigerfoot_ has joined #opensuse-admin 2017-10-31T00:44:14 *** tigerfoot has quit IRC 2017-10-31T01:38:02 *** Son_Goku has quit IRC 2017-10-31T01:55:57 *** Son_Goku has joined #opensuse-admin 2017-10-31T03:21:03 *** okurz has quit IRC 2017-10-31T03:21:51 *** okurz has joined #opensuse-admin 2017-10-31T03:23:45 *** Son_Goku has quit IRC 2017-10-31T07:14:26 *** tigerfoot_ has quit IRC 2017-10-31T08:08:33 *** mcaj has joined #opensuse-admin 2017-10-31T08:41:06 *** cboltz has joined #opensuse-admin 2017-10-31T08:41:06 *** cboltz has joined #opensuse-admin 2017-10-31T08:44:56 *** matthias_bgg has joined #opensuse-admin 2017-10-31T09:33:58 *** Son_Goku has joined #opensuse-admin 2017-10-31T10:34:39 *** kl_eisbaer has joined #opensuse-admin 2017-10-31T10:34:40 *** kl_eisbaer has joined #opensuse-admin 2017-10-31T10:46:20 PROBLEM: HTTP progress on redmine.infra.opensuse.org - HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 311 bytes in 0.001 second response time ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=redmine.infra.opensuse.org&service=HTTP%20progress 2017-10-31T10:52:41 *** fvogt has joined #opensuse-admin 2017-10-31T10:53:02 *** fvogt has quit IRC 2017-10-31T10:53:13 *** fvogt has joined #opensuse-admin 2017-10-31T10:53:39 *** fvogt has joined #opensuse-admin 2017-10-31T10:55:00 *** fvogt has joined #opensuse-admin 2017-10-31T10:56:19 RECOVERY: HTTP progress on redmine.infra.opensuse.org - HTTP OK: HTTP/1.1 200 OK - 8417 bytes in 0.234 second response time ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=redmine.infra.opensuse.org&service=HTTP%20progress 2017-10-31T11:00:28 PROBLEM: MySQL WSREP recv on galera2.infra.opensuse.org - CRIT wsrep_local_recv_queue_avg = 1.398417 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera2.infra.opensuse.org&service=MySQL%20WSREP%20recv 2017-10-31T11:00:29 PROBLEM: MySQL WSREP recv on galera3.infra.opensuse.org - CRIT wsrep_local_recv_queue_avg = 1.479675 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera3.infra.opensuse.org&service=MySQL%20WSREP%20recv 2017-10-31T11:10:36 cboltz: around for fast review? https://gitlab.infra.opensuse.org/infra/salt/merge_requests/70 2017-10-31T11:13:32 looks good 2017-10-31T11:14:07 thanks 2017-10-31T11:14:18 doing the gpg setup on the master now 2017-10-31T11:14:59 so we can finally store passwords in salt? nice :-) 2017-10-31T11:26:42 *** fvogt has quit IRC 2017-10-31T11:26:53 *** fvogt has joined #opensuse-admin 2017-10-31T11:34:02 *** ldevulder has quit IRC 2017-10-31T11:34:47 *** ldevulder has joined #opensuse-admin 2017-10-31T11:41:29 *** ldevulder has quit IRC 2017-10-31T11:52:20 *** ldevulder has joined #opensuse-admin 2017-10-31T11:54:01 *** kl_eisbaer has quit IRC 2017-10-31T12:05:12 cboltz: what's the machine in provo that we set up as saltmaster? 2017-10-31T12:05:26 water2 2017-10-31T12:09:24 *** kl_eisbaer has joined #opensuse-admin 2017-10-31T12:23:03 cboltz: https://docs.saltstack.com/en/latest/ref/renderers/all/salt.renderers.gpg.html#export-the-public-key the part about adding a secret to the file 2017-10-31T12:23:23 I understand that it needs one dedicated pillar file with the gpg encrypted variables 2017-10-31T12:23:30 and it can not be mixed with normal pillars 2017-10-31T12:23:34 how do you understand it? 2017-10-31T12:39:55 *** matthias_bgg has quit IRC 2017-10-31T12:50:58 the example looks like you are right 2017-10-31T12:51:23 that's bullshit 2017-10-31T12:51:26 I'll try it 2017-10-31T12:52:07 maybe there's a modifier you can apply to specific pillar data, which would avoid the separate files - no idea 2017-10-31T12:52:29 that said - wasn't the plan to use 'pass' to be able to easily re-encrypt password? If so, we'd need to have separate files anyway 2017-10-31T12:53:03 huh? 2017-10-31T12:53:18 pass uses .gpg files, while salt encrypted pillars need yaml files still 2017-10-31T12:55:17 ... which leads to the question if salt can read pillar data from external files... 2017-10-31T12:55:40 what do you mean? 2017-10-31T12:56:00 ideally something like 2017-10-31T12:56:13 dbpass: `cat dbpass.gpg` 2017-10-31T12:56:20 (yes, that's obviously pseudocode) 2017-10-31T12:57:32 no idea, I haven't dived so deep to see how it renders the encrypted data 2017-10-31T12:59:14 in theory something like 2017-10-31T12:59:26 {% import_text "secret.gpg" as secret %} 2017-10-31T12:59:33 dbpass: {{secret}} 2017-10-31T12:59:34 should work 2017-10-31T12:59:52 in practise, https://github.com/saltstack/salt/issues/39881 will probably bite us 2017-10-31T13:09:38 hmm, re-reading https://docs.saltstack.com/en/latest/ref/renderers/all/salt.renderers.gpg.html I wonder if we both misinterpreted 2017-10-31T13:09:42 To apply the renderer on a file-by-file basis add the following line to the top of any pillar with gpg data in it: 2017-10-31T13:09:52 it says "pillar with gpg data in it" 2017-10-31T13:10:01 but not "pillar with _only_ gpg data in it" 2017-10-31T13:29:42 one problem at a time 2017-10-31T13:29:57 for now I am trying to figure out why I can't encrypt a var with file of recipients 2017-10-31T13:30:32 echo -n "supersecret" | gpg --armor --batch --trust-model always --encrypt -f encrypted_pillar_recipients 2017-10-31T13:30:50 % cat encrypted_pillar_recipients ±[tampakrap_proxy_keepalived][●●] 2017-10-31T13:30:52 0x9640E4FA29485B97 2017-10-31T13:30:54 0xF1C33B7A1346F48E 2017-10-31T13:31:09 first is my key the second is from the saltmaster 2017-10-31T13:32:19 I get 2017-10-31T13:32:21 gpg: no valid addressees 2017-10-31T13:32:23 gpg: [stdin]: encryption failed: No user ID 2017-10-31T13:33:02 ah I htink I got it 2017-10-31T13:33:13 I need to ultimately trust all the recipients 2017-10-31T13:33:57 nope that didn't help :/ 2017-10-31T13:34:35 wild guess - try without the "0x" prefix? 2017-10-31T13:35:02 nope 2017-10-31T13:35:28 I probably misread the manual 2017-10-31T13:35:34 it says that the file should have only one key 2017-10-31T13:39:22 okay so googling says there are two ways to do it: 2017-10-31T13:39:37 1) pass multiple --recipient args 2017-10-31T13:40:05 2) create a `group mygroup = rcpt1 rcpt2 rcpt3` in gpg.conf 2017-10-31T13:44:54 I will create a bash script that will pass the multiple recipients from a file 2017-10-31T14:19:27 cboltz: good news, they can be at the same file 2017-10-31T14:31:04 *** Son_Goku has quit IRC 2017-10-31T14:32:19 I assume you are talking about encrypted vs. unencrypted pillar, right? 2017-10-31T14:33:18 sure 2017-10-31T14:35:09 https://gitlab.infra.opensuse.org/infra/salt/merge_requests/72/diffs 2017-10-31T14:41:40 *** nicolasbock has joined #opensuse-admin 2017-10-31T14:51:36 https://gitlab.infra.opensuse.org/infra/salt/merge_requests/73 2017-10-31T14:51:41 example of how it looks like 2017-10-31T14:51:43 cboltz: ^^ 2017-10-31T14:55:41 just looking on the "big picture" (without checking all the pillar details you set there): 2017-10-31T14:55:56 it's good to see we can embed encrypted pillar data between "normal" pillar data 2017-10-31T14:57:23 I somewhat dislike the length of the encrypted pillar data - it makes the pillar files very long (imagine 20 encrypted database passwords in pillar/role/wiki.sls) and much harder to read 2017-10-31T14:58:34 obviously we can't make the encrypted data shorter, so maybe we should store it in separate files 2017-10-31T14:58:48 (feel free to tell me if you disagree ;-) 2017-10-31T14:59:09 I pushed the creation of /etc/salt/gpgkeys 2017-10-31T14:59:21 I don't disagree, the big length is annoying 2017-10-31T14:59:46 separate files are fine, but it will mean more code to include them 2017-10-31T14:59:54 either in top.sls or from another sls file 2017-10-31T15:03:14 replied to your comments 2017-10-31T15:03:19 assuming we still want to use 'pass' (I never used it myself, but maybe I should ;-) 2017-10-31T15:03:39 - have a *.gpg file for each secret 2017-10-31T15:03:44 - {% import_text "filename.gpg" as secret %} and then use {{secret}} 2017-10-31T15:04:02 the gpg file in password-store is binary 2017-10-31T15:04:33 I'd guess/hope this is configurable ;-) 2017-10-31T15:04:44 https://gitlab.infra.opensuse.org/infra/pass/blob/master/01_Heroes/ariel.infra.opensuse.org/shell/root.gpg 2017-10-31T15:05:03 and even if we don't use pass, I still think having one file per encrypted secret would be nice 2017-10-31T15:05:48 again, one step at a time :) 2017-10-31T15:06:00 first let's get the gpg infrastructure code merged 2017-10-31T15:06:07 then we can discuss how to structure our data 2017-10-31T15:07:43 agreed - but we should decide soon, before we have 100 encrypted values ;-) 2017-10-31T15:07:57 yep 2017-10-31T15:08:00 so check my comments please 2017-10-31T15:13:41 silly question - why do you want to keep the -s option? 2017-10-31T15:14:16 IMHO "avoid having secrets in the bash history" would be a good reason to only allow `read` 2017-10-31T15:14:50 for testing mostly, but your point makes sense 2017-10-31T15:14:52 fixing it now 2017-10-31T15:18:52 fixed 2017-10-31T15:22:06 did you try --help after your change? ;-) 2017-10-31T15:22:19 lol no 2017-10-31T15:22:51 try it ;-) 2017-10-31T15:24:26 fixed 2017-10-31T15:25:45 better :-) 2017-10-31T15:26:14 the getopts block looks a bit superfluous now, but doesn't hurt - and will be useful once you want to add more options 2017-10-31T15:26:38 yep 2017-10-31T15:28:44 keep it, drop it or comment it out - whatever you prefer ;-) 2017-10-31T15:28:58 besides that, #72 looks good now :-) 2017-10-31T15:29:14 I'll be busy for a while, so don't expect more fast answers please 2017-10-31T15:29:21 cool thanks 2017-10-31T15:45:19 cboltz: (whenever you have time) suggestions on where to store the files? 2017-10-31T15:46:27 the password files I mean 2017-10-31T17:00:55 *** Son_Goku has joined #opensuse-admin 2017-10-31T17:43:11 good question ;-) 2017-10-31T17:44:05 I'm sure I don't want to have them in the same directory as the *.sls files, so maybe a subdirectory like pillar/role/secrets/ or even pillar/role/secrets/wiki/ 2017-10-31T17:45:01 I would say outside of pillar/roles 2017-10-31T17:45:24 how about pillar/secrets and then same path as pillar/... 2017-10-31T17:45:36 eg pillar/secrets/role/proxy.sls 2017-10-31T17:46:04 sounds like a good idea 2017-10-31T17:46:37 what about database passwords - should they live in secrets/role/wiki.sls or secrets/role/mysql.sls? 2017-10-31T17:47:03 (assuming we let salt handle mysql accounts and database creation, which would make sense IMHO) 2017-10-31T17:47:06 *** Son_Goku has quit IRC 2017-10-31T17:47:50 this is a different issue, I'll explain 2017-10-31T17:48:11 there are cases where a pillar value will have to be used by multiple pillar variables 2017-10-31T17:48:37 the db client user password is such a case, where the db server will need it to create the user 2017-10-31T17:48:47 and the webapp will need it to connect to the db server 2017-10-31T17:49:09 exactly 2017-10-31T17:49:16 in puppet there was the possibility to include one value from another variable, by eg setting up a top level pillar file 2017-10-31T17:49:22 with a name eg globals 2017-10-31T17:49:40 but I'm pretty sure it is not supported in salt 2017-10-31T17:49:44 I'll need to check 2017-10-31T17:50:33 well, it shouldn't be to hard to use a mysql.whatever pillar in the wiki, so I don't see a real problem with this 2017-10-31T17:50:51 it willl "just" cause some confusion with the namespace, but that's something we'll survive ;-) 2017-10-31T17:51:14 the problem is that the name of the variable will be different in those cases 2017-10-31T17:51:46 there will be eg 'mysql:wiki_en_user: s3cr3t' and 'webapp:mysql_en_user: s3cr3t' 2017-10-31T17:52:51 we'll have to use mysql:wiki_en_user in the mediawiki setup 2017-10-31T17:53:01 not the perfect solution, but still the one with least headache IMHO 2017-10-31T17:53:53 if we use formulas for both the mysql server and the webapp we won't be that flexible 2017-10-31T17:54:13 right, that will be more interesting[tm] 2017-10-31T17:54:53 so I'll need to check again if we can have the ability to load a pillar value from another pillar variable 2017-10-31T17:55:12 I never tried, but in theory it should work 2017-10-31T17:55:30 maybe the include order matters, but in general it sounds doable 2017-10-31T17:56:03 by quick googling, it's an open issue https://github.com/saltstack/salt/issues/6955 :( 2017-10-31T17:59:26 one guy worarounded it by creating a whatever yaml/json file with that info, and running a script that populates the secrets into the appropriate pillars 2017-10-31T17:59:44 so the information is duplicated in the code, but at least we as admins will have to put it in one place and run a script 2017-10-31T18:01:54 so the basic problem is that you can not do {{ pillar.get($variable) }} from another pillar file 2017-10-31T18:01:59 only from a state 2017-10-31T18:02:04 so we are hitting a limitation 2017-10-31T18:05:00 I scrolled a bit down, and there are two more workarounds: load_yaml and import_yaml 2017-10-31T18:05:51 import_yaml looks like it matches my "one secret per encrypted file" best ;-) 2017-10-31T18:06:54 you get the content as {{variable}} and can place it whereever it makes sense 2017-10-31T18:09:05 give me a few, I'm on the phone 2017-10-31T18:13:14 you'll get more than a few minutes - I have to ;-) leave for a barbecue 2017-10-31T18:17:16 we could try it, but this will mean that the secrets go to all machines I think 2017-10-31T18:17:22 encrypted though, so not a big deal 2017-10-31T18:18:42 if it's read into the pillar data, then I'd hope that it only goes to the machines that receive that pillar data 2017-10-31T18:22:45 *** solevi has joined #opensuse-admin 2017-10-31T18:40:48 *** cboltz has quit IRC 2017-10-31T20:50:28 RECOVERY: MySQL WSREP recv on galera2.infra.opensuse.org - OK wsrep_local_recv_queue_avg = 0.000000 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera2.infra.opensuse.org&service=MySQL%20WSREP%20recv 2017-10-31T20:50:29 RECOVERY: MySQL WSREP recv on galera3.infra.opensuse.org - OK wsrep_local_recv_queue_avg = 0.000000 ; See https://monitor.opensuse.org/icinga/cgi-bin/extinfo.cgi?type=2&host=galera3.infra.opensuse.org&service=MySQL%20WSREP%20recv 2017-10-31T21:11:30 *** plusky has quit IRC 2017-10-31T21:18:58 *** plusky has joined #opensuse-admin 2017-10-31T21:18:58 *** plusky has joined #opensuse-admin 2017-10-31T22:17:34 *** kl_eisbaer has left #opensuse-admin 2017-10-31T22:27:01 *** fvogt has quit IRC 2017-10-31T22:32:36 *** Son_Goku has joined #opensuse-admin 2017-10-31T22:36:34 *** Son_Goku has quit IRC 2017-10-31T22:45:33 *** Son_Goku has joined #opensuse-admin 2017-10-31T22:52:14 *** Son_Goku has quit IRC 2017-10-31T22:57:14 *** Son_Goku has joined #opensuse-admin 2017-10-31T23:39:21 *** nicolasbock has quit IRC