14:00:04 <bstinson> #startmeeting CBS/Infra
14:00:04 <centbot> Meeting started Mon Nov 10 14:00:04 2014 UTC.  The chair is bstinson. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:04 <centbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:00:23 <bstinson> #chair MerlinTHP alphacc Arrfab kbsingh
14:00:23 <centbot> Current chairs: Arrfab MerlinTHP alphacc bstinson kbsingh
14:00:29 <bstinson> #topic Agenda
14:00:35 <bstinson> #info Topic: Status Updates
14:00:40 <bstinson> #info Topic: Old Business: Ad-Hoc Upstream Development Repositories
14:00:57 <bstinson> #info Topic: Open Floor
14:01:14 <bstinson> hi everyone
14:01:19 <Arrfab> hey bstinson
14:01:21 <MerlinTHP> ello
14:01:28 <alphacc> hi all
14:01:39 <bstinson> #topic Status Updates
14:02:04 <bstinson> Arrfab: you had something regarding the requierments list?
14:02:42 <Arrfab> bstinson: for Central Auth ? yes : MerlinTHP was supposed to write it but I just collected some ideas and start a wiki page so that everybody can subscribe/track/edit
14:02:59 <MerlinTHP> OK
14:03:13 <MerlinTHP> I'll work out if I can edit the wiki, then merge in the stuff I've got
14:03:16 <Arrfab> #info wiki page around Centralized Authentication : http://wiki.centos.org/InfraWiki/CentralizedAuth
14:03:21 <bstinson> great!
14:04:04 <mikem> good morning
14:04:08 <Arrfab> but the more I think about it, the more I think that IPA is the way to go, as long as we don't use the integrated dns feature, nor expose the KDC to the outside world
14:04:35 <Arrfab> MerlinTHP: I can add you to that page in the acl, as soon as I know your wiki login
14:04:37 <MerlinTHP> OK, so on the IPA side, I've been told that we'll need something sooner rather later, so I've been working on the not-a-PoC version of the IPA front end.
14:04:41 <MerlinTHP> Arrfab: ok
14:05:11 <MerlinTHP> It's not ready yet, but people can start laughing at my python code at https://github.com/merlinthp/MyPA
14:05:19 <Arrfab> MerlinTHP: cool, you have access to a Poc machine, right ? so is there something we can see/test "live" ?
14:05:25 <alphacc> MerlinTHP: great
14:05:27 <MerlinTHP> Yep.
14:06:03 <kbsingh> woo
14:06:09 <MerlinTHP> One of the things I'll try and prioritise is fleshing out the README with architecture similar.
14:06:11 * kbsingh reads backlog
14:06:42 <MerlinTHP> #info Work in progress IPA front end code at https://github.com/merlinthp/MyPA
14:07:03 <bstinson> nice! let's bring MyPA up as an example when we talk about Ad-Hoc Upstreams later
14:07:09 <MerlinTHP> Sure
14:07:15 <Arrfab> MerlinTHP: good README.md about the fact that the default HBAC "allow all" exist : I've been surprized the first time I tested ipa-server myself :-)
14:07:21 <MerlinTHP> Arrfab: :)
14:07:32 <MerlinTHP> Yeah, that's a good thing to IMMEDIATELY DISABLE
14:07:36 <Arrfab> yeah
14:07:58 <MerlinTHP> I can see why it exists.  Without it IPA would look like it didn't work when you first tried it.
14:08:30 <MerlinTHP> And the number of times I've forgotten to add a new system to a hosts group, so I can't log in, then wonder why...
14:08:31 <Arrfab> so on a side note, we discussed about the fact that ipa-server would be the one included in el7/c7 meaning that , if we start with c7, those nodes will be managed "manually' as the current puppet version in centos.org doesn't support it (yet)
14:08:50 <MerlinTHP> Yeah, I'm doing all my dev work on c7
14:09:30 <MerlinTHP> Anyway, hopefully I'll get some more work on it done over the next few days, depending how much time the new Halo takes up
14:09:53 <bstinson> :)
14:10:10 <MerlinTHP> That's me, really.
14:10:56 <bstinson> anything else for status updates?
14:11:08 <kbsingh> I've done some work on the gitblit upgrade as well
14:11:25 <MerlinTHP> Is this the one that'll fix the extra / issue?
14:11:55 <kbsingh> yeah, that should get fixed - but more importantly, this also has the username/branchname mappings
14:12:07 <MerlinTHP> Ah :)
14:12:09 <kbsingh> I need to do a bit more work to make sure the failover machine is all setup to take up load and then we can flip over
14:13:02 <kbsingh> at some point in the near future we should get dev.git.centos.org up again and use that to work on the auth integration stuff
14:13:25 <MerlinTHP> *nod*
14:13:51 <alphacc> #info I have been testing the iamge creation from koji with imcleod. It seems it works fine with 1.9.0
14:14:00 <kbsingh> the other big win from the gitblit upgrade is that the new api has a bunch more features in there
14:14:18 <kbsingh> once that is done, we can then bring back the ban on people randomly scraping the site from top to bottom
14:14:42 <bstinson> that would be handy
14:14:43 <kbsingh> at the moment, were doing millions of hits a day, most of which just seem bizaree
14:14:49 <bstinson> alphacc: woohoo!
14:15:35 <alphacc> there are some more tests needed for atomic images, hope we can continue this week.
14:15:55 <Arrfab> alphacc: good news
14:16:07 <alphacc> iamge factory very well integrated with koji, hope the docker plugin will work too.
14:16:11 <bstinson> #info centpkg hasn't moved forward much this week, but kbsingh gave me access to the repo (which I still need to test)
14:16:46 <alphacc> imcleod is the plugin owner so I expected to see details with him.
14:16:57 * imcleod perks up
14:17:11 <imcleod> alphacc: Looking for docs/details of the image building in koji?
14:17:23 <imcleod> alphacc: Was going to send something about that to centos-devel today.
14:18:11 <alphacc> imcleod: excellent, please do.
14:18:26 <imcleod> alphacc: Is the code on the CBS newer than what we worked with on Friday afternoon?
14:18:56 <alphacc> imcleod: no not yet.
14:19:06 <alphacc> imcleod: let's talk about it after the meeting
14:19:30 <kbsingh> so ifyou guys are changing code on there - be good to see a note on centos-devel with highlights of what changes
14:19:44 <kbsingh> we should try and do that all around really, just as a good practise
14:20:00 <bstinson> +1
14:20:05 <alphacc> yes
14:20:23 <imcleod> kbsingh: Roger.  The work on Friday was not a code change, just feature-enablement (which is not actually a word, according to my spell checker).
14:21:19 <kbsingh> heh
14:21:39 <kbsingh> a slightly related question - what machine are you going to use for the image builders ?
14:21:56 <kbsingh> is that just a regular koji job so the regular builder will do its thing, or do / did we need something special there
14:22:04 <alphacc> kbsingh: so far we have 1 builder, bl7.
14:22:07 <MerlinTHP> It's the former
14:22:16 <MerlinTHP> It's just another thing kojid does
14:22:28 <imcleod> Correct, though it does require some additional packages to be installed.
14:22:33 * MerlinTHP nods.
14:22:39 <alphacc> and it has a channel associated so we can choose builder that do only this task
14:22:45 <imcleod> And it works _much_ better if the builder in question has HW accelerated virt enabled.
14:22:52 <MerlinTHP> I'd quite like to get some roundtuits to properly integrate mashing as a koji task too
14:23:08 <imcleod> MerlinTHP++
14:23:16 <alphacc> MerlinTHP: +1 I am looking at that too
14:23:24 <MerlinTHP> mash as it is is a horribly inefficient process.
14:23:49 <MerlinTHP> alphacc: ok, remind me sometime and I'll pull together somes notes on the hacks I made to $work's mash
14:24:03 <imcleod> dgilmore has had "optimizing mash" as a long term TODO for some time now, I believe.
14:24:03 <MerlinTHP> It's not in-koji, but it's a lot faster than normal mash
14:24:38 <bstinson> speaking of roundtuits, MerlinTHP: can you link me the upload.cgi stuff you posted a few weeks ago?
14:24:46 <MerlinTHP> Yeah, sure
14:24:46 <alphacc> MerlinTHP: Ok I'll share our stuff too
14:24:53 <imcleod> MerlinTHP: I'd be quite interested in those as well.  I was browsing the code on the flight back from Paris and had some thoughts.
14:24:56 <kbsingh> so: something to track: we have potentially more machines we can bring into builder roles
14:25:26 <kbsingh> ~  if / when we hit a point where people need to wait > 30 min for their package to build start, then we should visit that and see if we can bring in more h/w
14:25:41 <MerlinTHP> #action MerlinTHP to write/share some notes on more efficient mash
14:26:08 <alphacc> #action Identify more builders and install them with puppet.
14:26:11 <kbsingh> secondly, lets setup a section on wiki.c.o that everyone here can use to track / document / edit / howto / tips nad tricks etc around the buildsystem
14:26:19 <MerlinTHP> kbsingh: otoh, that builder is a fairly beasty blade, it can run a bunch of builds in parallel
14:26:35 <MerlinTHP> bstinson: here ok? http://www.merlinthp.org/centos/lookaside/
14:26:36 <kbsingh> MerlinTHP: yeah, so lets only look at more hardware when we need to, not needed right now
14:26:40 <mattymo> Evolution, are you around?
14:26:45 <MerlinTHP> bstinson: or do you want a mail or something?
14:27:05 <alphacc> kbsingh: I'd like to go forward with a puppet install if we can have additional blade. What do you think Arrfab ?
14:27:06 <bstinson> MerlinTHP: that's perfect, i'm going to put that in a repo somewhere
14:27:42 <alphacc> so we are ready when we need more hw.
14:27:45 <kbsingh> ther might be some dancing involved in finding more blades, but it can be done . i dont want to derail this meeting though
14:27:53 <Arrfab> alphacc: all for it
14:28:41 <MerlinTHP> Arrfab: do you want a ticket re: wiki perms?
14:29:01 <Arrfab> MerlinTHP: no, don't think so .. just give me your login name :-)
14:29:10 <Arrfab> no need to track ACL right for wiki :-)
14:29:12 <MerlinTHP> HowardJohnson
14:29:14 <MerlinTHP> ta
14:29:40 <bstinson> ok, anything else for status updates?
14:29:53 <Arrfab> #action arrfab to give permission to HowardJohnson to the CentralizedAuth wiki page
14:30:10 <MerlinTHP> ta
14:30:12 <Arrfab> bstinson: don't think so
14:30:33 <bstinson> #topic Old Business: Ad-Hoc Upstream Development Repositories
14:30:39 <kbsingh> if we are still doing updates, i have a question re: lookaside uploader.
14:30:45 <MerlinTHP> Shoot
14:30:47 <kbsingh> are ew blocked on having a cert / auth layer in place
14:30:52 <kbsingh> or are we able to push ahead with that
14:31:02 <MerlinTHP> Not really blocked, we can use the existing koji CA
14:31:12 <kbsingh> its a primary blocked ( not having the lookaside uploaders ) to getting mass git usres onboard
14:31:24 <MerlinTHP> I was going to ask you if what I'd implemented was ok for user-branch mapping
14:31:36 <kbsingh> i didnt know what you implemented and where
14:31:39 <MerlinTHP> And, obviously, failed
14:31:45 <MerlinTHP> Yeah, that too
14:32:04 <MerlinTHP> If you've got a few minutes after this meeting?
14:32:57 <bstinson> MerlinTHP, kbsingh: if you're going to work on that, i might leave the repo creation to you
14:33:04 <bstinson> but we should get that in git somewhere
14:33:28 <MerlinTHP> There's a repo on git.c.o it should go in
14:33:38 <kbsingh> today is call mayhem monday.. but we can try and work something out
14:33:44 <MerlinTHP> cbs-tools
14:33:57 <MerlinTHP> kbsingh: ok, we'll have a go
14:34:00 <alphacc> +1
14:34:16 <kbsingh> maybe we can use email to work this out - on the centos-devel list
14:34:21 <MerlinTHP> TBH, it's 2:30 in the afternoon and I'm still eating my lunch
14:34:24 <MerlinTHP> kbsingh: k
14:34:30 <kbsingh> i just had my lunch at 1:55 :)
14:34:56 <imcleod> Thanks to jet lag, I feel like eating lunch but it is only 9 AM.
14:35:19 <Arrfab> ah, it's that moment in the meeeting when we start discussing about lunch .. I guess next step is beer :-p
14:35:20 <bstinson> i'm about 1/4 of the way through my first coffee (the day is looking much brighter)
14:35:38 <MerlinTHP> Arrfab: Timmermans, please.
14:36:04 <Arrfab> bstinson, MerlinTHP, kbsingh, imcleod (and all others too) : feel free to read/edit/comment the requirement list on the wiki page around IPA/FAS please
14:36:23 <Arrfab> #link http://wiki.centos.org/InfraWiki/CentralizedAuth
14:36:32 <bstinson> Arrfab: cool
14:36:32 <MerlinTHP> Will do
14:36:35 <imcleod> Arrfab: Many thanks.  Will do.
14:36:52 <bstinson> Let's talk about Ad-Hoc upstreams
14:37:09 <Arrfab> I had some calls with some of the people involved in FAS, so we can "use" that knowledge, even if we want to use IPA
14:37:31 <MerlinTHP> Cool.
14:37:49 <Arrfab> like the FedOAuth system, written by Fedora guys, but being able to use multiple backends providers, so FAS, but also kerberos/ldap/other -> IPA ready too :-)
14:38:15 <kbsingh> bstinson: in principle i think we all agreed on having them
14:38:29 <kbsingh> bstinson: i guess the bits we need to workout are how to use them and hoto consume various parts of these upstreams
14:38:43 <Arrfab> meaning than most things supporting OAuth (probably phpbb for the forums and I have to verify for Mantis/bugs.c.o) would be able to integrate with IPA
14:38:56 <kbsingh> my understanding initially was that people would keep their dist-git repos in rpms/ project and use the upstreams stuff in their own sig-specific project area for adhoc content
14:39:26 <kbsingh> and what that does seem to be fine, people also want adhoc dist-git git repos in the upstreams areas - and to me thats confusing as to where koji is going to get its git tree's from
14:39:59 <kbsingh> eg. if someone wants to build an rpm from sig-blahblah/kernel.git; how is that going to map to what we already have in terms of expectations ( ir. branch names / sig names etc )
14:40:05 <MerlinTHP> If people want ad-hoc branches in g.c.o dist-git, that's one thing.
14:40:08 <bstinson> hmmmm, i'm not sure if we want to consume arbitrary dist-git repos
14:40:15 <MerlinTHP> We don't.
14:40:22 <MerlinTHP> Not least because it requires koji config.
14:40:25 <bstinson> i think if it builds in the buildsystem, it should be under rpms/ in our branch layout
14:40:26 <MerlinTHP> Plus... just no.
14:40:50 <kbsingh> arbitary.. as in its still on git.centos.org, just in the sig's own project rather than in rpms/
14:41:23 <Evolution> I think /rpms should stay separate for rh->push into the project.
14:41:37 <Evolution> but there should be something similar for project specific upstream/packages
14:42:00 <bstinson> so we need something like /sig-rpms?
14:42:19 <Evolution> well, it looks very much like we're going to be the upstream for software collections
14:42:34 * MerlinTHP puts a paper bag over their head
14:42:37 <Evolution> if we're doing one, we should plan for others.
14:44:10 <kbsingh> the other problem with dist-git is that we have no way to upload non text sources.
14:44:34 <kbsingh> and the user acl will map all rpm repos to force signame -> branch name
14:45:26 <kbsingh> so in a nutshell, even if we were to allow this - and realistically, if we have generic git repos,users can store whatever - even dist-git style stuff without needing any special foo
14:45:42 <kbsingh> just that they will need to use the srpm route to build in cbs.centos.org - there wont be any direct link from koji
14:45:58 <Evolution> how terrible would two separate git instances be? git.centos.org for official sources. rh-> us
14:46:17 <Evolution> then dev.centos.org or projects.centos.org or whatever for sig/upstream/etc git?
14:46:54 <kbsingh> for ths scl upstreams they dont need a direct koji link up do they ?
14:47:24 <kbsingh> afaik, they dont even want a git repo per rpm, they want a git repo per collection that totally destroys our ability to build from it
14:47:33 <kbsingh> since each git repo will have upto 200 spec files
14:47:43 <kbsingh> not sure if koji cam handle that
14:48:15 <imcleod> kbsingh: Do you already have patches in koji, or pending, to enable the structure you are proposing?
14:48:31 <imcleod> alphacc: I know you showed me a few things in the commit history on Friday.  Are those related to this?
14:48:36 <kbsingh> no
14:48:42 <kbsingh> also what do you mean by 'this'
14:49:15 <kbsingh> cbs.c.o can consume git repos from the rpms/ project. so i am guessing its already there, mikem was working on that ages ago with alphacc
14:49:19 <imcleod> kbsingh: "this" = The proposed structure for dist-git repos on b.c.o.
14:49:29 <kbsingh> i believe the underlaying rpkg code has support now for the centos lookaside sources as well
14:49:41 <imcleod> kbsingh: b.c.o = git.c.o
14:49:41 <kbsingh> imcleod: not proposed, its in production
14:50:00 <imcleod> kbsingh: Ahh.  OK.
14:50:06 <kbsingh> its where we get our upstream sources as well, so whatever there is now is what we've got for the short haul
14:50:33 <kbsingh> so to recap, and jus make sure we are all on the same page
14:51:10 <kbsingh> the rpms/ project on git.centos.org runs our dist-git stack, and branch names are considered 'sacred?' where branchname = c4/5/6/7/8 maps to the distro content, and sig's can branch to their own name == branch name
14:51:42 <kbsingh> then additionally, we have the signame projects, where they can have git repos for anything, where they get to do whatever branches / tags they want == these are the upstreams git repos
14:51:58 <alphacc> kbsingh, imcleod : we just want to roll our own koji package with the patches. nothing else
14:52:05 <kbsingh> however, we might not be able to link cbs.c.o to these Upstreams - and therefore if anyone wants to build rpms from these repos they need to use the srpm route
14:54:06 <bstinson> been thinking on this for a while, i'm not sure there's any other way
14:54:27 <kbsingh> yeah
14:55:14 <bstinson> of course i'm not particularly familiar with the SCL build process
14:56:51 <MerlinTHP> OK, we're running short of time.
14:57:01 <kbsingh> lets punt this over to the centos-devel list, and use it as a point to get started from
14:57:04 * MerlinTHP coughs and gives bstinson his hat back
14:57:14 <kbsingh> i dont think there is a clear solution to this in the short term that might result in anything different
14:57:17 <bstinson> heh
14:58:02 <bstinson> kbsingh: maybe we can hash out the rest of the use-cases also (and pick some of the low-hanging fruit)
14:58:08 <kbsingh> cool
14:58:21 <kbsingh> Ill get on the SCL list and rattle a few bolts around
14:58:30 <bstinson> our CBS-related tools for example
14:58:51 <bstinson> ok now for a very short...
14:58:54 <bstinson> #topic Open Floor
14:59:23 <MerlinTHP> 60 seconds, go ;)
15:00:08 <bstinson> thanks everyone!
15:00:21 <bstinson> #endmeeting