17:00:27 <tdawson> #startmeeting CentOS PaaS SIG
17:00:27 <centbot> Meeting started Wed Sep 20 17:00:27 2017 UTC.  The chair is tdawson. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:27 <centbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:00:33 <tdawson> #topic roll call
17:00:37 * _ari_ here
17:00:53 <tdawson> Hi _ari_
17:01:04 * _ari_ waves
17:01:17 * jdetiber waves
17:01:25 <tdawson> Hi jdetiber
17:04:08 <tdawson> Does anyone know if sdodson is around today?
17:04:19 * kbsingh waves - lurking
17:04:19 <sdodson> i'm here
17:04:25 <tdawson> hi sdodson
17:04:39 <tdawson> sdodson: I wanted to ask some questions about ansible during the rpm section
17:04:48 <tdawson> hi kbsingh
17:05:44 <tdawson> #chair tdawson jdetiber sdodson _ari_ kbsingh
17:05:44 <centbot> Current chairs: _ari_ jdetiber kbsingh sdodson tdawson
17:05:53 <tdawson> #topic OpenShift on CentOS Current Status
17:06:02 <tdawson> Starting with rpms
17:06:43 <tdawson> As far as I know, we don't need any new openshift packages, other than the automated builds.
17:07:16 <tdawson> ButI've had discussions with the other sigs about whether we'll continue to build the latest ansible or not.
17:07:30 <tdawson> We as in the Paas sig
17:07:52 <tdawson> On the one hand, ansible is now in CentOS extras
17:08:25 <tdawson> But, the rumor is that version won't get updated very often due to Ansible Enterprise (or whatever it's called)
17:08:57 <tdawson> So, the question is, do we continue to build the latest, or do we use the version in Extra's?
17:09:12 <tdawson> sdodson: I think you would know most about this.
17:09:51 <sdodson> Does centos extras automatically import rhel extras changes?
17:10:36 <tdawson> sdodson: As far as I know ... kbsingh would you know if CentOS extras automatically pulls in the latest rhel extras ?
17:10:56 <kbsingh> that is a good question
17:10:59 <sdodson> Because I can't say when or what version, but rhel extras is going to receive at least one update to ansible in the future I believe. I don't know if that may be the last and all subsequent updates go to AE or not.
17:11:16 <kbsingh> things delivered to git.centos.org where the content is in the c7-extras/ branch does get built through to CentOS Extras typically
17:11:19 <kbsingh> did we miss something ?
17:11:36 <tdawson> kbsingh: No, we're just wondering about the future
17:11:46 <tdawson> No, you didn't miss anything
17:12:09 <kbsingh> hughesjr: can confirm, but we do try and track all changes ( eg. docker has been delivered via extras for a while )
17:12:51 <sdodson> tdawson: openshift-ansible doesn't require anything newer than ansible 2.3
17:13:03 <kbsingh> so if ansible were to update, we'd see it here : https://git.centos.org/summary/?r=rpms/ansible - with the c7-branch getting the update
17:13:09 <sdodson> tdawson: If we do end up requiring ansible 2.4 and it's not in centos extras we'll just build that into the origin repos.
17:13:53 <sdodson> kbsingh: ansible- went to rhel extras ~ 14 days ago, but at least in the context of openshift-ansible that's not a critical update
17:13:55 <tdawson> sdodson: So for now, it's ok for us to just stay with what's in extras, and not worry about building the latest greatest ansible?
17:14:08 <sdodson> tdawson: Yeah, I don't think we'll never to change our stance there.
17:14:26 <tdawson> sdodson: OK
17:15:00 <tdawson> I'm sure the other SIG will be a little sad, but they always want the latest asible as soon as it gets released.  This way they'll be able to just do it themselves.
17:16:06 <tdawson> So, stance on ansible is to use what's in extras, and if that isn't good enough at some point in the future, deal with it then.  Sound good?
17:16:30 <sdodson> Yeah, that's fine
17:16:39 <tdawson> Great
17:17:04 <kbsingh> +1
17:17:08 <tdawson> That's the only non-automatic-build rpm subject I have ... anything else need to come up before we move to automation?
17:18:21 <tdawson> OK, moving on to automated rpm building and testing
17:18:32 <tdawson> _ari_:  Do you have an update for us?
17:18:59 <_ari_> tdawson: fixed an issue that jdetiber (thanks)  where the python-dnf dependency in fedora was removed
17:19:12 <_ari_> tdawson: also we were affected by the infra issues.
17:19:48 <_ari_> tdawson: I have started the conversion from JJB to Pipeline based on the changes to cico-pipeline-library and lessons we have learned in the ci-pipeline
17:19:52 <tdawson> _ari_: Yep, I saw that.  We're back up ... correct?
17:20:05 <_ari_> tdawson: yes I checked last night things are green
17:20:11 <tdawson> Nice
17:20:23 <_ari_> tdawson: nothing else from me
17:20:35 <tdawson> So, I had a couple questions
17:21:09 <tdawson> First ... who gets / wants to push the "stable build" button, whenever a new release comes out, especially for the Alpha / Beta origin releases.
17:21:38 <tdawson> Right now all the builds go to scratch, and we only build non-scratch manually ... correct?
17:21:41 <_ari_> tdawson: I could do it but really anyone with access can
17:21:49 <_ari_> tdawson: correct
17:22:05 <_ari_> we could make this automated I think
17:22:35 <tdawson> I've been doing it in the past, and I could do it also, but I'm no longer getting notices of when there are alpha/rc/releases
17:23:11 <tdawson> If people are ok with that stable build being a few days late, I could continue to do it.
17:23:40 <tdawson> While people mull that over ... my second question
17:23:42 <_ari_> tdawson: I think sdodson is the best candidate since he is closest to openshift releases
17:24:25 <tdawson> _ari_ could you show us an example of how we could find the cbs build packages and/or task number, of one of the scratch builds.
17:24:46 <sdodson> I'm fine with doing it whenever an origin release is tagged
17:25:02 <_ari_> sdodson: ++
17:25:37 <tdawson> sdodson: If you wouldn't mind, that would be great.
17:26:11 <_ari_> tdawson: you can go to the output and just search for taskid - https://ci.centos.org/job/paas-bfs-future-matrix/PYTHON=system-CPython-2.7,TOPOLOGY=duffy_3node_cluster,nodes=paas-sig-ci-slave01/629/consoleFull
17:26:13 <tdawson> sdodson: If, for some reason you don't have the time, feel free to ping me ... but like I said, I'm outa the loop, and it will be a few days
17:26:49 <_ari_> tdawson: if it is easier I could put it in the output for the build so it is visible
17:27:24 <tdawson> _ari_: That might be nice
17:28:09 <kbsingh> is there a way to have this available via an automated call ?
17:28:13 <_ari_> tdawson: I can work on that as part of the pipeline conversion so we can have more info in the display
17:28:15 <kbsingh> eg. latest/ build id or something
17:29:35 <_ari_> kbsingh: it is crude but you could just do https://ci.centos.org/job/paas-bfs-future-matrix/PYTHON=system-CPython-2.7,TOPOLOGY=duffy_3node_cluster,nodes=paas-sig-ci-slave01/lastStableBuild/consoleFull
17:30:24 <kbsingh> _ari_: ok, might haveto try that
17:30:28 * tdawson bookmarks that
17:30:51 * kbsingh makes a note to ask tdawson
17:31:10 <_ari_> tdawson: kbsingh permalinks are on the job page
17:31:30 <tdawson> I guess I can put something like that in our documentation .... we don't have anything on our build cycle.
17:31:41 <_ari_> curl https://ci.centos.org/job/paas-bfs-future-matrix/PYTHON=system-CPython-2.7,TOPOLOGY=duffy_3node_cluster,nodes=paas-sig-ci-slave01/lastSuccessfulBuild/ | grep taskID
17:31:54 <_ari_> tdawson: we will get better on the display long term
17:32:18 <kbsingh> ok
17:32:31 <tdawson> _ari_: Yep.  And I understand it's a process ... I was just taking a long time trying to find the latest build.
17:32:51 <kbsingh> I'm looking at doing some sort of nightly bringups, jfchevrette has got a proper cluster deployed now from a 3 day old build, but i want to try and automate parts of it
17:32:51 <tdawson> _ari_: Thanks, I'll get that in our build page ... while I'll setup.
17:33:00 <_ari_> tdawson: latest successful is the latest link you could find others as well
17:33:09 <_ari_> cool
17:33:20 <kbsingh> you know what would rock is being able to take a git short rev, and use that as a way to find the corrosponding rpms
17:33:42 * kbsingh lurks
17:33:45 <tdawson> _ari_: Understood ... some people might want the latest failed, and things like that ...
17:33:52 <jdetiber> kbsingh: are we not adding the git short rev to the version number in the paas sig builds?
17:34:05 <kbsingh> jdetiber: how would i search with that ?
17:35:16 <jdetiber> kbsingh: within jenkins, I'm not sure, but for rpms, you could do a grep on yum list available --show-duplicates
17:35:47 <jdetiber> well, at least for published RPMs...  unpublished, I'm not so sure
17:36:07 <tdawson> Anything else about automation before we move on to mult-arch?
17:36:25 <_ari_> none from me
17:37:15 <tdawson> Moving on to multi-arch
17:37:21 <jdetiber> I've been slowly starting on migrating my internal pipeline over to ci.centos.org: https://ci.centos.org/view/PaaS-SIG/job/paas-multiarch-test/
17:37:21 <tdawson> jdetiber: How are things on that front?
17:37:50 <jdetiber> I have a patched version of python-cicoclient that I am using to make things a bit easier (does key lookup in the home directory of the slave)
17:38:18 <jdetiber> it also has the multi-arch additions that are in master but not in the published version
17:38:46 <jdetiber> I just got provisioning/deprovisioning to work, now to get the actual build/test pipeline changes migrated over
17:38:53 <tdawson> jdetiber: Do we need to bring that into the main Paas repo's, or should that still be in the multi-arch branch?
17:39:13 <tdawson> /that/the patched version of python-cicoclient/
17:39:23 <jdetiber> I've also created the openshiftmultiarch org to publish the release/base images until those changes make it into upstream origin as well
17:39:40 <jdetiber> tdawson: I'm just building from repo currently, no need to publish as of yet
17:39:46 <tdawson> jdetiber: ok
17:39:47 <bstinson> jdetiber: if you have a good patch for the api key, we'd love to review it
17:40:05 <bstinson> as far as multiarch is concerned, i'm going to cut a release soon (probably friday)
17:40:06 <jdetiber> tdawson: I'm hoping that we can get those changes merged into the upstream repo and an official release cut if possible
17:40:29 <tdawson> very nice
17:40:33 <jdetiber> bstinson: PR is up on the python-cicoclient github currently :)
17:41:03 <bstinson> aha! shows you how much i pay attention to github
17:41:31 <jdetiber> tdawson: otherwise, still chugging along slowly but surely on upstreaming the build/test changes needed and keeping a working branch that is mergeable onto origin master for applying those changes for testing
17:41:58 <jdetiber> bstinson: figured I'd wait a bit to start bugging people about it ;)
17:42:28 <tdawson> jdetiber: bstinson: And as I always ask ... s390x ?
17:42:57 <bstinson> i'm actually meeting with a few folks this week about options. nothing to report yet
17:43:03 * tdawson needs it for the giant mainframe he keeps in the corner of his bedroom.
17:43:12 <jdetiber> bstinson: woohoo
17:43:14 <bstinson> tdawson: can we borrow it? :)
17:43:33 <tdawson> bstinson: I'd let you borrow it ... except it's imaginary ... my imaginary server.
17:43:55 <tdawson> My wife won't let me keep a real mainframe in the house.
17:44:18 <tdawson> Anything else multi-arch before we move on to documentation?
17:44:50 <jdetiber> tdawson: nothing from me
17:44:53 <tdawson> Moving on to documentation
17:45:05 <tdawson> Beofre the meeting started, I didn't have anything for documentation
17:45:33 <tdawson> But I guess, now that automation has progressed so far, it's time to write something up on the basics of how we do our builds.
17:46:15 <tdawson> So, that's a task for me next week, get that started.
17:46:28 <tdawson> Anything else for documentation?
17:46:40 <tdawson> Any blogs or articles that need to be linked to?
17:47:28 <tdawson> Very well, moving on to Images and Image building
17:47:53 <tdawson> Do any of the lurkers have anything they want to talk about for images and/or image building?
17:48:31 <tdawson> lalatenduM: ping - Do you have anything on minishift you want to talk about?
17:49:33 <tdawson> tkral28: ping - Did you have any progress on kompose that you would mind sharing with the Paas SIG ?
17:49:58 <tdawson> I hope I didn't just wait up either of them.
17:50:20 <tdawson> ok, moving on to Open Floor
17:50:34 <tdawson> Does anyone have anything to talk about that we haven't already discussed?
17:50:59 <sdodson> I'd like to follow up on whether or not it's ok for my team to go ahead and publish a centos based AMI with origin ready to bootstrap.
17:51:19 <sdodson> Both, short term we just do it, and longer term whether or not we should try to drive that build through this SIG.
17:52:10 <jdetiber> sdodson: I think ideally we would be able to leverage the automated builds to generate the AMI if possible
17:53:42 <tdawson> sdodson: Short term, where would you build it?
17:54:11 <sdodson> tdawson: Just one of the developers on my team would build it and push it to a limited number of regions
17:55:54 <tdawson> _ari_: You were talking weeks back about doing testing, and that would involve building images, would any of those be AMI images?  or just docker images?
17:55:55 <sdodson> We want to make the AWS provisioning via bootstrapped nodes as frictionless as possible so that it should just work from a github checkout of openshift-ansible.
17:56:33 <_ari_> tdawson: this was just docker images and I thought someone was focused on that to allow us to call what they are doing?
17:57:28 <_ari_> tdawson: we are able to provision instances in AWS with linchpin AFAIK
17:58:19 <_ari_> then again it sounds like there is a CI/CD team around this effort already which begs the question should they own the work for SIG-PaaS?
17:58:35 <tdawson> Very good point
17:59:00 <_ari_> Not that I don't enjoy this as much as you do but this isn't our day jobs AFAIK
17:59:05 <tdawson> sdodson: We're almost out of time ... let's bring this up via email and/or next weeks meeting.
17:59:11 <tdawson> :)
17:59:16 <_ari_> sounds good
17:59:26 <sdodson> there is no CI/CD around this, once makes it past incubation we'd like to have an AMI built each time there's an origin release.
17:59:32 <hughesjr> tdawson: for the things that show up in c7-extras on git.centos.org, I build and push them to 7/extras/
17:59:40 <sdodson> but right now we just want to build one and get it out there for people to try.
18:00:09 <tdawson> hughesjr: Awesome ... thanks for verifying that.
18:00:32 <tdawson> OK, we're out of time ... don't want to go over the next meeting.
18:00:38 <sdodson> that's fine
18:00:41 <tdawson> Talk to ya'll next week
18:00:48 <tdawson> #endmeeting