17:01:55 <tdawson> #startmeeting CentOS PaaS SIG
17:01:55 <centbot> Meeting started Wed Jun 14 17:01:55 2017 UTC.  The chair is tdawson. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:01:55 <centbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
17:02:02 <tdawson> #topic roll call
17:02:24 <tdawson> _ari_ has already let me know that he will not be here.
17:02:51 * jdetiber waves
17:03:00 <tdawson> Hi jdetiber
17:03:05 <sdodson> hello
17:03:21 <tdawson> Hi sdodson
17:05:44 <tdawson> #chair tdawson jdetiber sdodson
17:05:44 <centbot> Current chairs: jdetiber sdodson tdawson
17:05:52 <tdawson> #topic OpenShift on CentOS Current Status
17:05:57 <tdawson> Starting off with rpms
17:06:46 <tdawson> To be honest, I have no rpm updates.
17:07:16 <tdawson> 3.6 isn't out, and I don't know of any other things that might need to be built.
17:08:48 <tdawson> Anyone else have any rpm information?  Or know of any rpms that need to be built?
17:09:20 <tdawson> OK, moving on to Documentation
17:09:55 <sdodson> Sorry, there's a new 3.6 alpha that was tagged this week. I'm not sure when the RC will be tagged but it shouldn't be too long
17:10:14 <sdodson> https://github.com/openshift/origin/releases/tag/v3.6.0-alpha.2
17:10:17 <tdawson> The week before we got all the Documentation fixed up, so this week there isn't any Documentation stuff that I know of.
17:10:22 <tdawson> sdodson: OK
17:10:28 <sdodson> fine with moving on to docs
17:10:46 <tdawson> sdodson: I'll talk to _ari_ and see if he wants me to build it or if he wants to use it for automation practive.
17:10:52 <tdawson> practice
17:11:34 <sdodson> sounds good
17:12:00 <tdawson> Anything else?
17:12:16 <tdawson> sdodson: If I do build it, is there a recommended openshift-ansible to go with it?  Or just the latest?
17:12:28 <sdodson> tdawson: just the latest 3.6
17:12:50 <tdawson> OK
17:12:58 <tdawson> Any other rpm issues/news ?
17:13:56 <sdodson> not from me
17:14:01 <tdawson> Any other Documentation news?
17:14:23 <tdawson> Moving on to Automated rpm building and Automated testing
17:14:36 <tdawson> _ari_ isn't here, but gave me a brief report.
17:14:51 <tdawson> herlo got LinchPin 1.0.1 releases this past week.
17:14:55 <tdawson> Fixed several bugs there.
17:15:20 <tdawson> _ari_ has been swamped, so the automation hasn't moved too much
17:15:38 <tdawson> sk on _ari_'s team has started doing some work.
17:15:57 <tdawson> He didn't have enough time to brief me on exactly how much was done.
17:16:17 <tdawson> We are expecting more of a report next week.
17:16:39 <tdawson> And hopefully, one way or another 3.6 alpha2 will be built.  Hopefully via automation.
17:16:51 <tdawson> Otherwise they will need to practice on the rc
17:17:29 <tdawson> I think that's all I have for automation
17:17:35 <tdawson> Moving on to Images and Image building
17:17:50 <tdawson> Looks like they are starting to work on building for aarch64
17:18:12 <tdawson> There were a few emails this week asking about our aarch64 rpm builds.
17:18:34 <tdawson> We have no documentation on it, because nobody has tried it yet.
17:18:51 <herlo> thanks tdawson for pointing that out. It's all I've got.
17:19:05 <herlo> tdawson: oh, and my blog post is in review.
17:19:14 <tdawson> herlo: Your welcome ... and keep up the good work.
17:19:27 <tdawson> herlo: When it's done, be sure to ping me and I'll put it up on the page.
17:19:35 <herlo> will do, thankx
17:19:53 <jdetiber> tdawson: is there any hardware available to test the aarch64 builds/containers on?
17:20:06 <tdawson> jdetiber: I'm not sure
17:20:16 <tdawson> jdetiber: I guess you could always throw it on a rpi3
17:20:26 <jdetiber> I'd be happy to give it a go if someone can provide hardware, I just don't have any aarch64 hardware locally.
17:20:43 <jwb> rpi3 would be... bad
17:20:44 <jdetiber> tdawson: do we have a way to do 64bit on the pi3? I thought we only supported 32bit
17:20:46 <tdawson> jdetiber: From the sounds of the emails, they are trying to build aarch64 images on an x86_64 machine ... which I didn't think you could do.
17:21:09 <tdawson> jdetiber: No, we only support 64 bit, which is why it hasn't worked on the pi2's
17:21:15 <jdetiber> tdawson: k8s does all of their alt arches using cross compiling on x86_64
17:21:25 <tdawson> Although the new pi2's have 64 bit capability ... but might as well get a pi3 anyway.
17:21:45 <tdawson> jdetiber: But is cross compiling the same as docker building
17:22:04 <jdetiber> tdawson: you can cross-build the docker images as well, it's generally done under qemu
17:22:10 <tdawson> I pointed them at our aarch64 origin builds, but didn't have any information on their docker build's failling.
17:22:45 <tdawson> jdetiber: That makes sense
17:23:41 <tdawson> I like doing things on real hardware though .. but that's me ... never was the best as qemu customization ... I had one arm config that worked, and anytime I tweeked it, terrible thigns happened.
17:23:45 <jdetiber> jwb: rpi3 could probably be done as long as it was a multi-node cluster with etcd split from the api server
17:24:12 <jwb> jdetiber: i think you're going to find it to be very slow and memory constrained
17:24:30 <jwb> but ignoring that, who knows.  maybe it would work
17:24:31 <jdetiber> jwb: most definitely
17:25:06 <jwb> tdawson: is the meeting on the multi-arch topic?
17:25:17 <tdawson> jwb: Sure
17:25:21 <jwb> heh
17:25:33 <tdawson> jwb: It sorta combined with image building, since image building was starting to happen on aarch64
17:25:34 <jwb> short intro: Hi, I'm Josh.  I do multi-arch stuff at Red Hat
17:25:49 <tdawson> jwb: awesome ... welcome to the meeting.
17:26:11 <jwb> i saw a question around openshift, centos, and aarch64.  thought i would pop in and see what that was about.
17:26:15 <jwb> and ask a question
17:26:39 <jwb> i think i have the gist of aarch64.  so i'm curious if any other architectures are being looked at.  e.g. ppc64le?
17:27:19 <tdawson> jwb: Yes, we tried ppc64le and origin didn't build on it.  When talking around about the error I was told that it was a golang bug that wouldn't get fixed until golang 1.8
17:27:43 <jdetiber> tdawson: we are using golang 1.8 internally, would need to do the same for CentOS as well
17:27:45 <tdawson> jwb: But that's been several golang 1.7 versions ago, and for all I know, the bugs been fixed already in 1.7
17:27:59 <jwb> pretty sure it still requires 1.8
17:28:07 <tdawson> jdetiber: Are we?  I thought there was a 1.8 bug that was preventing us from using it.
17:28:24 <jwb> so does CentOS build their own golang?
17:28:24 <jdetiber> tdawson: we are for the ppc64le builds
17:28:39 <tdawson> jwb: Yes/No ...
17:28:45 <tdawson> jdetiber: Ahh ... ok
17:28:54 <jwb> there's a performance issue with golang 1.8 and kube in general that i know of.  it's been fixed internally
17:29:08 <tdawson> jwb: I'll be honest, I don't know about CentOS proper, but we (the Paas SIG) have been building our own.
17:29:20 <jwb> then there's another issue with a behavior change in 1.8 that can introduce races.  not sure where that is in regards to origin
17:29:29 <jwb> i think clayton knows about it
17:29:41 <jwb> tdawson: ok, that's good (i think?)
17:30:10 <jwb> anyway, i don't mean to derail your meeting.  i was just curious what the overall goal was for multi-arch and this SIG
17:30:16 <tdawson> jwb: jdetiber: So the latest 1.7 goland doesn't fix the ppc64le build problem I take it?  Or should I give it another try?
17:30:30 <jwb> i don't believe it does, no
17:30:38 <tdawson> jwb: It's not derailling, it's right on topic actually.
17:31:16 <tdawson> We've had the multi-arch builds for a while (month or two) and nobody has asked about them, so I haven't done any more work.
17:31:39 <jwb> looks like some container pipeline thing is now asking?
17:31:53 <jwb> i'm curious if they know what they're getting into
17:31:58 <tdawson> jwb: Yep
17:32:13 <tdawson> jwb: That, I have no idea ... they are brave souls.
17:33:10 <jwb> origin doesn't deal with manifest list images to my knowledge, which means any container pipeline that is building for multi-arch either has to build arch specific named containers (e.g. foo-aarch64), namespace them somehow (e.g. project/aarch64/foo), or have a separate registry per arch or something
17:33:33 <tdawson> So, if ya'll want, I can get a ppc64le build of golang and see if we can build origin on ppc64le again.
17:33:48 <jwb> that would be neat
17:34:00 <jdetiber> jwb: I think even with the manifest list images you still need to do arch specific naming
17:34:20 <tdawson> It would only be a scratch build, that's all the permissions I have for ppc64le right now, but we could verify that things build.
17:34:33 <jwb> jdetiber: i'm told manifest list images are the truth, light, and way to NOT have to have arch specific naming
17:34:46 <jwb> jdetiber: in that it's basically a fat image with all arches, and the client pulls what is needed
17:35:09 <jdetiber> jwb: my understanding is that thee manifest list images are basically links to the arch specific images. But I may be going off of outdated info :)
17:35:52 <jwb> jdetiber: blind leading the blind here :).  either way, origin doesn't support them
17:36:07 <tdawson> :)
17:36:16 <jdetiber> :)
17:36:40 <tdawson> So ... if you were to have an aarch64 or ppc64le openshift cluster .... does that mean there would be no mixing allowed with other arches?
17:37:05 <jwb> tdawson: fwiw, the go version we have built internally is 1.8.3 and is generally working well-ish.  there's another bug being worked on right now preventing openshift 3.6 from building
17:37:29 <jdetiber> jwb: that is the linking bug?
17:37:48 <jdetiber> tdawson: single arch per cluster only for right now
17:37:54 <jwb> tdawson: right.  homogeneous within the cluster for now.  though jdetiber just pointed me to an upstream kube talk about mixing
17:37:59 <jwb> jdetiber: correct
17:38:06 <tdawson> Good to know
17:38:11 <jdetiber> tdawson: jwb: mixing would require manifest list image support :)
17:38:34 <jdetiber> at a minimum
17:38:52 <tdawson> Which is still expermental/developmental, not totally baked, if I have heard right.  Correct?
17:39:19 <jdetiber> it would need to be supported by our docker version, registry version, image streams, etc
17:39:29 <jdetiber> not to mention the images themselves
17:39:47 <tdawson> So.. still got a ways to go it sounds like.
17:39:54 <jdetiber> indeed
17:40:11 <jwb> i think docker supports consuming them, but creating them with 'docker build' doesn't work.  there's a separate tool needed
17:40:22 <jdetiber> that as well
17:40:35 <jdetiber> I wasn't sure about the version of docker that we ship/support
17:40:56 <jwb> no idea what CentOS has :)
17:42:06 <jdetiber> tdawson: are we still shipping docker in the OpenShift repo?
17:42:13 <tdawson> docker-1.12.6-28
17:42:15 <tdawson> jdetiber: No
17:42:30 <tdawson> jdetiber: Well, there is an older version still in there, but no, we're getting it from extras
17:42:39 <jdetiber> +1
17:42:59 <tdawson> We only supplied it because at the time, we had to have a patch in for ... I don't remember, but for something.
17:43:38 <tdawson> Unless we have something else multi-arch sepecific, I'd live to move on to Minishift
17:43:49 <jwb> nope, thanks for your time
17:44:05 <tdawson> marcindulak: lalatenduM: Any minishift news / updates ?
17:44:26 <tdawson> jwb: Glad you were here, good to see interest in multi-arch starting to pick up.
17:46:50 <tdawson> I didn't see any minishift email this week, so I don't think there has been a new release.
17:47:21 <tdawson> I also don't see any minishift rpms in cbs, so I don't know what the rpm progress is for it either.
17:47:25 <marcindulak> I don't have any more minishift RPM news. Last time we stopped at discussing whether minishift can bundle vendor dependencies, or maybe another way to get all sources of the dependencies (parsing glide.lock and creating a tarball?))
17:48:12 <tdawson> marcindulak: OK, thanks for the update.
17:49:05 <tdawson> marcindulak: I'm hoping that minishift decides to bundle the vendor dependencies, but I have no pull in that community ... but if I did, I'd give it a +1
17:49:43 <tdawson> Then I think we'll move onto the Open Floor
17:49:59 <tdawson> #topic Open Floor
17:50:39 <tdawson> I talked to Brenton about my replacement in his group also replacing me here.
17:51:12 <tdawson> It's all going to depend on who replaces me, so we'll hopefully know in a month or two ... hopefully not 3.
17:51:43 <jdetiber> tdawson: hopefully my replacement can continue to be a participant as well :)
17:51:56 <tdawson> jdetiber: That's right, you've changed groups as well.
17:52:26 <tdawson> jdetiber: But in theory, this will still be applicable to your new group too, won't it?
17:52:28 <jdetiber> tdawson: I will be sticking around, though. The multi-arch efforts overlap with my new role
17:52:52 <tdawson> jdetiber: That's good to hear.
17:54:23 <tdawson> I sent out an email this week, and thus far, there hasn't been anyone rushing forward to take my place as the chairman ... so I guess I'll be leading these meetings for a couple more months.
17:54:35 <tdawson> Anything else before we close the meeting?
17:55:25 <tdawson> We'll talk to ya'll next week.
17:55:30 <tdawson> Have a good week.
17:55:36 * jdetiber waves goodbye
17:55:49 <tdawson> #endmeeting