16:00:14 <dcavalca> #startmeeting Hyperscale SIG
16:00:14 <centbot> Meeting started Wed May 12 16:00:14 2021 UTC.  The chair is dcavalca. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:14 <centbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:00:18 <dcavalca> morning folks
16:00:50 <jvreeland> morning
16:01:17 <dcavalca> #chair dcavalca jvreeland
16:01:17 <centbot> Current chairs: dcavalca jvreeland
16:01:22 <dcavalca> let's wait a couple of minutes as usual
16:02:31 <anitazha> hello
16:02:38 <Eighth_Doctor> hey!
16:02:49 <dcavalca> alright, I think we have a quorum
16:02:51 <dcavalca> #topic Followups
16:03:19 <dcavalca> I have one followup from last week, which is that I've scheduled the monthly hangout / hack day in the centos calendar
16:03:45 <dcavalca> first one is May 19 3pm PDT
16:03:56 * michel_slm regretfully missing this. Have to go get someone shot *with vaccines
16:04:02 <dcavalca> there's a link to the VC bridge in the centos calendar entry
16:04:10 <dcavalca> ping me if you'd like an actual calendar invite too
16:04:13 * Eighth_Doctor got shot up with vaccine yesterday :)
16:04:56 <dcavalca> #info scheduled monthly VC hangout in the centos calendar
16:05:24 * dcavalca got his second shot last Mon :)
16:05:40 <dcavalca> anything else for followups?
16:06:21 <dcavalca> k, let's move to
16:06:23 <dcavalca> #topic Announcements
16:06:37 <dcavalca> #info CentOS Dojo is tomorrow!
16:06:42 <Eighth_Doctor> woot
16:06:58 <dcavalca> I have a hyperscale SIG update talk, I'll put the slides in the issue later today
16:07:10 <dcavalca> if there's stuff you'd like to highlight lemme know
16:07:45 <Eighth_Doctor> libvirt is built in hyperscale now :)
16:07:59 <dcavalca> yay
16:08:07 <dcavalca> actually, that should go in the minutes
16:08:16 <dcavalca> #info libvirt is built in hyperscale now
16:08:22 <Eighth_Doctor> #link https://cbs.centos.org/koji/buildinfo?buildID=32902
16:09:02 <dcavalca> #info anitazha released a systemd 248 build yesterday
16:09:19 <dcavalca> this also comes with a backport of less from Fedora, as it's a dependency of the new systemd
16:09:44 <dcavalca> I've tested the new systemd on a stock c8s VM and it seems to work fine, including the selinux stuff
16:09:52 <Eighth_Doctor> yay
16:09:59 <anitazha> nice
16:10:13 <Eighth_Doctor> new libvirt should be fine on c8s too
16:10:37 <Eighth_Doctor> I haven't specifically tested the CBS build, but I've been building it for el8 for a while, and upstream tests against el8 continuously
16:11:01 <dcavalca> Eighth_Doctor: might be worth it to setup some CI for libvirt on openshift as well
16:11:08 <Eighth_Doctor> yeah for sure
16:11:16 <dcavalca> it's complex enough that I wouldn't want to track regressions manually
16:11:21 <Eighth_Doctor> oh blech
16:11:26 <Eighth_Doctor> I just tried installing it
16:11:30 <Eighth_Doctor> swtpm-tools is missing :(
16:11:41 <dcavalca> Eighth_Doctor: you're in luck, I just built that for EPEL8 last week :)
16:11:51 <Eighth_Doctor> nice!
16:12:15 <Eighth_Doctor> that should handle that nicely
16:12:39 <Eighth_Doctor> when our buildroot includes epel, then our mock configs can require it too
16:12:39 <dcavalca> https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-d32a6ed9c9 looks like it's in stable already
16:12:51 <Eighth_Doctor> ahh, I tested with the c8s-hyperscale mock root
16:12:55 <Eighth_Doctor> which has no epel
16:13:12 <dcavalca> yeah I was waiting for the infra ticket to get sorted out before adding it there
16:13:25 <Eighth_Doctor> makes sense
16:13:40 <dcavalca> #info LLVM 12 packages in CBS
16:13:50 <dcavalca> I've built a set of llvm/clang/etc in CBS tracking LLVM 12
16:14:02 <dcavalca> it "works" in the sense that the tests are passing and I can compile hello.c
16:14:07 <Eighth_Doctor> lol
16:14:14 <dcavalca> waiting to get some signal from our compiler folks before I tag a release
16:14:32 <dcavalca> this is just temporary, once we get LLVM 12 in centos proper as part of 8.5 I'll probably drop it
16:14:41 <Eighth_Doctor> we probably should see if we can get the annobin and hardening flag stuff from Fedora backported too if we keep doing this
16:15:06 <Eighth_Doctor> I'd really like to have the distro-wide hardening features available if we're backporting clang
16:15:49 <dcavalca> Eighth_Doctor: I'm trying to figure out if FB actually has clang/llvm stuff on top of mainline they want to publish
16:16:06 <dcavalca> if that's the case, we'll probably maintain it in the SIG long term (or at least until it's actually mainlined)
16:16:15 <Eighth_Doctor> right
16:16:25 <Eighth_Doctor> if we're just going to drop it soon, then it doesn't matter
16:16:37 <Eighth_Doctor> but if we wind up maintaining a clang tree, then we should get that stuff backported from Fedora too
16:16:42 <dcavalca> yeah, agreed
16:16:53 <dcavalca> I'll ping people again this week and see what they want to do here
16:17:09 <Eighth_Doctor> excellent
16:17:35 <dcavalca> the only other thing I had here is rbowen's post in centos-devel asking for feedback on SIG process and tooling
16:17:44 <dcavalca> y'all should read that and reply with your feedback
16:17:59 <Eighth_Doctor> I think at this point I'm going to write some tiny wrappers for SIG workflows
16:18:01 <Eighth_Doctor> it sucks right now
16:18:11 <Eighth_Doctor> and I nearly built the wrong thing by accident this morning
16:18:21 <dcavalca> yeah, likewise
16:19:05 <dcavalca> Eighth_Doctor: I briefly looked at adding support to centpkg but it won't work, as it's just a thin wrapper on rpkg
16:19:15 <dcavalca> so we probably need a separate tool here
16:19:23 <Eighth_Doctor> rpkg supports git.c.o now
16:19:27 <Eighth_Doctor> it has for a year or so
16:19:36 <dcavalca> Eighth_Doctor: does it support the centos layout though?
16:19:40 <Eighth_Doctor> yes
16:19:44 <Eighth_Doctor> it was merged in 8 months ago
16:19:56 <dcavalca> oh cool, that that's probably the best option here I think
16:20:27 <rbowen> Thanks to everyone that has responded to the SIG feedback email, both on and off list. :)
16:20:52 <Eighth_Doctor> rbowen: hey :)
16:21:28 <dcavalca> anything else for announcement?
16:21:29 <Eighth_Doctor> dcavalca: https://pagure.io/rpkg/c/a95bad2ed57e500a9f9c1ec50d1c5cdacffca5f4?branch=master
16:21:57 <dcavalca> nice
16:21:58 <Eighth_Doctor> DistGitLayout and SRPMLayout are supported
16:22:03 <Eighth_Doctor> the latter being the CentOS one
16:22:29 <dcavalca> yeah so I guess it's a matter of hooking that up, and then maybe adding the right config for cbs
16:23:12 <dcavalca> alright, let's move to
16:23:17 <dcavalca> #topic Tickets
16:23:37 <dcavalca> for #47 add EPEL to our build tags we're currently waiting on infra
16:23:50 <dcavalca> there's been a good discussion on -devel about this
16:24:09 <ignatenkobrain> Hi folks, sorry for being late
16:24:15 <Eighth_Doctor> hey Igor Raits
16:24:17 <dcavalca> hi ignatenkobrain
16:25:00 <dcavalca> we already talked about llvm for #46
16:25:01 <steelcowboy> I'm also here, just lurking lol
16:25:21 <dcavalca> for #40, the systemd daily CI is working well so far
16:25:26 <dcavalca> I should probably close this issue actually
16:26:05 <dcavalca> jvreeland: any update on the kernel stuff for #7 ?
16:27:07 <dcavalca> I do have a kernel related update on my end, which is that I've build kpatch in the SIG
16:27:30 <dcavalca> stock centos doesn't ship kpatch-build for some reason, but it was pretty easy to turn that back on
16:27:34 <Eighth_Doctor> nice
16:27:53 <dcavalca> my kernel folks tell me it seems to work, though I don't have any first hand experience with it
16:28:05 <jvreeland> Not particularly, the kernel builds and works. looking at differences between what we have internally and what is there now
16:28:16 <jvreeland> i saw kpatch and wanted to verify thats working.
16:28:32 <jvreeland> Do we need to wait for the signing to release to experimental?
16:28:38 <dcavalca> the secure boot issue is also still pending with infra
16:28:47 <dcavalca> jvreeland: I think experimental is probably fine now
16:29:11 <Eighth_Doctor> we're probably going to need to use the carl approach for releasing kernel package
16:29:14 <Eighth_Doctor> *packages
16:29:28 * jvreeland cool, i think that'd be helpful to figure out what changes people might want/need. We also need to come up with a plan for how we'll manage the linux repo for backports/features
16:29:41 <jvreeland> oh, sorry swapped clients and didn't realize that was a hotkey
16:30:16 <dcavalca> jvreeland: for the linux repo, I was thinking of having a CI job to keep the main branch in sync with Linus
16:30:33 <jvreeland> dcavalca: yeah i've been looking at your systemd scripts for that
16:30:41 <dcavalca> and then have release branches for our releases (e.g. release/5.10)
16:30:44 <Eighth_Doctor> and we can have a hyperscale branch off the tag
16:30:49 <Eighth_Doctor> please don't use slashes though :)
16:30:54 <dcavalca> and feature branches if we need to stage/devel stuff
16:31:08 <dcavalca> Eighth_Doctor: yeah I was just parroting what we do internally, but I'm fine with whatever here :)
16:31:28 <Eighth_Doctor> I just don't want git giving me confusing output because we used slashes in the branch names :D
16:31:37 <dcavalca> yeah that's fair
16:32:23 <dcavalca> did I miss anything else for tickets?
16:32:29 <jvreeland> hyperscale-5.10 seems reasonable. Does the normal pull request workflow work on pagure? I remember issues on the rpm repos
16:32:37 <Eighth_Doctor> yes
16:32:40 <Eighth_Doctor> it works fine there
16:32:40 <dcavalca> jvreeland: yeah PRs work fine
16:33:08 <Eighth_Doctor> I will likely have a source tree for libvirt in our namespace soon too
16:33:19 <dcavalca> Eighth_Doctor: can we get something going to make scratch builds off PRs on pagure.io?
16:33:20 <Eighth_Doctor> depends on how ugly things get with upstream work
16:33:25 <dcavalca> that would be cook for things like kernel and systemd
16:33:26 <Eighth_Doctor> dcavalca: I don't see why not
16:33:38 <dcavalca> I know anitazha had been asking for that before
16:33:43 <Eighth_Doctor> we have the CI thingamajiggies
16:33:49 <Eighth_Doctor> so we can wire things up for that as we please
16:34:08 <dcavalca> cool, so that's something else to look into then
16:34:14 <Eighth_Doctor> main issue is credentials for CBS, which you've already got taken care of sort of
16:34:41 <dcavalca> Eighth_Doctor: yeah the bot account was remarkably easy to set up actually
16:35:56 <Eighth_Doctor> I think we're going to want CI producing images
16:36:02 <Eighth_Doctor> I want something with all the stuff put together
16:36:20 <dcavalca> Eighth_Doctor: you mean cloud images, or something else?
16:36:27 <dcavalca> I don't think we can do full composes on openshift
16:36:30 <Eighth_Doctor> install ISOs and cloud images
16:36:42 <Eighth_Doctor> both of which should be possible in OpenShift based CI
16:36:50 <dcavalca> Eighth_Doctor: so for install ISOs, I know there was a plan for that, as I remember bstinson mentioning it a while ago
16:36:57 <dcavalca> for cloud images, with should work with Cloud SIG
16:37:02 <dcavalca> * we
16:37:03 <Eighth_Doctor> yup
16:37:15 <Eighth_Doctor> we'll need to get those anaconda + lorax + pykickstart things done for that though
16:37:18 <dcavalca> but install ISOs also have the same signing issue I think
16:37:24 <dcavalca> so we need to sort that out first
16:37:33 <dcavalca> yep, that too
16:38:07 <Eighth_Doctor> we should probably make that a priority since we have the other things humming a long
16:38:09 <Eighth_Doctor> *along
16:38:50 <dcavalca> steelcowboy: ^^ did you get to test the pykickstart stuff?
16:39:40 <steelcowboy> I haven't yet!
16:40:35 <steelcowboy> But will follow up internally with some q's about that (just where and how we can use hyperscale rn in prod)
16:40:51 <dcavalca> steelcowboy: sounds good
16:41:04 <dcavalca> alright, let's move to
16:41:07 <dcavalca> #topic Membership
16:41:18 <dcavalca> I don't think we have anything here?
16:41:24 <Eighth_Doctor> not really, no
16:41:35 <dcavalca> then we go to
16:41:37 <dcavalca> #topic Misc
16:41:52 <dcavalca> rbowen: do we need to reschedule this meeting to not conflict with the office hours?
16:42:22 <rbowen> No. I have fixed the office hours calendar entry.
16:42:30 <rbowen> It was correct on https://www.centos.org/community/calendar/#CentOS_Stream_Office_Hours_(Google_Meet,_not_IRC)
16:42:40 <rbowen> But somehow made it into my Google calendar for the wrong time.
16:42:44 <rbowen> Time zones are hard.
16:42:50 <Eighth_Doctor> 😆
16:42:54 <dcavalca> rbowen: ok, so we'll keep Hyperscale at 9am PDT and the office hours at 10am PDT
16:43:33 <Eighth_Doctor> question: are we going to need our own branding packages for Hyperscale SIG produced media?
16:43:54 <dcavalca> what do other SIGs do?
16:43:54 <Eighth_Doctor> or at least a `centos-stream-release-hyperscale` package so that we're declared as the Hyperscale variant?
16:44:00 <Eighth_Doctor> nobody does anything we do :)
16:44:02 <dcavalca> actually, is there any other SIG making media?
16:44:05 <Eighth_Doctor> we're breaking new ground here
16:44:15 <dcavalca> oh ok
16:44:22 <dcavalca> Eighth_Doctor: we already have a release package
16:44:32 <dcavalca> currently that just installs the repos
16:44:41 <Eighth_Doctor> In Fedora-land, spins have to get their own os-release data in fedora-release
16:45:04 <dcavalca> I don't know anything about branding, and personally I don't really care, so I'm fine doing whatever folks agree with there
16:45:11 <Eighth_Doctor> e.g. Fedora Server, Fedora Cloud, and Fedora KDE all have their own os-release files with their variants set
16:45:17 <Eighth_Doctor> CentOS SIGs have been mostly dead in the water until last year, so I dunno what we do here
16:45:50 <Eighth_Doctor> I don't care too much other than whether we need it to be easier to identify that it's a Hyperscale variant over the normal one when installed or booted from images
16:46:05 <dcavalca> Eighth_Doctor: wanna file a ticket for the board on this?
16:46:09 <Eighth_Doctor> sure
16:46:14 <dcavalca> this sounds like a policy question, which is probably within their purview
16:46:22 <Eighth_Doctor> yeah
16:47:03 <Eighth_Doctor> we're actually using SIGs for what they're for, which is something CentOS has never dealt with before :)
16:47:07 <dcavalca> #action Eighth_Doctor file a board ticket about hyperscale branding
16:47:31 <rbowen> I am working on (re)writing the SIG process/policy documents. I appreciate you asking hard questions, and hope that you can be patient with us as we try to figure this out.
16:47:48 <rbowen> I just got done with the Board governance docs, and SIGs are next.
16:47:58 <dcavalca> I think this is a learning experience for everybody involved, I'm sure we'll figure it out
16:48:02 <Eighth_Doctor> excellent
16:48:08 <Eighth_Doctor> yeah, I figured nobody knows the answers
16:48:16 <Eighth_Doctor> I just figured I'd ask as we go along so we can think this through
16:48:57 <Eighth_Doctor> it occurred to me that we might need to given that our variant will be very different from CentOS itself
16:49:07 <Eighth_Doctor> updated kernel, patched anaconda, fresh systemd, etc.
16:49:42 <dcavalca> Eighth_Doctor: my only concern with this is that I don't want to give the impression that hyperscale is a fork
16:49:51 <dcavalca> as that's precisely what we're trying to avoid here :)
16:49:57 <Eighth_Doctor> right, that's why I would only propose to set `VARIANT` in the os-release data
16:50:10 <Eighth_Doctor> we'd still call ourselves CentOS Stream 8 and have no other visible delta
16:50:19 <dcavalca> but yeah, I think a way to denote a variant/flavor/whatever is fine
16:50:30 <Eighth_Doctor> I don't even want to branch artwork :)
16:51:14 <Eighth_Doctor> with the dnf better counting stuff, we could even use it to gather metrics on how well our variant is received as a whole
16:52:31 <dcavalca> oh I didn't even know there was metrics collection for centos
16:52:37 <dcavalca> is that like the popcon stuff in debian?
16:52:39 <Eighth_Doctor> no
16:52:46 <Eighth_Doctor> we don't have it for packages, just os-release variants
16:53:21 <Eighth_Doctor> that's where this stuff comes from: https://twitter.com/mattdm/status/1392133014629519366
16:53:29 <dcavalca> ah got it
16:54:50 <dcavalca> we have 5 min left, any other topics folks wanna discuss?
16:54:56 <Eighth_Doctor> I got nothing :)
16:55:49 <dcavalca> I think that's it then
16:55:52 <dcavalca> have a good one folks!
16:55:55 <dcavalca> #endmeeting