16:00:13 <dcavalca> #startmeeting Hyperscale SIG
16:00:13 <centbot> Meeting started Wed Apr 28 16:00:13 2021 UTC.  The chair is dcavalca. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:13 <centbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:00:18 <dcavalca> morning everybody
16:00:28 <anitazha> 'morning
16:00:35 <dcavalca> gonna wait a few minutes in case folks are watching the centos session at redhat summit
16:01:03 <dcavalca> #chair dcavalca jvreeland
16:01:03 <centbot> Current chairs: dcavalca jvreeland
16:01:20 <jvreeland> hello
16:02:07 <dcavalca> alright, looks like session has wrapped up, let's get started
16:02:15 <dcavalca> #topic Followups
16:02:30 <dcavalca> we had two things here from the last meeting
16:02:57 <dcavalca> Michel booked an afternoon to hack around anaconda, which yielded a pykickstart build in the SIG
16:03:23 <dcavalca> in general, we found these sessions useful, so I'm considering booking a regular monthly for them
16:04:06 <jvreeland> sounds usefull
16:04:32 <dcavalca> #action dcavalca book monthly hack sessions
16:05:01 <dcavalca> King_InuYasha: has setup https://pagure.io/centos-sig-hyperscale/package-bugs to hold package bug reports
16:05:19 <dcavalca> we still need to add tags there for all the packages, and then file a ticker for infra to update bugurl in our tags so they point to that repo
16:05:35 * Eighth_Doctor waves
16:05:40 <dcavalca> Eighth_Doctor: welcome, we were just taking about you :)
16:06:07 <Eighth_Doctor> well I didn't realize I was kicked from everything
16:06:07 <dcavalca> I think that's it for followups
16:06:15 <Eighth_Doctor> yay netsplits
16:06:32 <Eighth_Doctor> what'd I do?
16:06:47 <michel_slm> hello all!
16:07:16 <dcavalca> Eighth_Doctor: I was calling out https://pagure.io/centos-sig-hyperscale/package-bugs
16:07:31 <Eighth_Doctor> oh right
16:07:41 <Eighth_Doctor> I didn't finish setting it up, but at least it exists
16:07:59 <dcavalca> yeah once the tags are all setup we'll need to get the bugurl updated on our tags
16:08:54 <dcavalca> alright, let's move to
16:09:03 <dcavalca> #topic Announcements
16:09:33 <dcavalca> #info we have a gitlab repo for secrets
16:10:07 <dcavalca> add your gitlab username to the ticket if you need access
16:10:15 <dcavalca> https://pagure.io/centos-sig-hyperscale/sig/issue/45
16:10:46 <michel_slm> ah yeah, meant to do this and forgot
16:10:56 <Eighth_Doctor> you're already added, iirc
16:11:04 <Eighth_Doctor> I'm pretty sure I added you from the beginning
16:11:26 <michel_slm> yeah, saw Conan Kudo commenting. did you add `michel-slm` ?
16:11:44 <Eighth_Doctor> yes
16:12:05 <dcavalca> #info we have access to the OpenShift CentOS CI
16:12:07 <dcavalca> https://console-openshift-console.apps.ocp.ci.centos.org
16:12:10 <michel_slm> ok, I have it (wow, haven't logged in for ages)
16:12:34 <dcavalca> right now jvreeland and I are admins there, if you want/need access, ask in https://pagure.io/centos-infra/issue/291
16:12:41 <Eighth_Doctor> ooh fancy
16:13:04 <dcavalca> note that this uses the Noggin credentials, despite the misleading splash screen that looks like the old ACO
16:13:04 <Eighth_Doctor> do I have that access yet?
16:13:23 <dcavalca> Eighth_Doctor: not sure, you may wanna ping siddharthvipul
16:13:29 <dcavalca> I haven't seen any update on that ticket
16:13:33 <Eighth_Doctor> will do
16:14:14 <dcavalca> #info we have daily systemd builds kicked off from the CI
16:14:35 <dcavalca> the "code" for this is in https://pagure.io/centos-sig-hyperscale/systemd-releng
16:14:43 <Eighth_Doctor> those aren't being released automatically, right?
16:14:55 <dcavalca> they're tagged -testing and end up on buildlogs
16:15:04 <dcavalca> on the FB side, I'm using cbs download-build to consume them
16:15:24 <Eighth_Doctor> perfecto
16:15:31 <michel_slm> oomd for centos!
16:15:39 <dcavalca> note that these are built off the systemd git master
16:15:46 <dcavalca> so I would recommend not running them in production :)
16:15:47 <Eighth_Doctor> once we start doing more of this, I think we'll move things around to have a -ci target
16:16:15 <Eighth_Doctor> I fully expect to want to do the same eventually for transactional-update and libvirt
16:16:20 <dcavalca> yeah, we can do that
16:16:37 <dcavalca> ideally we'd keep the same build tags and just get new destination tags, but I'm not sure if that's a thing in CBS
16:16:44 <dcavalca> we can ask Infra when the time comes
16:16:47 <Eighth_Doctor> indeed
16:16:58 <Eighth_Doctor> I suspect that tag separation doesn't exist in CBS like it does in Fedora Koji
16:17:05 <dcavalca> #info also moved systemd-git-update to the CentOS CI
16:17:22 <dcavalca> that's the job that keeps https://pagure.io/centos-sig-hyperscale/systemd in sync with github
16:17:27 <dcavalca> sources are also in the releng repo
16:17:44 <dcavalca> #info tpm2-tools and tpm2-tss available in the SIG
16:17:58 <dcavalca> I had to rebuild these two for an internal project, so I figured we might as well add them here
16:17:59 <Arrfab> dcavalca: you can always ask for some tags and then simply 'tag-build' between tags
16:18:14 <Arrfab> that's how SIGs are reusing pkgs built/maintained by other SIGs :)
16:18:22 <dcavalca> thanks Arrfab, good to know
16:18:29 <michel_slm> #info pykickstart available in the SIG
16:18:40 <michel_slm> (since it was not in the last meeting)
16:18:52 <Eighth_Doctor> yay :)
16:18:56 <michel_slm> this is so we can run workflows that use ksflatten and ksvalidate to modify Fedora images, on CentOS
16:19:07 <michel_slm> I need to package lorax now because the c8s copy doesn't have mkksiso
16:19:15 <Eighth_Doctor> hah
16:19:22 <Eighth_Doctor> that means anaconda too
16:19:50 <Eighth_Doctor> though it might be better to ask for mkksiso to be in official c8s lorax first
16:19:56 <michel_slm> not if the intention is to just use mkksiso right?
16:20:06 <michel_slm> yeah, I'll ask first, good point
16:20:19 <michel_slm> bugzilla?
16:21:18 <dcavalca> #info CentOS Dojo on May 13-14
16:21:22 <Eighth_Doctor> Michel Alexandre Salim: yes
16:21:23 <dcavalca> https://wiki.centos.org/Events/Dojo/May2021
16:21:27 <michel_slm> the patched version that's in https://github.com/facebookincubator/linux-desktop-kickstarts does seem to work (we added it before the fixed lorax hit Fedora) on CentOS, so that one binary does not depend on updating Anaconda
16:21:34 <dcavalca> I'll put up slides for the hyperscale talk once I actually write them :)
16:21:45 * michel_slm needs to submit his talk draft for approval
16:21:55 <dcavalca> oh yeah I should do that too
16:21:57 <michel_slm> dcavalca: hehe, I guess you're going to ask for forgiveness? :)
16:22:04 <dcavalca> meh, nobody cares :)
16:22:09 <michel_slm> I calculate today is the last day to send it out if your talk is on Thursday
16:22:24 <Eighth_Doctor> I wish we had been further along or I would have done something for transactional update :(
16:22:35 <Eighth_Doctor> but we can't do any btrfs things yet
16:22:41 <dcavalca> btw, I have a one hour slot at Dojo
16:22:47 <dcavalca> there is no way I'm gonna have an hour of content
16:22:51 <michel_slm> I'm going to add btrfs to my desktop talk wishlist
16:22:55 <dcavalca> so if any of you want to drop in and talk about stuff, be my guest
16:23:02 <michel_slm> can I also add upgrading between major versions or is that too much :p
16:23:18 <Eighth_Doctor> no, I think it's reasonable to think about major version upgrades
16:23:20 <michel_slm> yeah, I will probably run out at about 30 mins too
16:23:34 <Eighth_Doctor> I could go on and on about transactional update stuff :D
16:23:37 <michel_slm> surely we can backport dnf system-upgrade and see what breaks :)
16:23:48 <Arrfab> talking about upgrade
16:23:59 <Arrfab> #info cbs migration next week
16:24:04 <Eighth_Doctor> I might be able to do libvirt without modularity stuff (I did a test build with hyperscale sig mock config)
16:24:06 <michel_slm> ooh. to c8?
16:24:15 <dcavalca> oh right, thanks Arrfab for the reminder
16:24:17 <Eighth_Doctor> so we might have that this week :)
16:24:31 <Arrfab> moving all to 8-stream so should fix https://pagure.io/centos-infra/issue/233
16:24:37 <dcavalca> https://lists.centos.org/pipermail/centos-devel/2021-April/076792.html
16:24:40 <Eighth_Doctor> sweet!
16:24:41 <dcavalca> ^^ for the minutes
16:25:32 <michel_slm> ooh, no ppc64/armhfp
16:25:37 <dcavalca> 6am UTC is 11pm PDT, so it should be fine for us
16:25:43 <Eighth_Doctor> yeah I'm good with that
16:25:58 <Eighth_Doctor> I'm still asleep at 2am EDT
16:26:50 <dcavalca> let's move to
16:26:54 <dcavalca> #topic Tickets
16:26:59 <Arrfab> michel_slm: no SIG is consumming these architectures anymore (at least from SIG PoV)
16:27:24 <dcavalca> #47 add EPEL to our build tags
16:27:27 <michel_slm> Arrfab: yeah, I don't personally mind
16:27:39 <michel_slm> what does adding EPEL do?
16:27:46 <dcavalca> so, this has come up a couple of times before, the tl;dr is that we have a few things that need dependencies from EPEL to build properly
16:27:48 <michel_slm> oh we can depend on EPEL packags
16:27:56 <Eighth_Doctor> It means that if we add packages to EPEL, we don't need to rebuild crap in CBS
16:28:01 <dcavalca> notably the latest rpm (which pulls in fsverity-utils-devel) and clang 12 (which needs some python thing)
16:28:30 <dcavalca> I'm personally fine with depending on EPEL, as I think anybody willing to consume our packages would very likely be using it already
16:29:08 <dcavalca> if nobody has objections, I'll file an infra ticket to get this started
16:29:32 <Eighth_Doctor> let's do it!
16:29:35 <Arrfab> dcavalca: infra ticket ? can you elaborate ?
16:29:51 <dcavalca> Arrfab: I'm assuming we'll need to add EPEL to the build tags for this to actually work
16:29:51 <Eighth_Doctor> Arrfab: our build tags have to be reworked to add EPEL to the external repo list
16:30:03 <Arrfab> dcavalca: hmm
16:30:15 <dcavalca> and then update our release package to pull in epel-release
16:30:19 <Eighth_Doctor> yup
16:30:25 <Arrfab> that was against the initial policy for cbs, reason why SIGs are rebuilding pkgs from epel branch through cbs
16:30:34 <Eighth_Doctor> `Requires: epel-release = %{centos}`
16:30:37 <Eighth_Doctor> err
16:30:42 <Eighth_Doctor> `Requires: epel-release = %{rhel}`
16:31:00 <Eighth_Doctor> Arrfab: I personally do not want to rebuild EPEL packages in CBS
16:31:14 <Eighth_Doctor> because it incentivizes not doing anything in EPEL in the first place
16:31:38 <Arrfab> a ticket is fine but just to start a thread with board
16:32:11 <dcavalca> Arrfab: yeah, happy to have this conversation wherever is appropriate
16:32:13 <Arrfab> as if you build in cbs pkg today with epel and try same build in two weeks, you'll end up with different buildroot but no way to easily rebuild it
16:32:25 <Eighth_Doctor> that's a problem now anyway
16:32:37 <michel_slm> this might be of concern to other SIGs, but Hyperscale is meant to move fast, so.. is that really an issue?
16:32:39 <Eighth_Doctor> because CBS depends on composes via external repo
16:32:45 <Arrfab> note that it's an issue that myself (through infra sig btw) I'm facing, reason why we build and then tag in our own tags
16:33:02 <Arrfab> michel_slm: good point
16:33:04 <Arrfab> *but*
16:33:15 <Eighth_Doctor> the downside of having all these different Kojis not merged means that we're always having this problem
16:33:16 <dcavalca> I can see this being a problem for SIGs targeting CentOS Linux, but I don't really see how EPEL would make things work when targeting Stream
16:33:17 <Arrfab> koji builders can't access external mirrors
16:33:20 <dcavalca> it's already a moving target
16:33:28 <dcavalca> * worse
16:33:38 <michel_slm> ahh. what counts as external?
16:33:41 <Arrfab> so the only way to implement this would be to have internal epel mirror
16:33:51 <Arrfab> michel_slm: anything not in isolated vlan for cbs :)
16:33:53 <Eighth_Doctor> Michel Alexandre Salim: probably IP boundary
16:34:08 <Arrfab> only node that kojid builders have a route for is git.centos.org
16:34:23 <michel_slm> oof
16:34:41 <Arrfab> michel_slm: just to ensure that at build time, nothing is fetched from external
16:35:24 <Arrfab> anyway, let's first have thread about policy around this and if there is agreement, see what/how to implement it :)
16:36:00 <dcavalca> #action dcavalca file infra ticket to add EPEL to build tags
16:36:15 <Arrfab> I remember even a thread I started long time ago about having a bot building epel automatically in internal tags in cbs, so that people could just tag-build on demand
16:36:24 <Arrfab> but it never had momentum
16:37:12 <dcavalca> Arrfab: that seems a bit wasteful, but if there's no other option I guess it could work
16:37:31 <dcavalca> anyway, let's continue this discussion in the ticket
16:37:46 <Arrfab> dcavalca: external repo would work and be faster, but yeah, ticket and even thread on centos-devel for visibility
16:38:05 <dcavalca> oh good call, I'll post to -devel too
16:38:08 * Arrfab is called for dinner /me &
16:38:14 <dcavalca> next up is #46 branch and build clang 12
16:38:31 <Eighth_Doctor> we have also another advantage in that Hyperscale SIG is not a Red Hat product development team in disguise :)
16:38:52 <dcavalca> some context here: FB wants to use clang for our internal kernel builds
16:38:57 <Eighth_Doctor> which means we don't have the hidden requirement that we can't use EPEL
16:39:05 <Eighth_Doctor> eeeegh
16:39:07 <Eighth_Doctor> oh man
16:39:12 <dcavalca> clang 12 initially, but our compilers people have a bunch of patches they want to backport eventually
16:39:14 <Eighth_Doctor> you want to use clang for kernel builds?!
16:39:31 <michel_slm> we use clang for everything else :p
16:39:35 <Eighth_Doctor> man, that's going to be so much suck
16:39:49 <dcavalca> Eighth_Doctor: well, not myself personally, but yeah, our kernel folks want to use it
16:39:53 <dcavalca> I can ask for more context if needed
16:40:06 <michel_slm> oh, huh, that would be interesting how it interplays with having a community LTS kernel
16:40:13 <michel_slm> I guess that kernel would ideally build using both gcc and clang
16:40:25 <dcavalca> I don't necessarily think we should build the hyperscale kernel with clang
16:40:52 <dcavalca> specifically for clang in c8s, there's some modularity fun involved
16:40:52 <Eighth_Doctor> dcavalca: there's going to be a _lot_ of subtle breakage to fix for using clang built kernels
16:41:15 <Eighth_Doctor> I've spent years working in a Linux distribution that builds everything with Clang, it was miserable
16:41:20 <dcavalca> but from my initial tests a non-modular build of clang and llvm should work
16:41:47 <dcavalca> Eighth_Doctor: yeah, I don't have a dog in this race; I'm not even sure whether this is meant for production builds or just for testing tbh
16:41:49 <Eighth_Doctor> but if that's the road you wanna go down, I'm just gonna say it's gonna be a bumpy ride...
16:42:41 <dcavalca> I'm also trying to get our compiler folks to publish their llvm tree somewhre
16:42:53 <dcavalca> as they have a bunch of stuff that's not upstream yet (though it should be submitted soon)
16:44:26 <dcavalca> anyway, if anybody wants to help out (or wants to try these packages once they become available), lemme know
16:44:42 <dcavalca> let's see what's next
16:44:49 <dcavalca> we already talked about #40
16:45:11 <dcavalca> jvreeland: did you hear anything back from pbisaacs for #31?
16:46:25 <jvreeland> Not in a few weeks. I'll send one last one and then  in a week or two we can close it if there's no response?
16:47:49 <jvreeland> *one last email
16:48:15 <dcavalca> works for me
16:48:32 <dcavalca> jvreeland: any update for #7 Modern/Stable/Scalable kernel?
16:49:15 <jvreeland> Kernel builds an runs fine from what I can tell. Happy to get input on the config or the way i've modfiied the fedora packaging.
16:49:28 <jvreeland> Don't really know how to presue signing
16:49:58 <jvreeland> *pursue   and it doesn't build from the git repo artifacts yet just the upstream kernel.org source tarballs
16:50:24 <jvreeland> Pretty small things to fix i think other than the signing which i'm not really sure where to start for secure boot
16:52:02 <Eighth_Doctor> dcavalca: do we have a ticket about the secure boot stuff yet?
16:52:08 <dcavalca> no, but we should
16:53:30 <Eighth_Doctor> can you file that?
16:54:12 <jvreeland> I can file it if someone tells me where it should be filed, don't  want to overload dcavlca
16:54:32 <dcavalca> thanks jvreeland
16:54:34 <dcavalca> https://pagure.io/centos-infra/issues
16:54:47 <dcavalca> #action jvreeland file infra ticket for kernel signing for secure boot
16:55:05 <jvreeland> I also have questions about kabi compatibility, which i've turned off for these builds. It's basically do we care as a sig?  Internally we don't care at all.
16:55:24 <dcavalca> that's a good question
16:55:47 <dcavalca> I don't personally care, but I suspect people consuming our packages might
16:56:32 <Eighth_Doctor> Fedora builds have kabi stuff turned off
16:56:56 <Eighth_Doctor> but we might want to if we're maintaining on the same version series for a period of time
16:56:56 <jvreeland> good enough prior art for me
16:57:48 <dcavalca> either way, this is something we'll have to document / communicate well when we announce this
16:58:18 <Eighth_Doctor> we haven't even figured out our maintenance policy for the kernel yet
16:58:30 <Eighth_Doctor> so kabi discussion is probably premature
16:58:33 <dcavalca> eh, baby steps :)
16:58:39 <jvreeland> yeah, doesn't need to be figured out anytime soon, just something to think about
16:58:57 <dcavalca> ok we're almost out of time
16:59:01 <dcavalca> gonna go to
16:59:06 <dcavalca> #topic Membership
16:59:10 <dcavalca> as I wanted to welcome steelcowboy
16:59:27 <dcavalca> steelcowboy works at FB and was responsible for adding pykickstart to the SIG
16:59:35 <Eighth_Doctor> yay Jim Heald :D
16:59:45 <steelcowboy> Hi all! Happy to be here, excited to contribute :)
17:00:27 <dcavalca> #topic Misc
17:00:34 <dcavalca> we are out of time, so I think I'll close it out here
17:00:43 <dcavalca> thanks everyone for coming, this was a very productive meeting
17:00:54 <dcavalca> #endmeeting