16:00:05 <dcavalca> #startmeeting Hyperscale SIG
16:00:05 <centbot> Meeting started Wed Apr 14 16:00:05 2021 UTC.  The chair is dcavalca. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:05 <centbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:00:07 <jvreeland> hello
16:00:10 <dcavalca> morning everybody
16:00:16 <dcavalca> #chair dcavalca jvreeland
16:00:16 <centbot> Current chairs: dcavalca jvreeland
16:00:28 <ignatenkobrain> .hello2
16:00:35 <ignatenkobrain> .hello ignatenkobrain
16:00:40 <ignatenkobrain_> :(
16:00:43 <dcavalca> ignatenkobrain: that doesn't work on centos channels
16:00:46 <ignatenkobrain_> ah
16:01:06 <ignatenkobrain> anyway, hi everyone :)
16:01:22 <ignatenkobrain> dcavalca: thanks for the ping - I almost forgot about meeting :)
16:01:38 * King_InuYasha wants zodbot
16:01:43 <King_InuYasha> anyway, hey all :)
16:01:45 <dcavalca> funnily enough, centguard put me in the doghouse after I pinged everybody
16:01:54 <dcavalca> I think there's some check if you alert too many people at once
16:01:57 <King_InuYasha> wow
16:02:12 <dcavalca> anyway, let's get started
16:02:13 <dcavalca> #topic Followups
16:02:22 <dcavalca> plenty of news this round
16:02:40 <King_InuYasha> oh?
16:02:47 <ignatenkobrain> good ones? :)
16:02:54 <dcavalca> #info SIG report is out: https://blog.centos.org/2021/04/centos-hyperscale-sig-quarterly-report/
16:03:09 <dcavalca> #info CBS builds are now signed with RSA/SHA256
16:03:17 <michel_slm> hello all!
16:03:24 <dcavalca> also, we made major progress on systemd
16:03:41 <dcavalca> #info systemd 247 released to mirrors with experimental selinux support
16:03:59 <dcavalca> thanks again King_InuYasha for helping out with the selinux stuff
16:04:01 <King_InuYasha> \o/
16:04:04 <ignatenkobrain> 🥳
16:04:11 <michel_slm> sha256, nice
16:04:18 <dcavalca> we also got a fixed binutils in the repo, so I've reenabled LTO for systemd builds
16:04:24 <King_InuYasha> yay!
16:04:25 <anitazha> yay
16:04:35 <jvreeland> nice
16:04:42 <dcavalca> #info systemd staging repo is ready: https://pagure.io/centos-sig-hyperscale/systemd
16:04:57 <dcavalca> the main branch on that one mirrors github every morning
16:05:30 <King_InuYasha> we probably want to do something similar for pkg-git repos and other software we maintain stuff for
16:05:39 <dcavalca> plan is to use this to stage patches for systemd releases, like fb has done internally so far
16:05:43 <King_InuYasha> nice
16:05:47 <michel_slm> how's the github mirroring set up?
16:05:58 <dcavalca> the sync job for this currently runs on fb infra
16:06:15 <dcavalca> mostly because I couldn't figure how to store a secret in toddlers
16:06:15 <King_InuYasha> I may want to do something similar for btrfs-progs because I want to add a patch to hard-disable raid56 modes until upstream gets around to reworking them
16:06:28 <dcavalca> King_InuYasha: works for me
16:06:31 <King_InuYasha> or at least only permit ro for raid56
16:07:09 <dcavalca> yeah that's a good call
16:07:24 <dcavalca> anybody else has followups?
16:07:28 <King_InuYasha> but as we don't even have a kernel to work from, that's all future-type stuff
16:07:28 <michel_slm> I'll be working on getting dracut in, and there's an interest in doing pykickstart (so you can use c8s to generate kickstarts for newer Fedora releases) and lorax as well (to get a bugfix in)
16:07:46 <King_InuYasha> we'll need to pull the full anaconda stack anyway
16:07:57 <King_InuYasha> lorax, pykickstart, anaconda, etc.
16:08:12 <King_InuYasha> because we need to undo the btrfs block on them
16:08:25 <michel_slm> King_InuYasha: for this limited usecase we don't need Anaconda
16:08:43 <King_InuYasha> lorax req anaconda
16:08:43 <dcavalca> I vaguely remember bstinson mentioning that SIGs can't currently build ISOs a while ago
16:08:47 <King_InuYasha> it runs it
16:08:52 <King_InuYasha> that's how disk images are made
16:08:52 <dcavalca> so that could be an issue if we want to ship a new installer stack
16:09:01 <michel_slm> the btrfs block, IIRC, is solely in pykickstart. it redefines the language parser to error out on btrfs commands.
16:09:11 <King_InuYasha> no, there are changes in anaconda to also block it
16:09:20 <King_InuYasha> 3~5 commits to filter it out
16:10:29 <King_InuYasha> I haven't checked lorax, but I wouldn't be surprised if there's also something there
16:10:41 <dcavalca> michel_slm: King_InuYasha: should we book an afternoon to coordinate work on this and hack on it together?
16:10:45 <King_InuYasha> yes
16:10:58 <michel_slm> yup
16:11:02 <dcavalca> michel_slm: wanna take point on this? let's pull Jim in as well
16:11:12 <michel_slm> yeah, I'll arrange a meeting
16:11:45 <dcavalca> #action michel_slm schedule meeting to hack on anaconda/kickstart/etc
16:11:59 <dcavalca> cool, let's move to
16:12:04 <dcavalca> #topic Announcements
16:12:19 <dcavalca> #info dwarves 1.21 coming up
16:12:29 <dcavalca> I'm building this one right now actually, as we need it internally
16:12:55 <dcavalca> anybody else has other announcements?
16:13:44 <dcavalca> looks like no, let's move to
16:13:47 <dcavalca> #topic Tickets
16:14:01 <dcavalca> I filed #40 yesterday to setup a CI pipeline for systemd
16:14:32 <dcavalca> basically take the current packaging and use it to build a daily snapshot of systemd based off the git master
16:14:55 <dcavalca> this forces the unit tests to run, hence catching issues early on, and produces packages one can use for further testing as a byproduct
16:15:11 <dcavalca> it should be pretty straightforward, and I've put a basic implementation in the ticket already
16:15:32 <dcavalca> mostly waiting on infra to figure out how to hook up to CentOS CI at this point, as that seemed the more reasonable place to run this from
16:15:57 <King_InuYasha> we should have access to centos-ci from pagure
16:16:01 <King_InuYasha> so that shouldn't be too bad
16:16:04 <jvreeland> For #7 I've got the kernel building and working with the fedora 33 packaging for 5.10.21. Still need to build from our hosted kenrel tree source and bump to a newer version but I'll port more details in the ticket.
16:16:13 <dcavalca> also, if folks have ideas for other tests and things we could do with this, please comment on the ticket
16:16:23 <dcavalca> jvreeland: awesome, thanks for the update
16:17:07 <King_InuYasha> I'd like to do this for the software management stack backport as soon as we have it
16:17:28 <King_InuYasha> and there's also interest in openSUSE MicroOS using rpmcow+microdnf+txnupd together
16:17:38 <King_InuYasha> so we should test this combo in Hyperscale as well
16:17:43 <dcavalca> oh, speaking of rpm
16:17:51 <dcavalca> there's some discussion internally around fsverity
16:18:01 <King_InuYasha> with the new plugin that landed?
16:18:09 <dcavalca> that's currently slated for 4.17, but it's not enabled by default at the moment
16:18:34 <King_InuYasha> rpm backports are going to be _interesting_
16:18:42 <dcavalca> my current thinking was to maybe put up a Change for F35 to see if we could get it enabled by default
16:18:53 <dcavalca> but I need to run this by the folks that actually know anything about this first
16:18:55 <King_InuYasha> yeah, will we have btrfs fsverity support by then?
16:19:02 <King_InuYasha> otherwise it's kind of pointless
16:19:03 <dcavalca> that's an open question
16:19:23 <dcavalca> we have folks working on it right now, but it's hard to read the tea leaves of when it'll land in the kernel
16:19:26 <michel_slm> which other filesystems support it?
16:19:34 <dcavalca> michel_slm: ext4
16:19:40 <King_InuYasha> ext4
16:19:49 <michel_slm> which, from the CentOS POV, is ... also not that useful
16:19:58 <King_InuYasha> and Fedora just doesn't use it anymore
16:20:01 <dcavalca> I think the current guess is that it's possible for 5.13, but more likely for 5.14, so we'll have to see
16:20:16 <michel_slm> if it's 5.14 it's doable for F35 right?
16:20:20 <King_InuYasha> yes
16:20:28 <michel_slm> but we're digressing
16:20:43 <dcavalca> anyway, in the centos context: fsverity is tricky because it requires embedding stuff into the rpm themselves
16:20:52 <King_InuYasha> in any case, fsverity is not useful without btrfs supporting it
16:20:55 <dcavalca> so either it's enabled at build time, or you need to resign all rpms to use it later
16:21:01 <dcavalca> which is obviously not ideal
16:21:11 <dcavalca> King_InuYasha: agreed, same here
16:21:25 <dcavalca> I'm gonna try and get some data around what's realistic here and circle back
16:21:44 <ignatenkobrain> we can avoid RPM backports if we upgrade it to latest :D
16:21:59 <King_InuYasha> well, upgrading is backporting from our perspective
16:22:07 <King_InuYasha> it's a rebase, sure, but it's still pain and agony
16:22:09 <dcavalca> ignatenkobrain: we can do that, but we'll need to rebuild all the downwards packages
16:22:22 <dcavalca> we did that at FB for a while and it's "fun"
16:22:34 * King_InuYasha is just trying to do rpm 4.14.1->4.14.3 update for openSUSE Leap and it sucks
16:22:36 <dcavalca> especially when you find out that e.g. grub, of all things, ends up depending on rpm somehow
16:23:02 <ignatenkobrain> dcavalca: I did that at RH too 🙂 I ended up hacking SONAME as it changed ABI only in few specific places which were not really used by anybody :)
16:23:09 <dcavalca> I don't suppose we can get koschei for centos? :p
16:23:21 <King_InuYasha> if only
16:23:31 <King_InuYasha> but we could make a repoclosure bot to check these things for us
16:23:32 <dcavalca> ignatenkobrain: yeah, that's an option too
16:23:40 <ignatenkobrain> the good thing is that there are only few packages which actually depend on librpm*.so so we can even rebuild those
16:23:42 <dcavalca> King_InuYasha: that would be really useful
16:24:13 <King_InuYasha> another strategy is to have a stub for the old soname that redirects to new symbols
16:24:20 <ignatenkobrain> dcavalca: King_InuYasha I suppose we can discuss proposals in https://pagure.io/centos-sig-hyperscale/sig/issue/39 about this
16:24:25 <King_InuYasha> yes
16:25:16 <dcavalca> yeah, let's do that
16:25:31 <dcavalca> I'll make sure to rope in malmond as well, as this is relevant to his CoW work
16:25:47 <dcavalca> let's see what else we have
16:26:04 <dcavalca> jvreeland: for #31, have you heard anything back from pbisaacs?
16:26:49 <jvreeland> not yet
16:27:04 <dcavalca> alright, let's table this one again then
16:27:19 <dcavalca> for #13, did anybody start the conversation with Cloud?
16:28:08 <King_InuYasha> no
16:28:28 <King_InuYasha> I don't think that's worth doing until we have stuff in place with kernel and filesystem and package manager stack
16:28:46 <dcavalca> yeah that's fair
16:28:59 <dcavalca> alright, that's it for tickets I think
16:29:02 <King_InuYasha> sweet
16:29:05 <dcavalca> #topic Membership
16:29:14 <dcavalca> don't think we have anything here?
16:29:20 <dcavalca> the noggin migration seems to have gone well
16:29:48 <michel_slm> yeah, I had the more complicated case (mismatched account ID) and looks like permissions get migrated fine
16:30:07 <King_InuYasha> yup
16:30:12 <King_InuYasha> mine was easy :D
16:30:17 <dcavalca> yeah, I went over the group the day after and things looked sane
16:30:18 <King_InuYasha> same name and email on both sides
16:30:39 <dcavalca> alright, let's close with
16:30:40 <dcavalca> #topic Misc
16:30:54 <dcavalca> I've asked centos-docs@ for a way to publish content to docs.centos.org
16:31:09 <dcavalca> as I'd like to start writing some user-facing documentation on the packages and stuff we have in the SIG
16:31:18 <dcavalca> no reply yet, but we'll see
16:31:53 <ignatenkobrain> that would be great. we would need to have list of packages + ownership + some tickets link or something like that
16:32:10 <dcavalca> yeah that's the idea
16:32:11 <ignatenkobrain> so that people could very easily see content, why and whom to bug :)
16:32:15 <ignatenkobrain> +1
16:32:22 <dcavalca> if this doesn't pan out we can also just use the wiki
16:32:28 <dcavalca> but I'd personally prefer a static site
16:32:59 <dcavalca> does anybody have anything else they wanna discuss?
16:33:43 <King_InuYasha> we probably should have a pagure repo we can set as bugurl for our packages
16:34:04 <King_InuYasha> similar to the ticket repo, except for people to just file issues on packages for
16:34:36 <dcavalca> is bugurl a thing you setup with a rpm macro?
16:34:40 <King_InuYasha> yes
16:34:47 <dcavalca> cool, we can just get it added to our tags then
16:34:54 <King_InuYasha> it sets the BugURL field in rpm metadata
16:34:55 <michel_slm> do we want people to file bugs in a single bug tracker, or should we just use the same repo as the package itself for bugs?
16:35:14 <dcavalca> michel_slm: you can't file issues on git.centos.org
16:35:16 <King_InuYasha> well we can't use git.centos.org repos for issues
16:35:28 <dcavalca> and I'd rather people not file in BZ for our packages, as it'll get confusing for the maintainers
16:35:35 <King_InuYasha> yup
16:35:39 <michel_slm> oh, I mean, if each Hyperscale package has its own repo, use that repo for tickets (not bugzilla)
16:35:58 <dcavalca> that seems like a lot of work tbh, especially for minor packages like dwarves
16:36:07 <michel_slm> but... yeah, just realized https://pagure.io/centos-sig-hyperscale/systemd/tree/main is for the systemd source tree, not src-git
16:36:18 <King_InuYasha> yep, I think it'd just be better to have a repo and set it up with tags for each package
16:36:25 <michel_slm> we can start with a single bug repo and revisit if it doesn't scale
16:36:29 <King_InuYasha> yup
16:36:33 <dcavalca> yeah, we can always change it later
16:36:42 <dcavalca> King_InuYasha: wanna take point on this?
16:36:51 <King_InuYasha> sure
16:36:54 <michel_slm> ah yes, and if bugs get misreported, retagging is easier than... huh, in Pagure you probably can't even move an issue to a different repo anyway
16:37:06 <dcavalca> #action King_InuYasha setup bug tracker and bugurl for our packages
16:37:39 <dcavalca> alright, anything else?
16:38:11 <King_InuYasha> don't think so
16:38:21 <dcavalca> yeah I think that's it for today
16:38:30 <dcavalca> thanks everyone, great discussion today
16:38:35 <dcavalca> #endmeeting