16:00:09 <dcavalca> #startmeeting Hyperscale SIG
16:00:09 <centbot> Meeting started Wed Mar 30 16:00:09 2022 UTC.  The chair is dcavalca. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:09 <centbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
16:00:18 <dcavalca> #chair dcavalca jvreeland
16:00:18 <centbot> Current chairs: dcavalca jvreeland
16:00:22 <dcavalca> morning everyone
16:00:34 <michel> Morning
16:00:38 <jvreeland> morning
16:00:41 <Eighth_Doctor> hey all
16:00:45 <aekoroglu> Morning
16:01:20 <oidoming> hi
16:01:37 <daandemeyer> Morning
16:01:50 <dcavalca> let's get started
16:01:52 <dcavalca> #topic Followups
16:02:02 <dcavalca> did we have anything to address from the previous meeting?
16:02:05 * dcavalca missed that one
16:02:29 * Eighth_Doctor doesn't remember
16:03:05 <daandemeyer> Not from the meeting, but someone (sorry, forgot the name) asked for mkosi support for the hyperscale repos in the last video hangout. I'm adding that in https://github.com/systemd/mkosi/pull/943
16:03:08 <dcavalca> looking at the notes, the only thing I see was an action to create a ticket to track gpg sha changes for jvreeland
16:03:10 <Eighth_Doctor> and it looks like we don't have the meeting minutes recorded on our sites
16:03:28 <dcavalca> Eighth_Doctor, yup, will fix that right after this is done
16:03:51 <jvreeland> daandemeyer: thank you!
16:04:20 <dcavalca> daandemeyer: oh sweet, so this allows for adding arbitrary additional repos
16:04:28 <dcavalca> this will definitely come in handy, thanks for putting it together
16:04:28 <Eighth_Doctor> daandemeyer: to note (while it doesn't matter here), you probably want to move apt source setup from the deprecated .lists format to the deb822 .sources format
16:04:40 * davdunc waves
16:04:48 <Eighth_Doctor> hey davdunc
16:05:28 <Eighth_Doctor> oh, I guess relatedly, I've made livesys-scripts for externalizing the livesys init scripts used for live media into systemd services that any image build tool can use
16:05:39 <Eighth_Doctor> https://pagure.io/livesys-scripts
16:05:48 <Eighth_Doctor> the package is in Fedora and EPEL 8/9
16:05:55 <dcavalca> sweet
16:06:00 <dcavalca> let's get some of these things in the minutes
16:06:11 <dcavalca> #info mkosi support for hyperscale repos https://github.com/systemd/mkosi/pull/943
16:06:22 <dcavalca> #info livesys-scripts https://pagure.io/livesys-scripts
16:06:36 <chantra_> RPMCoW updates: pckged for both c8s and c9s hsx repos. Latest releases has 2 fixes: don't use nested functions so we don't need executable stack, which makes it (or should now) work with selinux enabled. When using rpm -i, no need to use --nodigest anymore.
16:06:58 <chantra_> Also dnf-plugin-cow is packaged for both c8s and c9s in hsx
16:07:59 <dcavalca> #info RPMCoW updates: packaged for both c8s and c9s hsx repos, should now work with selinux
16:08:10 <dcavalca> one more thing I remembered: did we end up sorting out the -release packages?
16:08:13 <Eighth_Doctor> reminder: rpm in c9s uses /usr/lib/sysimage/rpm for the rpmdb path
16:08:16 <Eighth_Doctor> not /var/lib/rpm
16:08:43 <dcavalca> Eighth_Doctor: is that a c9s upstream thing or an hyperscale thing?
16:08:44 <Eighth_Doctor> as far as I know, they're mostly sorted out
16:09:04 <Eighth_Doctor> I wound up having to obsolete the hotfix repo package because we don't actually have it right now
16:09:09 <Eighth_Doctor> dcavalca: hyperscale
16:09:10 <chantra_> last, but not least.... I split the monolitic patch into the 30 patch serie https://git.centos.org/rpms/rpm/c/629b278b5d408cb8cc864b68625c7d8a08d78f6c?branch=c8s-sig-hyperscale-experimental
16:09:21 <dcavalca> cool, I mentioned it as I just saw arrfab email about extras for c8
16:09:28 <dcavalca> Eighth_Doctor: ah makes sense
16:09:43 <chantra_> >  should now work with selinux , let me confirm quickly that this works as expected
16:09:51 <dcavalca> I think the only thing we had left in hotfix was libvirt?
16:09:56 <Eighth_Doctor> yeah
16:10:18 <Eighth_Doctor> you also periodically used it for llvm stuff
16:10:29 <dcavalca> yeah but that's all upstream now
16:10:42 <Eighth_Doctor> for c9s, I don't think we're going to need it yet so I'm fine with it not existing for now
16:10:48 <dcavalca> works for me
16:11:10 <Eighth_Doctor> I'm also making progress on the kiwi descriptions for hyperscale images
16:11:22 <Eighth_Doctor> https://pagure.io/centos-sig-hyperscale/kiwi-descriptions/pull-request/1
16:11:40 <Eighth_Doctor> for Hyperscale 9, my plan is to use kiwi for all artifacts
16:11:45 <davdunc> I'm also working on packaging mash to support you there Eighth_Doctor
16:11:56 <Eighth_Doctor> excellent
16:12:10 <dcavalca> will we be able to run kiwi in CBS, or will we need to spin up our own image build infra?
16:12:33 <Eighth_Doctor> that's dependent on https://pagure.io/centos-infra/issue/696
16:12:56 <davdunc> Eighth_Doctor++ for thinking ahead.
16:12:59 <dcavalca> oh I'd somehow missed that one
16:13:05 <dcavalca> let's bring it up at the next office hours
16:13:10 <Eighth_Doctor> 👍️
16:13:25 <dcavalca> let's move to
16:13:26 <dcavalca> #topic Announcements
16:13:35 <dcavalca> #info SIG activity report due by April 4
16:13:45 <Eighth_Doctor> mmm
16:13:54 <dcavalca> I'll start a draft later today, let's try and get this wrapped up by Fri
16:13:57 <Eighth_Doctor> I wonder if I can get our initial experimental 9 images ready by then
16:14:05 <dcavalca> that'd be really cool
16:15:04 <Eighth_Doctor> hey dbrandon
16:15:48 <Eighth_Doctor> or not?
16:15:56 <kcwells> bye dbrandon
16:16:00 <dcavalca> lol
16:16:08 <aekoroglu> hey dbrandon
16:16:18 <dbrandon> hi everyone
16:16:21 <Eighth_Doctor> anyway, yeah, I'm hoping to have 9-based images this week to release as experimental
16:16:29 <michel> Eighth_Doctor: I'll be happy to reimage my personal laptop with it
16:16:33 <Eighth_Doctor> :D
16:17:08 <Eighth_Doctor> I think that there will be a couple of SELinux things that need fixing because of the rpmdb move
16:17:14 <dcavalca> #info https://hackmd.io/WJOCJxf0RgaBHPLOvvK8Qg SIG report draft
16:17:16 <Eighth_Doctor> so that's going to be fun :)
16:18:17 <dcavalca> daandemeyer: we also need to validate that selinux still works fine with the latest systemd build
16:18:44 <dcavalca> anything else for announcements?
16:19:10 <Eighth_Doctor> dbrandon is now a member of the SIG, I think?
16:19:17 <Eighth_Doctor> I know we approved it last meeting, dunno if the buttons were pushed
16:19:46 <daandemeyer> dcavalca I might need some help there on what exactly to test, I have no experience with selinux at all
16:19:58 <dcavalca> daandemeyer: that makes two of us :)
16:19:58 <Eighth_Doctor> oh, and our systemd is now older than upstream c9s
16:20:00 <Eighth_Doctor> so that's a problem
16:20:18 <Eighth_Doctor> don't listen to dcavalca, he grew expertise on selinux last year :)
16:20:23 <dcavalca> Eighth_Doctor: dbrandon is not in the ACO group
16:20:32 <michel> I guess just boot up a VM and see what broke compared to when you set selinux to permissive
16:20:36 <Eighth_Doctor> dbrandon: what's your FAS/ACO account?
16:20:41 <dcavalca> yeah, that's pretty much what I do
16:20:45 <dbrandon> dbrandonjohnson
16:20:49 <dcavalca> https://pagure.io/centos-sig-hyperscale/sig/issue/106 right?
16:21:03 <jvreeland> yeah
16:21:03 <michel> But never ever disable selinux or you might have to relabel the entire FS
16:21:17 <daandemeyer> I think I did 250 builds for 8 and 9 already but didn't release them yet pending internal testing, maybe we can go ahead with that now since the internal canary seems OK?
16:21:18 <dcavalca> ok added to ACO
16:21:25 <dcavalca> I'll doublecheck the other stuff after the meeting
16:21:31 <dcavalca> daandemeyer: ship it!
16:21:32 <dbrandon> thats me
16:22:04 <daandemeyer> I still need to ship the systemd-extras stuff, I'll do a new release with that included and ship that
16:22:11 <dcavalca> 251 is also around the corner
16:22:20 <Eighth_Doctor> we probably want to contribute those obsoletes+provides to fedora systemd spec
16:22:39 <Eighth_Doctor> wiring them up to conditionals that turn off and on those components
16:22:45 <dcavalca> Eighth_Doctor: how would you end up with systemd-extras installed on Fedora?
16:22:56 <Eighth_Doctor> ELN?
16:23:05 <dcavalca> oh good point
16:23:13 <dcavalca> yeah, then it makes sense
16:23:16 <Eighth_Doctor> ELN<->Rawhide is a reasonable thing to do
16:23:34 <dcavalca> yeah and with ELN Extras now in play we could have systemd-extras building there
16:23:42 <daandemeyer> Will do that as well then, although it doesn't matter for 250 I guess since we'd have to backport it regardless
16:24:00 <dcavalca> yeah it's more for future proofing
16:24:06 <dcavalca> 251 is also around the corner
16:24:31 <Eighth_Doctor> yup
16:24:53 <daandemeyer> Let's hope I release that one faster than 250
16:25:16 <dcavalca> eheh we'll see
16:25:18 <Eighth_Doctor> heh
16:25:22 <dcavalca> alright, anything else for announcements?
16:25:26 <Eighth_Doctor> I'm just happy c9s will have 250
16:25:36 <Eighth_Doctor> important baseline functionality for KDE is in there :D
16:25:52 <dcavalca> oh on the topic of systemd upgrades: https://github.com/systemd/systemd/issues/22898
16:26:45 <daandemeyer> I looked into what the fix would be and after an initial look it seems like it would get messy
16:27:03 <Eighth_Doctor> if we do figure out a fix, the centos community would love it
16:27:18 <Eighth_Doctor> since that would make dnf system-upgrade from c8s to c9s much more workable
16:28:06 <daandemeyer> I'll put it on the agenda for the next systemd biweekly if it doesn't get a response by then
16:28:38 <dcavalca> yeah this warrants some upstream discussion
16:29:07 <dcavalca> alright, time for
16:29:10 <dcavalca> #topic Tickets
16:30:02 <dcavalca> jvreeland: Eighth_Doctor: what's the latest on the kernel work?
16:30:23 <jvreeland> Still need to update the CI jobs, but the kernel is the same version last I checked
16:30:30 <Eighth_Doctor> I've been increasing the scope of the automation for c9s kernel releases
16:30:44 <Eighth_Doctor> but I'm pretty close to the "rewrite into Python" water mark
16:31:24 <dcavalca> probably not a bad idea
16:31:31 <dcavalca> daandemeyer: did the same for the systemd CI/CD thing
16:32:22 <Eighth_Doctor> someone mentioned a while back that centpkg-sig is supposed to work for working with dist-git for sigs
16:32:34 <Eighth_Doctor> I don't know if that was carlwgeorge or bstinson
16:32:44 <Eighth_Doctor> but it's something I want to look into in the near future
16:33:29 <Eighth_Doctor> I personally wound up having a problem that the c9s official containers on quay.io don't work on docker from el7 (which powers my internal OKD 3.11 instance)
16:33:55 <Eighth_Doctor> bstinson said yesterday that he'd look into it, as I have no idea why it doesn't let me pull those images and says signature key is missing
16:33:58 <chantra_> > should now work with selinux (confirmed it works by pulling straight from what is in hsx repo) @dcavalca
16:34:15 <dcavalca> yay thanks for verifying that chantra_
16:35:26 <Eighth_Doctor> I bring up the container thing because we might have the same problem with our images on quay.io once we make them
16:36:32 <aekoroglu> +1 remote koji builder for sigs :)
16:37:32 <dcavalca> related to containers, I just realized we never replied on https://pagure.io/centos-sig-hyperscale/sig/issue/91 so I just did so now
16:38:26 <dcavalca> other tickets folks would like to discuss?
16:39:19 <aekoroglu> we're still waiting for https://pagure.io/centos-infra/issue/694
16:39:52 <dcavalca> aekoroglu: you may wanna follow up with bstinson on that one if you haven't already
16:40:53 <kcwells> We have an email thread going.
16:41:18 <kcwells> Current main issue is that there isn't currently facilities to select specific CI for specific runners
16:42:35 <dcavalca> interesting, that sounds like something that would be generally useful
16:44:16 <oidoming> for 91, the build automation refers to "trigger when a new commit is pushed to build the image and publish to quay.io" or "rebuild the container once a day (for example) to make sure everithing is okay for any updates on the dependencies" or both?
16:44:44 <kcwells> Yeah. Digging deeper into how it's set up and what it'd take to add this capability. The initial stuff we were trying to build for here isn't dependent on it, but in the future some optimizations require specific hardware to build, or we'll just want to be able to test specific features on next-gen platforms, so we'll want to be able to assign
16:44:44 <kcwells> builds/test runs to specific hardware
16:45:23 <dcavalca> oidoming: probably both
16:45:33 <dcavalca> we'd want to rebuild if the container build script is updated
16:45:42 <dcavalca> and we'd want to rebuild regularly as packagers flow into Stream
16:45:57 <oidoming> great
16:46:07 <dcavalca> we could probably just do a periodic build (daily or weekly or whatever we decide makes sense)
16:46:36 <dcavalca> kcwells: makes sense
16:46:52 <dcavalca> moving on to
16:47:01 <dcavalca> #topic Membership
16:47:12 <dcavalca> we already addressed dbrandon
16:47:19 <dcavalca> anything else to discuss here?
16:48:56 <dcavalca> sounds like not
16:49:00 <dcavalca> #topic Misc
16:49:39 <dcavalca> I know it's early to talk about travel, but I'm considering going to devconf.us in Boston on Aug 18-20
16:49:55 <dcavalca> if a bunch of us want to join, we can try and organize some Hyperscale event
16:50:15 <dcavalca> there also may be a CentOS Dojo happening concurrently (or at least, there was for previous devconfs)
16:50:52 <Eighth_Doctor> that would be awesome
16:50:56 <Eighth_Doctor> I'd like to go if it's a thing
16:50:58 <dcavalca> also, the CfP is open in case any of you want to submit a talk
16:52:00 <chantra_> I may be able to go
16:52:24 <aekoroglu> how should we proceed to have hyperscale/x86_64/packages-intel ? I'll started to port Lu Hongjiu's glibc and binutils optimizations to current c8s and c9s glibc+binutils. We already ported our zlib optimization to c8s zlib and working on R + OpenBLAS etc..
16:52:56 <Eighth_Doctor> is that something we want to do?
16:53:01 <dcavalca> aekoroglu: we can request a tag for this, but did you validate that the rest of the distro still works properly?
16:53:13 <dcavalca> usually if you rebuild a core library you need to rebuild the downward deps
16:53:17 <dcavalca> unless it's ABI/API compatible
16:53:43 <aekoroglu> We already talked with Lu Hongjiu and there will be no API/ABI changes
16:54:35 <dcavalca> I'm not opposed to having this in the SIG, but I'd like to also see a path for upstreaming these optimizations in Fedora
16:54:46 <aekoroglu> so I'll create an issue for that
16:56:28 <aekoroglu> dcavalca: Boston sounds good I'd like to go as well
16:57:11 <dcavalca> alright, let's circle back in a month or so and if things still look good we can start making plans
16:59:15 <dcavalca> we're out of time, thanks everyone for the great discussion today
16:59:47 <dcavalca> #endmeeting