Add Intel Raptor Lake FW binaries to v2.2.2

[mengdonglin]

We plan to add 2 FW binaries symbolic links for Intel Raptor Lake (RPL) platforms to v2.2:

• sof-rpl.ri for RPL-P
• sof-rpl-s.ri for RPL-S/-SBGA (desktop / mobile workstation)

The sof version will be bumped to v2.2.2: https://github.com/thesofproject/sof/releases/tag/v2.2.2

There are no FW src code updates from v2.2.
And there are no topology updates from v2.2.1

So a new PR for v2.2.2 will be submitted, including

• new FW binaries symbolic links for RPL with version v2.2.2 in the manifest header:
  sof-rpl.ri -> sof-adl.ri
  sof-rpl-s.ri -> sof-adl-s.ri

• FW binaries of all other platforms (non-RPL platforms). But they're same as those in v2.2. And their version in FW manifest header is still v2.2

• No topology binaries.

@perexg @mrhpearson @marc-hb @keqiaozhang @jason77-wang Are you good with the plan? Please let us know if you see any risks.

[mrhpearson]

Just checking my understanding - but will all the FW files be in sof-bin/v2.2.x/sof-v2.2.2 folder, or will they now be split over two folders? I'd rather have all the FW files in one directory (sof-v2.2.2) please

Thanks
Mark

[marc-hb]

I'm finalizing a new sof-bin script (EDIT: #91 draft) that will combine platforms from multiple folders into a single tarball. It could be useful for this RPL 2.2.2 release. It is required for the next 2.3 release which will combine binaries from 2.2.x and 2.3.

This means some future releases will NOT come from a single sof-bin.git folder anymore, they will be the combination of multiple sof-bin.git folders with dif. @mrhpearson reading between the lines of #87 I'm afraid this will be incompatible with how you build debian packages (pointer?)

Release tarballs will still contain all platforms in a single tarball as always.

[mrhpearson]

This seems awfully complicated and designed to break in the future. Is there any reason you don't just tag a release for vX.Y.Z for both topologies and binaries (even if they haven't changed) and go "that's version X.Y.Z".

FWIW - the Readme on this site doesn't mention the tarballs as a recommended install path - it will need updating

For my specific situation of creating the Debian package:

• I'm using the git repo because it ties in with the Debinan git-buildpackage tool that I use. I'm pretty new to Debian packaging so maybe there are smarter alternatives but...it works so for now in the interest of my sanity I'm sticking with it. Admittedly I still haven't actually released v2.2.1 because of some symlink issues but that's by the by.
• I can't currently use your install scripts because Debian doesn't have rsync installed by default. (Don't know why)

I'd personally love to just go and be able to clone the repo and copy files from sof and sof-tplg to the install location and be done. To not have to worry about what you've changed in your release every time would be sweet - but I'll work around whatever happens.

[perexg]

The tar balls are used for distribution of a package for decades. The repositories are good for the development to manage the history. Why you need to fetch (work with) all files ?

Also, usually the distros cache the source archives for packages which is a bit problematic for git repos.

[mrhpearson]

The history of this is that nobody packaged SOF firmware for Debian - and it was like that for a couple of years. We had lots of Debian users on Lenovo products quite frustrated by that. I had no experience with packaging at all and tried to find experienced people to do it. Side story: we even sent a developer an X1 Carbon 7 to help with it (he said he would) and he basically took the unit and stuck his fingers up at us and didn't do anything (and didn't return it). That also completely set back me getting more Lenovo HW to developers as it was our first time doing that kind of exercise and it left a bad taste with the product teams (which was frustrating).

Eventually, one evening, I figured I'd have a go at it myself and see what was involved. I started reading the documentation and putting together the bare bones and got a .deb package built. I put it online and got a bunch of really helpful feedback on how it should work - and eventually ended up with the first Debian package release of sof firmware. It was quite fun and something I should have tried a year earlier.

That method uses tools that Debian has to clone a git repo and do the packaging - presumably normally building a package from source which doesn't apply here. In there I have a rules file that pulls out the bits that matter and it's mostly worked with some tweaks needed.
I never even thought of using the tarballs and nobody else mentioned it so my apologies if it was obvious (but I'm not alone). I'll highlight that they're not mentioned in the Readme - which very specifically mentions using the the git repo. It's probably a well known thing to those who do packaging all the time - but I don't have that experience I'm afraid.

FWIW the question posed in this thread also applies to the tarballs. There are now sometimes sof-bin tarballs with everything, and sometimes sof-tplg tarballs with only the topology files...and you have to figure that out for each release. It doesn't make sense to me - it means that I as a packager have to go and figure out the pieces.

If it's only me then no worries, I appreciate it really isn't that big a deal and I can live with it - but the point of this ticket was to ask our opinion :)

[perexg]

FWIW the question posed in this thread also applies to the tarballs. There are now sometimes sof-bin tarballs with everything, and sometimes sof-tplg tarballs with only the topology files...and you have to figure that out for each release. It doesn't make sense to me - it means that I as a packager have to go and figure out the pieces.

I initiated discussion in #76 to pack only different files for the partial releases (against the full package) to see the differences on the first look, but it seems that it was decided to pack the whole topology directory. I know, it's even a different way - but doable when consistent.

The statistics for 2.2 and 2.2.1 (topology files): Current: 256 Added: 48 Different: 36 Removed: 0 - approx. 30% topology files changed against the full release.

The full package release may be simplest for packagers, of course. But more space will be wasted.

[marc-hb]

First, thank you @mrhpearson and @perexg for the time you spend coming here and providing such feedback. Even when we miss the bandwidth or the skills (or both) to act quickly on it which may give wrong impressions[1], it's still incredibly useful to us because it's how we know for sure how sof-bin gets used without spending time finding and staring at potentially out of date Debian or Fedora scripts and misunderstanding them[2] and it does affect how we do things in the longer run. I can't overstate how grateful we are.

I'm with @perexg on the sof-bin.git vs tarball question. The purpose of sof-bin.git is to be 1. a complete and authoritative record of every binary that was ever released. 2. a convenience for developers, testers and other power users who want to experiment. I believe minimizing duplication is required for both 1. and 2. I mean we don't want sof-bin "power" users to waste time checksumming and comparing binaries, wondering which ones are different and which ones are identical. Comparing across platforms (e.g. thesofproject/sof@d6906d3aebd1f02) is time-consuming enough; we really want to avoid even more duplication across versions.

Unfortunately, the next 2.3 release is going to be a mix of new 2.3 binaries and old 2.2 binaries and I think copying 2.2 binaries into a future 2.3 directory in sof-bin would be extremely confusing and potentially time-consuming. This is why I submitted in draft #91 a new script to combine 2.2 and 2.3 binaries only in final tarballs (which will now include a manifest). This means it won't be possible to just grab the 2.3 release from a single sof-bin directory any more. Sorry about that.

From a slightly more "selfish" perspective, tarball creation also provides a strong "release checkpoint" and time to cross all the t's and dot all the i's and even perform some automated checks whereas commits in sof-bin are more incremental and "fast-paced". Sure, git tags exist but they can be "too easy" to create and too tempting to move (bad!) - most recent temptation in #84. Replacing a tarball is a more "outrageous" step that fewer people dare take. Note in the past we even had sof-bin git history rewrites + force-pushes! The horror... #25

Finally, tarballs contain only one release at a time which completely clears any doubt about which version is being used.

tl;dr: please try to switch to final release tarballs instead of downloading from sof-bin directly, sorry for the additional effort. Pretty sure Debian is well equipped for that.

FWIW - the Readme on this site doesn't mention the tarballs as a recommended install path - it will need updating

Thanks, will fix ASAP. So far it didn't make a difference.

I can't currently use your install scripts because Debian doesn't have rsync installed by default. (Don't know why)

I just checked The README in the tarball clearly states that rsync and the install.sh script are optional:

You don't have to use install.sh, you can use any recursive copy of your preference.

At least one README is up to date :-) I recommend rsync because it's more flexible, faster and last but not least idempotent but it's optional and will stay that way.

FWIW the question posed in this thread also applies to the tarballs. There are now sometimes sof-bin tarballs with everything, and sometimes sof-tplg tarballs with only the topology files...and you have to figure that out for each release. It doesn't make sense to me - it means that I as a packager have to go and figure out the pieces.

Fair point previously reported by @perexg in #76. Please give us more time about this. BTW the "semi-loose" coupling between SOF firmware and SOF topologies seems to be in some kind of "uncanny valley" where releasing them neither together nor separately feels ideal...

[1] at least we're not stealing laptops... wow!
[2] Please share URLs anyway, can never hurt.

[mengdonglin]

@perexg @mrhpearson @marc-hb @keqiaozhang @jason77-wang

Sorry for previous inaccurate communication. RPL has the same signature as ADL for audio firmware.

So for 2.2.2, it seems we needn't add new FW binaries for RPL. We can add new FW symbolic links for RPL:
sof-rpl.ri -> sof-adl.ri
sof-rpl-s.ri -> sof-adl-s.ri

I also updated the description.

[mrhpearson]

Ack - and thanks for all the details :)
I'll be looking at moving to the tarballs...just going to take me a bit of research

[mengdonglin]

PR #98 has been submitted for v2.2.2. Can you help review? @perexg @mrhpearson @marc-hb @keqiaozhang @jason77-wang

@mrhpearson I don't know why I cannot add you to the reviewer list of PR #98

[mengdonglin]

Close this issue as v2.2.2 is done and tarball is uploaded: https://github.com/thesofproject/sof-bin/releases/tag/v2.2.2