@Randy,

To summarize the discussion, I should try - 
1. Upgrade Kirkstone/Lagdale rust to 1.66.1 and
2. backporting the fixes of Cargo/Rust for kirkstone (patches given in https://github.com/rust-lang/wg-security-response/tree/main/patches/CVE-2022-46176)

Is this correct?


--
Sundeep K.

On Wed, Jan 18, 2023 at 8:08 AM Mikko Rapeli <mikko.rapeli@...> wrote:

Hi,

On Wed, Jan 18, 2023 at 08:45:56AM +0100, Alexander Kanavin wrote:

On Wed, 18 Jan 2023 at 03:08, Randy MacLeod <randy.macleod@...> wrote:

So far, there haven't been many Rust/Cargo CVEs so maybe we're making
too big a deal out of this. I certainly don't miss the deluge of memory management CVEs that
C/C++ applications suffer from!

For what it's worth I'm with you here, and I actually have an even
more radical view (that may offend some - apologies). I think this
whole 'CVE backporting' business is both enormously wasteful and never
complete (or even close to it). Backporting CVEs and the stable
release policy is basically a cover-up for bad (or altogether absent)
CI at the project users side. If you upgrade a component, and it
causes trouble, the trouble should be caught by pipeline, and not in
the end product when the update has shipped.

It's usually not lack of CI, continuous integration, but lack of testing
coverage which is visible as risk management where any changes are
avoided and someties FUD replaces hard facts. A CI run may not show that
SW compatibility or some use cases are broken.

We've been running with rust in production for a couple of years, I
think we hit our first instance of an upgrade issue last week - a ureq
crate upgrade changed something (I suspect in its pooling code) which
meant that we ended up using all file descriptors. Wasn't caught in CI
as nothing ran for long enough, but was caught in test (we still need
to get to the root cause - right now we just rolled back that crate).

But over that time we've aggressively upgraded rust from something
like 1.48 to 1.66.1, all the crates we build against and to the best
of my knowledge this is the first time we've been tripped up.

The saner policy would have been 'a Yocto stable release contains
component versions all within their support windows by respective
upstreams'. If the only supported version is the latest one, then so
be it.

When mainting yocto SW stack for a long time, I made the distinction
between development tools and target SW. Basically all -native and
nativesdk- recipes and those target recipes which are not in the product
by default are development tools.

Development tools can be updated to new versions to fix severe bugs and
CVEs. For target SW, this can be done if ABI compatibility is preserved,
or in case of severe issues where ABI is part of the problem or
backporting the fix is more risky than an update, but this requires a
managed transition with the product platform where all users of the SW
are checked. ABI compatibility is important because frequently real
products often need to integrate binaries where source code is not available
to the bitbake build.

With this approach, I've updated tools and compiler versions. I would
not mind similar policy in upstream. Maintaining multiple rust/cargo
versions well is not likely going to happen so I'd accept this and
accept same updates in all branches.

If the same tool version works well and is tested in another yocto
branch and there are no major issues porting the same changes to another
LTS branch, this gives confidence that the update is ok.

For vendor reasons, we're stuck on thud (I know...), however we've a
pile of backports so we can mostly just pick current recipes out of
master or stable branches (python packages are the one which we're
coming up against now as a blocker - fortunately we've expunged python
from our target code base).

My view for a very long time has been that you basically have to stay
very close to the most current release of whatever upstream code
you're consuming, otherwise you're doomed to spending more and more of
your life finding/fixing issues (security or otherwise) which were
long since addressed upstream.

--
Alex Kiernan

Hi,

On Wed, Jan 18, 2023 at 08:45:56AM +0100, Alexander Kanavin wrote:

On Wed, 18 Jan 2023 at 03:08, Randy MacLeod <randy.macleod@...> wrote:

So far, there haven't been many Rust/Cargo CVEs so maybe we're making
too big a deal out of this. I certainly don't miss the deluge of memory management CVEs that
C/C++ applications suffer from!

For what it's worth I'm with you here, and I actually have an even
more radical view (that may offend some - apologies). I think this
whole 'CVE backporting' business is both enormously wasteful and never
complete (or even close to it). Backporting CVEs and the stable
release policy is basically a cover-up for bad (or altogether absent)
CI at the project users side. If you upgrade a component, and it
causes trouble, the trouble should be caught by pipeline, and not in
the end product when the update has shipped.

It's usually not lack of CI, continuous integration, but lack of testing
coverage which is visible as risk management where any changes are
avoided and someties FUD replaces hard facts. A CI run may not show that
SW compatibility or some use cases are broken. In the worst case this is
detected by customers after SW has been deployed in the field.

The saner policy would have been 'a Yocto stable release contains
component versions all within their support windows by respective
upstreams'. If the only supported version is the latest one, then so
be it.

When mainting yocto SW stack for a long time, I made the distinction
between development tools and target SW. Basically all -native and
nativesdk- recipes and those target recipes which are not in the product
by default are development tools.

Development tools can be updated to new versions to fix severe bugs and
CVEs. For target SW, this can be done if ABI compatibility is preserved,
or in case of severe issues where ABI is part of the problem or
backporting the fix is more risky than an update, but this requires a
managed transition with the product platform where all users of the SW
are checked. ABI compatibility is important because frequently real
products often need to integrate binaries where source code is not available
to the bitbake build.

With this approach, I've updated tools and compiler versions. I would
not mind similar policy in upstream. Maintaining multiple rust/cargo
versions well is not likely going to happen so I'd accept this and
accept same updates in all branches.

If the same tool version works well and is tested in another yocto
branch and there are no major issues porting the same changes to another
LTS branch, this gives confidence that the update is ok.

Cheers,

-Mikko

On Wed, 18 Jan 2023 at 03:08, Randy MacLeod <randy.macleod@...> wrote:

So far, there haven't been many Rust/Cargo CVEs so maybe we're making
too big a deal out of this. I certainly don't miss the deluge of memory management CVEs that
C/C++ applications suffer from!

For what it's worth I'm with you here, and I actually have an even
more radical view (that may offend some - apologies). I think this
whole 'CVE backporting' business is both enormously wasteful and never
complete (or even close to it). Backporting CVEs and the stable
release policy is basically a cover-up for bad (or altogether absent)
CI at the project users side. If you upgrade a component, and it
causes trouble, the trouble should be caught by pipeline, and not in
the end product when the update has shipped.

The saner policy would have been 'a Yocto stable release contains
component versions all within their support windows by respective
upstreams'. If the only supported version is the latest one, then so
be it.

Alex

On Tue, Jan 17, 2023 at 12:00 PM Alexander Kanavin
<alex.kanavin@...> wrote:

Option 1 looks like a new policy too. If we can upgrade rust across
many major versions in a stable release, then why not other items?

In oe-core we have a trivial exception for vim but of course it's an application not a toolchain.

Similarly, in other layers, applications such as Chromium (and Tensorflow?) only maintain one version and
users are expected to upgrade to get bug fixes.

The Rust developers / community seems to want their software to work in a similar way.
They have a quite exhaustive test suite (Crater) to check for regressions.
I'll look for some actual test results from Crater and reply here when I find them.
I not sure what to make of the "Rust Editions" and how they'd fit into our distro support
but the clear mandate from upstream is that only the 'stable' release is supported.

   https://doc.rust-lang.org/edition-guide/editions/index.html  

I think there's an argument to be made that until Rust releases 2.x, we just update
to the latest version. If you haven't please read:

  https://internals.rust-lang.org/t/cargo-cve-2022-46176-fix-for-older-releases/18152/3?u=sundeep-kokkonda

Assuming that some day, Rust-2.x is released, then one would hope that 1.x would be maintained
for a few years on it's own branch but again the semantic versioning rules would allow them to
release say 1.288, 1.289, ... and at that time, it would likely be more clear that we should be following the
1.x releases for older branches and 2.x releases for newer branches.

According to the stable release "rules" option 1 would require an
exception granted by the TSC.  Otherwise it is a no go.

So it seems to me that this is a classic case for using a mix-in layer.

Once I have enough data, I'll present it here and if it makes sense to ask
the TSC, we can do that.

Upstream is not backporting fixes for a CVE such as this or for less notable bug fixes,
so we'd be giving up those improvements. I suppose we'd only backport
bug fixes should someone encounter the bug or it becomes equally notable.

So far, there haven't been many Rust/Cargo CVEs so maybe we're making
too big a deal out of this. I certainly don't miss the deluge of memory management CVEs that
C/C++ applications suffer from!

Sundeep,

Please also try to backporting the fixes to say Cargo/Rust for kirkstone.
This CVE resulted in ~10 patches so it's hopefully one
of the more complicated back ports and will prove to be a good test case.

../Randy

Steve

On Tue, 17 Jan 2023 at 22:57, Randy MacLeod <randy.macleod@...> wrote:

On 2023-01-17 16:54, Richard Purdie wrote:

On Tue, 2023-01-17 at 15:29 -0500, Randy MacLeod wrote:

On 2023-01-16 10:20, Kokkonda, Sundeep via lists.openembedded.org
wrote:

  Rust community said the security fixes are only for the current
stable relases.

https://internals.rust-lang.org/t/cargo-cve-2022-46176-fix-for-older
-releases/18152/3?u=sundeep-kokkonda
  For old release we've to backport the patches ourselves.

The other alternatives are
1. upgrade to 1.66.1 on older branches,
2. add 1.66.1, which will be the PREFERRED_VERSION but keep the older
version for those who are risk averse,
3. add a mix-in layer (1) with the upgrade to 1.66.1.
  For langdale, we'd update from 1.63 and for kirkstone, we'd update
from 1.59
See the link above for a discussion about what Fedora/RHEL and other
distros are doing
  and a description of the rust / crates.io test system known as
crater that builds the
  world for any significant rust change.

Is there any objection to doing 2 and if we don't see any problems
after some time,
  then removing the older version?
Sundeep,
If no one object, please update kirkstone and see if librsvg or
python-cryptography or
  anything else encounters a problem.

I'm afraid I don't like option 2. We don't do this for anything else so
we're now inventing some new policy. Either the upgrade is what we
decide is right or it isn't, I don't really like the idea of hedging
our bets and providing two versions, one of which nobody will use until
they're forced. It will also make security scans tricky as is the issue
dealt with or not?

Just for context, going back in time OE used to be awash with many
version of recipes and I'm reluctant to go back there as it was
horrible. They were added for exactly this kind of reasoning, which at
first seemed like a good idea.

Okay, make sense, option 1 it is!

../Randy

Cheers,

Richard

--
# Randy MacLeod
# Wind River Linux

-- 
# Randy MacLeod
# Wind River Linux

On Tue, Jan 17, 2023 at 12:00 PM Alexander Kanavin
<alex.kanavin@...> wrote:

Option 1 looks like a new policy too. If we can upgrade rust across
many major versions in a stable release, then why not other items?

According to the stable release "rules" option 1 would require an
exception granted by the TSC. Otherwise it is a no go.

So it seems to me that this is a classic case for using a mix-in layer.

Steve

On Tue, 17 Jan 2023 at 22:57, Randy MacLeod <randy.macleod@...> wrote:

On 2023-01-17 16:54, Richard Purdie wrote:

On Tue, 2023-01-17 at 15:29 -0500, Randy MacLeod wrote:

On 2023-01-16 10:20, Kokkonda, Sundeep via lists.openembedded.org
wrote:

Rust community said the security fixes are only for the current
stable relases.

https://internals.rust-lang.org/t/cargo-cve-2022-46176-fix-for-older
-releases/18152/3?u=sundeep-kokkonda
For old release we've to backport the patches ourselves.

The other alternatives are
1. upgrade to 1.66.1 on older branches,
2. add 1.66.1, which will be the PREFERRED_VERSION but keep the older
version for those who are risk averse,
3. add a mix-in layer (1) with the upgrade to 1.66.1.
For langdale, we'd update from 1.63 and for kirkstone, we'd update
from 1.59
See the link above for a discussion about what Fedora/RHEL and other
distros are doing
and a description of the rust / crates.io test system known as
crater that builds the
world for any significant rust change.

Is there any objection to doing 2 and if we don't see any problems
after some time,
then removing the older version?
Sundeep,
If no one object, please update kirkstone and see if librsvg or
python-cryptography or
anything else encounters a problem.

I'm afraid I don't like option 2. We don't do this for anything else so
we're now inventing some new policy. Either the upgrade is what we
decide is right or it isn't, I don't really like the idea of hedging
our bets and providing two versions, one of which nobody will use until
they're forced. It will also make security scans tricky as is the issue
dealt with or not?

Just for context, going back in time OE used to be awash with many
version of recipes and I'm reluctant to go back there as it was
horrible. They were added for exactly this kind of reasoning, which at
first seemed like a good idea.

Okay, make sense, option 1 it is!

../Randy

Cheers,

Richard

--
# Randy MacLeod
# Wind River Linux

Option 1 looks like a new policy too. If we can upgrade rust across
many major versions in a stable release, then why not other items?

Alex

On 2023-01-17 16:54, Richard Purdie wrote:

On Tue, 2023-01-17 at 15:29 -0500, Randy MacLeod wrote:

On 2023-01-16 10:20, Kokkonda, Sundeep via lists.openembedded.org
wrote:

 Rust community said the security fixes are only for the current
stable relases.
https://internals.rust-lang.org/t/cargo-cve-2022-46176-fix-for-older
-releases/18152/3?u=sundeep-kokkonda
 For old release we've to backport the patches ourselves.

The other alternatives are
1. upgrade to 1.66.1 on older branches,
2. add 1.66.1, which will be the PREFERRED_VERSION but keep the older
version for those who are risk averse,
3. add a mix-in layer (1) with the upgrade to 1.66.1.
 For langdale, we'd update from 1.63 and for kirkstone, we'd update
from 1.59
See the link above for a discussion about what Fedora/RHEL and other
distros are doing
 and a description of the rust / crates.io test system known as
crater that builds the
 world for any significant rust change.

Is there any objection to doing 2 and if we don't see any problems
after some time,
 then removing the older version?
Sundeep,
If no one object, please update kirkstone and see if librsvg or
python-cryptography or
 anything else encounters a problem.

I'm afraid I don't like option 2. We don't do this for anything else so
we're now inventing some new policy. Either the upgrade is what we
decide is right or it isn't, I don't really like the idea of hedging
our bets and providing two versions, one of which nobody will use until
they're forced. It will also make security scans tricky as is the issue
dealt with or not?
Just for context, going back in time OE used to be awash with many
version of recipes and I'm reluctant to go back there as it was
horrible. They were added for exactly this kind of reasoning, which at
first seemed like a good idea.

Okay, make sense, option 1 it is!

../Randy

Cheers,
Richard

--
# Randy MacLeod
# Wind River Linux

On Tue, 2023-01-17 at 15:29 -0500, Randy MacLeod wrote:

On 2023-01-16 10:20, Kokkonda, Sundeep via lists.openembedded.org
wrote:
 

 Rust community said the security fixes are only for the current
stable relases.
 
https://internals.rust-lang.org/t/cargo-cve-2022-46176-fix-for-older
-releases/18152/3?u=sundeep-kokkonda
 For old release we've to backport the patches ourselves.

The other alternatives are
1. upgrade to 1.66.1 on older branches,
2. add 1.66.1, which will be the PREFERRED_VERSION but keep the older
version for those who are risk averse,
3. add a mix-in layer (1) with the upgrade to 1.66.1.
 For langdale, we'd update from 1.63 and for kirkstone, we'd update
from 1.59
See the link above for a discussion about what Fedora/RHEL and other
distros are doing
 and a description of the rust / crates.io test system known as
crater that builds the
 world for any significant rust change.

Is there any objection to doing 2 and if we don't see any problems
after some time,
 then removing the older version?
Sundeep,
If no one object, please update kirkstone and see if librsvg or
python-cryptography or
 anything else encounters a problem.

I'm afraid I don't like option 2. We don't do this for anything else so
we're now inventing some new policy. Either the upgrade is what we
decide is right or it isn't, I don't really like the idea of hedging
our bets and providing two versions, one of which nobody will use until
they're forced. It will also make security scans tricky as is the issue
dealt with or not?

Just for context, going back in time OE used to be awash with many
version of recipes and I'm reluctant to go back there as it was
horrible. They were added for exactly this kind of reasoning, which at
first seemed like a good idea.

Cheers,

Richard

Rust community said the security fixes are only for the current stable relases.
https://internals.rust-lang.org/t/cargo-cve-2022-46176-fix-for-older-releases/18152/3?u=sundeep-kokkonda
For old release we've to backport the patches ourselves.

The other alternatives are

1. upgrade to 1.66.1 on older branches,

2. add 1.66.1, which will be the PREFERRED_VERSION but keep the older version for those who are risk averse,

3. add a mix-in layer (1) with the upgrade to 1.66.1.

For langdale, we'd update from 1.63 and for kirkstone, we'd update from 1.59

See the link above for a discussion about what Fedora/RHEL and other distros are doing
and a description of the rust / crates.io test system known as crater that builds the
world for any significant rust change.

Is there any objection to doing 2 and if we don't see any problems after some time,
then removing the older version?

Sundeep,

If no one object, please update kirkstone and see if librsvg or python-cryptography or
anything else encounters a problem.

../Randy

1) https://wiki.yoctoproject.org/wiki/Stable_Release_and_LTS#LTS_.E2.80.9CMixin.E2.80.9D_repositories

So, for the Kirkstone & Langdale we've to back port the CVE fix.

-- 
# Randy MacLeod
# Wind River Linux

Rust community said the security fixes are only for the current stable relases.
https://internals.rust-lang.org/t/cargo-cve-2022-46176-fix-for-older-releases/18152/3?u=sundeep-kokkonda
For old release we've to backport the patches ourselves.

So, for the Kirkstone & Langdale we've to back port the CVE fix.

Topic created on Rust community https://internals.rust-lang.org/t/cargo-cve-2022-46176-fix-for-older-releases/18152

Based on the community feedback we will take a decision for Kirkstone & Langdale branches update.

On 2023-01-12 01:40, Alex Kiernan via lists.openembedded.org wrote:

Changes:
Added validation of SSH host keys for git URLs in Cargo (CVE-2022-46176)

Thanks Alex!

According to:
https://nvd.nist.gov/vuln/detail/CVE-2022-46176

"All Rust versions containing Cargo before 1.66.1 are vulnerable. "

so we'll have to fix:

kirkstone and langdale.



Sundeep, Naveen, or anyone,

Please find out what the upstream Rust team's plan is for older releases.

If they aren't going to release a dot update, we'll have to start back-porting the patches
listed here:

https://github.com/rust-lang/wg-security-response/tree/main/patches/CVE-2022-46176

--
# Randy MacLeod
# Wind River Linux

Changes:
Added validation of SSH host keys for git URLs in Cargo (CVE-2022-46176)

Signed-off-by: Alex Kiernan <alex.kiernan@...>
---

meta/recipes-devtools/rust/{cargo_1.66.0.bb => cargo_1.66.1.bb} | 0
.../rust/{libstd-rs_1.66.0.bb => libstd-rs_1.66.1.bb} | 0
...t-cross-canadian_1.66.0.bb => rust-cross-canadian_1.66.1.bb} | 0
.../rust/{rust-llvm_1.66.0.bb => rust-llvm_1.66.1.bb} | 0
meta/recipes-devtools/rust/rust-source.inc | 2 +-
meta/recipes-devtools/rust/{rust_1.66.0.bb => rust_1.66.1.bb} | 0
6 files changed, 1 insertion(+), 1 deletion(-)
rename meta/recipes-devtools/rust/{cargo_1.66.0.bb => cargo_1.66.1.bb} (100%)
rename meta/recipes-devtools/rust/{libstd-rs_1.66.0.bb => libstd-rs_1.66.1.bb} (100%)
rename meta/recipes-devtools/rust/{rust-cross-canadian_1.66.0.bb => rust-cross-canadian_1.66.1.bb} (100%)
rename meta/recipes-devtools/rust/{rust-llvm_1.66.0.bb => rust-llvm_1.66.1.bb} (100%)
rename meta/recipes-devtools/rust/{rust_1.66.0.bb => rust_1.66.1.bb} (100%)

diff --git a/meta/recipes-devtools/rust/cargo_1.66.0.bb b/meta/recipes-devtools/rust/cargo_1.66.1.bb
similarity index 100%
rename from meta/recipes-devtools/rust/cargo_1.66.0.bb
rename to meta/recipes-devtools/rust/cargo_1.66.1.bb
diff --git a/meta/recipes-devtools/rust/libstd-rs_1.66.0.bb b/meta/recipes-devtools/rust/libstd-rs_1.66.1.bb
similarity index 100%
rename from meta/recipes-devtools/rust/libstd-rs_1.66.0.bb
rename to meta/recipes-devtools/rust/libstd-rs_1.66.1.bb
diff --git a/meta/recipes-devtools/rust/rust-cross-canadian_1.66.0.bb b/meta/recipes-devtools/rust/rust-cross-canadian_1.66.1.bb
similarity index 100%
rename from meta/recipes-devtools/rust/rust-cross-canadian_1.66.0.bb
rename to meta/recipes-devtools/rust/rust-cross-canadian_1.66.1.bb
diff --git a/meta/recipes-devtools/rust/rust-llvm_1.66.0.bb b/meta/recipes-devtools/rust/rust-llvm_1.66.1.bb
similarity index 100%
rename from meta/recipes-devtools/rust/rust-llvm_1.66.0.bb
rename to meta/recipes-devtools/rust/rust-llvm_1.66.1.bb
diff --git a/meta/recipes-devtools/rust/rust-source.inc b/meta/recipes-devtools/rust/rust-source.inc
index bfb625fb363d..dcef50eeab60 100644
--- a/meta/recipes-devtools/rust/rust-source.inc
+++ b/meta/recipes-devtools/rust/rust-source.inc
@@ -1,6 +1,6 @@
RUST_VERSION ?= "${@d.getVar('PV').split('-')[0]}"
SRC_URI += "https://static.rust-lang.org/dist/rustc-${RUST_VERSION}-src.tar.xz;name=rust"
-SRC_URI[rust.sha256sum] = "0dc176e34fae9871f855a6ba4cb30fa19d69c5b4428d29281a07419c4950715c"
+SRC_URI[rust.sha256sum] = "07ac4e6c93e0d8ecfaf3b86c4c78bbbde3f5be675f0334e7fb343cb4a0b81ebe"

SRC_URI:append:class-target:pn-libstd-rs = "\
file://0001-Do-not-use-LFS64-on-linux-with-musl.patch;patchdir=../.. \
diff --git a/meta/recipes-devtools/rust/rust_1.66.0.bb b/meta/recipes-devtools/rust/rust_1.66.1.bb
similarity index 100%
rename from meta/recipes-devtools/rust/rust_1.66.0.bb
rename to meta/recipes-devtools/rust/rust_1.66.1.bb
--
2.39.0