promoting Rust to first class citizen in oe-core


Alexander Kanavin
 

Hello all,

I just read this article, called "Supporting Linux kernel development in Rust"
and it looks like the future is set, and particularly the Yocto project should prepare for it.

Thoughts?

Alex


Otavio Salvador
 

Em qui., 10 de set. de 2020 às 16:51, Alexander Kanavin
<alex.kanavin@gmail.com> escreveu:
I just read this article, called "Supporting Linux kernel development in Rust"
https://lwn.net/Articles/829858/
and it looks like the future is set, and particularly the Yocto project should prepare for it.

Thoughts?
I support this for sure. I've been using Rust with Yocto Project for a
while now and it does fit well to be in OE-Core.


--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750


Andreas Müller
 

On Thu, Sep 10, 2020 at 9:52 PM Alexander Kanavin
<alex.kanavin@gmail.com> wrote:

Hello all,

I just read this article, called "Supporting Linux kernel development in Rust"
https://lwn.net/Articles/829858/
and it looks like the future is set, and particularly the Yocto project should prepare for it.

Thoughts?
As a gnome in Yocto 'enthusiast' there is not much to say but: yes yes
yes. We have all these rust blockers as librsvg, mozjs. ->
gnome-shell/mutter

Andreas


Richard Purdie
 

On Thu, 2020-09-10 at 22:03 +0200, Andreas Müller wrote:
On Thu, Sep 10, 2020 at 9:52 PM Alexander Kanavin
<alex.kanavin@gmail.com> wrote:
Hello all,

I just read this article, called "Supporting Linux kernel
development in Rust"
https://lwn.net/Articles/829858/
and it looks like the future is set, and particularly the Yocto
project should prepare for it.

Thoughts?
As a gnome in Yocto 'enthusiast' there is not much to say but: yes
yes
yes. We have all these rust blockers as librsvg, mozjs. ->
gnome-shell/mutter
This has been talked about a lot but there is work to be done to get
this into core. Not many people seem willing to step up and do that
work so progress has been slow.

The hardest part may be getting the crate fetcher into bitbake in an
acceptable form.

I'm all in favour too, as long as it really is sorted to be a first
class citizen.

Cheers,

Richard


Alexander Kanavin
 

On Thu, 10 Sep 2020 at 22:24, Richard Purdie <richard.purdie@...> wrote:
This has been talked about a lot but there is work to be done to get
this into core. Not many people seem willing to step up and do that
work so progress has been slow.

The hardest part may be getting the crate fetcher into bitbake in an
acceptable form.

I'm all in favour too, as long as it really is sorted to be a first
class citizen.

That's why I specifically CCd Randy: he's done some work towards this, so I was hoping for some kind of current update or maybe remaining items where help is needed.

Alex


Khem Raj
 

On Sat, Sep 12, 2020 at 12:07 PM Alexander Kanavin
<alex.kanavin@gmail.com> wrote:

On Thu, 10 Sep 2020 at 22:24, Richard Purdie <richard.purdie@linuxfoundation.org> wrote:

This has been talked about a lot but there is work to be done to get
this into core. Not many people seem willing to step up and do that
work so progress has been slow.

The hardest part may be getting the crate fetcher into bitbake in an
acceptable form.

I'm all in favour too, as long as it really is sorted to be a first
class citizen.

That's why I specifically CCd Randy: he's done some work towards this, so I was hoping for some kind of current update or maybe remaining items where help is needed.
Rust and to a certain extent go has a bit different dynamics, where
the language tools are pretty much inherently cross compilers. they
provide easy installers and updaters for tools
and they release very often, they also have their own package
management systems, the programs are quite standalone in the ( like
static programs) so there is not much need for them
from system. end-users update compilers very often than not they are
using the latest compilers due to the above reasons.
these are real concerns when you consider timed releases schedules
like yocto and now we have LTS too.

I agree with Richard's sentiment that we need robust fetcher
integration as a starting point and perhaps full knowledge of
dependency management to offer a compelling solution. I would like to
see it used when in core and we need answers for the above topics,
currently, meta-rust e.g. follows a release cadence of its own which
is good fit for developers, maybe not as much for release
engineering.

Alex


Randy MacLeod
 

My filters hide this email in a folder even though I was CCed. Updated now.

On 2020-09-12 10:19 p.m., Khem Raj wrote:
On Sat, Sep 12, 2020 at 12:07 PM Alexander Kanavin
<alex.kanavin@...> wrote:
On Thu, 10 Sep 2020 at 22:24, Richard Purdie <richard.purdie@...> wrote:
This has been talked about a lot but there is work to be done to get
this into core. Not many people seem willing to step up and do that
work so progress has been slow.

The hardest part may be getting the crate fetcher into bitbake in an
acceptable form.
I was stuck on the librsvg build error for a while so
I haven't yet looked into how the cargo fetcher works.

The cargo bitbake tool can generate bitbake recipes from a toml file:

   https://github.com/meta-rust/cargo-bitbake

It certainly lists all the crates in the generated recipe's SRC_URI
and it looks like they are fetched during do_fetch but I haven't
done the fetch and then disabled the network to be sure yet.


I'm all in favour too, as long as it really is sorted to be a first
class citizen.

That's why I specifically CCd Randy: he's done some work towards this, so I was hoping for some kind of current update or maybe remaining items where help is needed.

Rust and to a certain extent go has a bit different dynamics, where
the language tools are pretty much inherently cross compilers. they
provide easy installers and updaters for tools
and they release very often, they also have their own package
management systems, the programs are quite standalone in the ( like
static programs) so there is not much need for them
from system.  end-users update compilers very often than not they are
using the latest compilers due to the above reasons.
these are real concerns when you consider timed releases schedules
like yocto and now we have LTS too.
Yes, I've been mulling that over as well.
Please could do app development using their
distro's rust or the 'rustup' toolchain by specifying

the target:

$ cargo build --target TRIPLE

and then for production releases, use bitbake.

I agree with Richard's sentiment that we need robust fetcher
integration as a starting point and perhaps full knowledge of
dependency management to offer a compelling solution. I would like to
see it used when in core and we need answers for the above topics,
currently, meta-rust e.g. follows a release cadence of its own which
is good fit for developers, maybe not as much for release
engineering.


After months of neglect, I did update the merge of
meta-rust to oe-core over the long weekend. Of course now
different things are broken than before!

Here's the oneline summary:

d3d419e11b (HEAD -> rust-wip-sept-5) librsvg: update to 2.49.5
dd921fee61 librsvg: Update from 2.40.20 to 2.46.4
ea484c5069 ripgrep: add temporarily
6a00ee0909 Add rust 1.46.0
7332db1316 rust: use PARALLEL_MAKE instead of BB_NUMBER_THREADS
8fab4132ee rust.inc: whitelist BB_NUMBER_THREADS in do_compile
dbf873714f Bump to Rust version 1.43
b40c54e810 Revert "cargo: fix progress output"
d223ab9b58 cargo: fix progress output
06e16e3475 rust.inc: cut build time in half
71dd219d97 rust.inc: run bootstrap.py in parallel
9e92ceda37 rust.inc: make max-atomic-width an integer
60ab501447 rust-native shouldn't depend on TARGET variables
afed138555 rustfmt: Upgrade to 1.4.2
50cde902c9 Avoid extra sh process from shell wrapper
0851bb0f1d Update 0001-Disable-http2.patch for cargo 1.41.0
787d064ca1 Update to Rust 1.41.0
756b950b5e rust: add a language demo image to test reproducibility
f543ee0909 cargo: Refresh http2 disable patch
e6ea4ca57c Update 0001-Disable-http2.patch for Cargo shipped with Rust 1.40.0
4189c968df Update to Rust and Cargo 1.40.0.
18af1ae487 rust: Use Python3 native for build
76de2d7175 rust: Improve TUNE_FEATURE parsing
043446750a Update to Rust and Cargo 1.39.0
5a962934f4 rust: mv README.md to recipes-devtools/rust/README-rust.md
b7f42be3f3 meta-rust: move code to oe-core from meta-rust layer
6d4d6b888f Add libgit2, libssh2 from meta-oe for rust
bdca4796ff (origin/master, origin/HEAD, master) weston-init: Enable RDP screen share


I'll get things in somewhat better order and push what I have to poky-contrib
or wherever so that other people can join in the fun or see what's going on.

The latest error when doing 'bitbake librsvg' on the newer librsvg is:

ERROR: librsvg-2.49.5-r0 do_compile: Execution of '/ala-lpggp31/rmacleod/src/distro/yocto/b/rust-sep-5/tmp-glibc/work/core2-64-oe-linux/librsvg/2.49.5-r0/temp/run.do_co:
error: more than one source location specified for `source.crates-io`
WARNING: exit code 101 from a shell command.

but bitbake rust-hello-world  or bitbake ripgrep works fine.


After I (we?) get librsvg to build again, I believe there were
problems with gstreamer as well.

../Randy


Alex


-- 
# Randy MacLeod
# Wind River Linux


Randy MacLeod
 

Added Jan from Pengutronix since Richard said he might be interested
in the rust in oe-core work.

See below if you want to kick the tires using my poky-contrib branch.

On 2020-09-14 12:24 p.m., Randy MacLeod wrote:
My filters hide this email in a folder even though I was CCed. Updated now.

On 2020-09-12 10:19 p.m., Khem Raj wrote:
On Sat, Sep 12, 2020 at 12:07 PM Alexander Kanavin
<alex.kanavin@...> wrote:
On Thu, 10 Sep 2020 at 22:24, Richard Purdie <richard.purdie@...> wrote:
This has been talked about a lot but there is work to be done to get
this into core. Not many people seem willing to step up and do that
work so progress has been slow.

The hardest part may be getting the crate fetcher into bitbake in an
acceptable form.
I was stuck on the librsvg build error for a while so
I haven't yet looked into how the cargo fetcher works.

The cargo bitbake tool can generate bitbake recipes from a toml file:

   https://github.com/meta-rust/cargo-bitbake

It certainly lists all the crates in the generated recipe's SRC_URI
and it looks like they are fetched during do_fetch but I haven't
done the fetch and then disabled the network to be sure yet.

I'm all in favour too, as long as it really is sorted to be a first
class citizen.
That's why I specifically CCd Randy: he's done some work towards this, so I was hoping for some kind of current update or maybe remaining items where help is needed.
Rust and to a certain extent go has a bit different dynamics, where
the language tools are pretty much inherently cross compilers. they
provide easy installers and updaters for tools
and they release very often, they also have their own package
management systems, the programs are quite standalone in the ( like
static programs) so there is not much need for them
from system.  end-users update compilers very often than not they are
using the latest compilers due to the above reasons.
these are real concerns when you consider timed releases schedules
like yocto and now we have LTS too.
Yes, I've been mulling that over as well.
Please could do app development using their
distro's rust or the 'rustup' toolchain by specifying

the target:

$ cargo build --target TRIPLE

and then for production releases, use bitbake.
I agree with Richard's sentiment that we need robust fetcher
integration as a starting point and perhaps full knowledge of
dependency management to offer a compelling solution. I would like to
see it used when in core and we need answers for the above topics,
currently, meta-rust e.g. follows a release cadence of its own which
is good fit for developers, maybe not as much for release
engineering.


After months of neglect, I did update the merge of
meta-rust to oe-core over the long weekend. Of course now
different things are broken than before!

Here's the oneline summary:

d3d419e11b (HEAD -> rust-wip-sept-5) librsvg: update to 2.49.5
dd921fee61 librsvg: Update from 2.40.20 to 2.46.4
ea484c5069 ripgrep: add temporarily
6a00ee0909 Add rust 1.46.0
7332db1316 rust: use PARALLEL_MAKE instead of BB_NUMBER_THREADS
8fab4132ee rust.inc: whitelist BB_NUMBER_THREADS in do_compile
dbf873714f Bump to Rust version 1.43
b40c54e810 Revert "cargo: fix progress output"
d223ab9b58 cargo: fix progress output
06e16e3475 rust.inc: cut build time in half
71dd219d97 rust.inc: run bootstrap.py in parallel
9e92ceda37 rust.inc: make max-atomic-width an integer
60ab501447 rust-native shouldn't depend on TARGET variables
afed138555 rustfmt: Upgrade to 1.4.2
50cde902c9 Avoid extra sh process from shell wrapper
0851bb0f1d Update 0001-Disable-http2.patch for cargo 1.41.0
787d064ca1 Update to Rust 1.41.0
756b950b5e rust: add a language demo image to test reproducibility
f543ee0909 cargo: Refresh http2 disable patch
e6ea4ca57c Update 0001-Disable-http2.patch for Cargo shipped with Rust 1.40.0
4189c968df Update to Rust and Cargo 1.40.0.
18af1ae487 rust: Use Python3 native for build
76de2d7175 rust: Improve TUNE_FEATURE parsing
043446750a Update to Rust and Cargo 1.39.0
5a962934f4 rust: mv README.md to recipes-devtools/rust/README-rust.md
b7f42be3f3 meta-rust: move code to oe-core from meta-rust layer
6d4d6b888f Add libgit2, libssh2 from meta-oe for rust
bdca4796ff (origin/master, origin/HEAD, master) weston-init: Enable RDP screen share


I'll get things in somewhat better order and push what I have to poky-contrib
or wherever so that other people can join in the fun or see what's going on.


Well, it's not in 'better order' but it builds and runs rust-hello-world and ripgrep.

I've pushed what I have to poky-contrib in case anyone wants to debug

the librsvg build over the weekend! :)

 * [new branch]              rmacleod/rust-wip-sept-5 -> rmacleod/rust-wip-sept-5

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=rmacleod/rust-wip-sept-5

../Randy

The latest error when doing 'bitbake librsvg' on the newer librsvg is:

ERROR: librsvg-2.49.5-r0 do_compile: Execution of '/ala-lpggp31/rmacleod/src/distro/yocto/b/rust-sep-5/tmp-glibc/work/core2-64-oe-linux/librsvg/2.49.5-r0/temp/run.do_co:
error: more than one source location specified for `source.crates-io`
WARNING: exit code 101 from a shell command.

but bitbake rust-hello-world  or bitbake ripgrep works fine.


After I (we?) get librsvg to build again, I believe there were
problems with gstreamer as well.

../Randy

Alex


-- 
# Randy MacLeod
# Wind River Linux


    


-- 
# Randy MacLeod
# Wind River Linux