Thoughts on the eSDK


Richard Purdie
 

I've been asked about my thoughts on the eSDK. I'm going to try and
write down some of my ideas and thinking about it. I don't have a fully
thought out plan to pull off the shelf but can at least share the
ideas.

In summary, I think many of the concepts in the eSDK are good but the
implementation isn't great, it is effectively a beta where we needed to
go back and learn from the implementation to improve things.

To understand why, keep in mind it was put together as an experiment.
The idea was to have an environment which rather than being an SDK or a
full build system, could be configured to be different steps in a
spectrum between those two positions. The goals included:

* using prebuild binary objects rather than building from source
* a prebuilt toolchain
* being able to generate customised images quickly
* could be updated to new versions of the underlying environment
* making commands like devtool available to the "SDK" user
* hiding the build system complexity from the "SDK" user where possible
* allowing usage as per the current "standard SDK".

The design decisions this resulted in were:

* being able to lock the sstate hashes to a known version
* using sstate as the binary object mechanism (so not package 
management based)
* needing to capture a locked down version of the metadata (i.e. the 
layers in use)

The eSDK as implemented did do many of the things it was designed to
do. As with most engineering, there were challenges that were
encountered, things that didn't work as expected and also unanticipated
issues. It really needed someone to go back over it armed with that
knowledge for a v2 and that is still where we stand today as
development stopped.

The things which bother me about our current situation:

* there is no way to unlock the sstate hash of a specific component
like a bblock/bbunlock command.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13425
* we need to work out the layer setup and capture it into the eSDK. If 
we had a standard mechanism for layer setup and config this would be 
much easier. The current approach is very sensitive to different 
layouts and lacks any tests for anything that isn't poky
* where sstate checksum mismatches occur, they are fiendishly 
difficult to debug (a wider problem with bitbake but they affect 
eSDK particularly badly)
* no mechanism to turn an existing build into an eSDK
* no mechanism to turn an eSDK install into a full build
* can't build eSDK within eSDK

You can see the list of open bugs against eSDK:

https://bugzilla.yoctoproject.org/buglist.cgi?quicksearch=esdk&list_id=649733

In particular there are the following issues:

* it only works targeting a system of the same as it is being built 
on (an aarch64 eSDK needs to be built on an aarch64 system)
* multilib may or may not work (and has no tests)
https://bugzilla.yoctoproject.org/show_bug.cgi?id=10693
* devtool commands may need to be disabled depending on context
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8789
* sdk-update has issues, e.g. 
https://bugzilla.yoctoproject.org/show_bug.cgi?id=10798
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14462
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14462
* several things don't build within the eSDK, e.g. ovmf, multilib, 
lttng, gstreamer-plugins-good


I've put off fixing some elements as I've hoped that if we fixed the
layer setup problem, it would then help elements of the eSDK. I think
we probably do need to go ahead and try and fix other elements without
that piece for now though (layer setup is a different problem in it's
own right).

I'm sharing this in the hopes it gets people thinking a little more and
can at least start the discussion.

Cheers,

Richard


Adrian Freihofer <adrian.freihofer@...>
 

Hi Richard

I've been asked about my thoughts on the eSDK. I'm going to try and
write down some of my ideas and thinking about it. I don't have a
fully
thought out plan to pull off the shelf but can at least share the
ideas.

In summary, I think many of the concepts in the eSDK are good but the
implementation isn't great, it is effectively a beta where we needed
to
go back and learn from the implementation to improve things.

To understand why, keep in mind it was put together as an experiment.
The idea was to have an environment which rather than being an SDK or
a
full build system, could be configured to be different steps in a
spectrum between those two positions. The goals included:

* using prebuild binary objects rather than building from source
* a prebuilt toolchain
* being able to generate customised images quickly
* could be updated to new versions of the underlying environment
* making commands like devtool available to the "SDK" user
* hiding the build system complexity from the "SDK" user where
possible
* allowing usage as per the current "standard SDK".
I think the goals are still valid.

The design decisions this resulted in were:

* being able to lock the sstate hashes to a known version
* using sstate as the binary object mechanism (so not package
management based)
* needing to capture a locked down version of the metadata (i.e. the
layers in use)
Also these principles in implementation continue to seem sensible to
me.

The eSDK as implemented did do many of the things it was designed to
do. As with most engineering, there were challenges that were
encountered, things that didn't work as expected and also
unanticipated
issues. It really needed someone to go back over it armed with that
knowledge for a v2 and that is still where we stand today as
development stopped.

The things which bother me about our current situation:

* there is no way to unlock the sstate hash of a specific component
like a bblock/bbunlock command.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13425
* we need to work out the layer setup and capture it into the eSDK.
If
we had a standard mechanism for layer setup and config this would be
much easier. The current approach is very sensitive to different
layouts and lacks any tests for anything that isn't poky
* where sstate checksum mismatches occur, they are fiendishly
difficult to debug (a wider problem with bitbake but they affect
eSDK particularly badly)
* no mechanism to turn an existing build into an eSDK
* no mechanism to turn an eSDK install into a full build
* can't build eSDK within eSDK

You can see the list of open bugs against eSDK:

https://bugzilla.yoctoproject.org/buglist.cgi?quicksearch=esdk&list_id=649733

In particular there are the following issues:

* it only works targeting a system of the same as it is being built
on (an aarch64 eSDK needs to be built on an aarch64 system)
* multilib may or may not work (and has no tests)
https://bugzilla.yoctoproject.org/show_bug.cgi?id=10693
* devtool commands may need to be disabled depending on context
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8789
* sdk-update has issues, e.g.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=10798
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14462
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14462
* several things don't build within the eSDK, e.g. ovmf, multilib,
lttng, gstreamer-plugins-good


I've put off fixing some elements as I've hoped that if we fixed the
layer setup problem, it would then help elements of the eSDK. I think
we probably do need to go ahead and try and fix other elements
without
that piece for now though (layer setup is a different problem in it's
own right).

I'm sharing this in the hopes it gets people thinking a little more
and
can at least start the discussion.
We have also spent some time trying different approaches to use the
eSDK. There are many things which work pretty well. But there are also
use cases where we were not successfully with using the eSDK so far.

There are probably two main technical reasons why some application
developers do not use it:
* Speed:
- devtool re-parses the recipes, which takes some time
- devtool sometimes forces a clean build which would not be necessary.
* IDE integration: VSCode, for example, allows devtool to be
integrated via the tasks.json configuration file, as well as
performing remote debugging, configured through a launch.json file.
However, it is complicated and not on the level of e.g. a cmake
plugin.

I would imagine that the following improvements would greatly enhance
the user experience of the eSDK and devtool:
* devtool/bitbake runs in memory-resident mode.
* devtool should offer an option to skip a "make clean" (On dunfell
this gets called by the configure task e.g. for cmake but also other
build frameworks).
* A plugin for VSCode could significantly simplify the integration of
devtool. Ideally, the plugin connects to the bitbake server via the
Tinfoil API instead of manually including devtool via the tasks.json
file. However, this is likely to be quite time-consuming to
implement. Would there perhaps be synergies with Toaster here?

The second big challenge we face is the amount of data we have to deal
with. We are currently creating full installers per machine (various
similar products) as the update implementation has not really worked
for us (stability, security, download speed).
If an SDK could support more than one MACHINE and more than one image,
this would also be beneficial.

We would really like to contribute to improve and fix the eSDK. I am
pleased to see that this topic made it to the top of the agenda of
https://www.openembedded.org/wiki/OEDVM_2022_05.

I see different approaches how the eSDK could be improved:
* Just fix the existing code and make it work as described in the
documentation.
* Discard the concept with the creation of an installer. I guess it is
feasible to provide the functionality of the eSDK directly in the
bitbake environment, which has access to a sstate-cache mirror. Not
sure if that's the idea behind "if we fixed the layer setup problem,
it would then help elements of the eSDK".
* Providing the SDK as an OCI container might be beneficial as well.
Ideally the container would be based on the poky distro and be
generated by bitbake only. (Not like crops where the installer is
installed into a Ubuntu container with additional tools processing a
Dockerfile). This could simplify the deployment, make it more robust
and decoupled from any host distribution. It could also be a basis
for the integration with e.g. VSCode.

Thank you very much for initiating this discussion and sharing your
thoughts.

Adrian
Cheers,

Richard


Pascal Bach
 

Hi Richard,

On Mon, 2022-05-09 at 15:30 +0100, Richard Purdie via
lists.openembedded.org wrote:
I've been asked about my thoughts on the eSDK. I'm going to try and
write down some of my ideas and thinking about it. I don't have a
fully
thought out plan to pull off the shelf but can at least share the
ideas.

In summary, I think many of the concepts in the eSDK are good but the
implementation isn't great, it is effectively a beta where we needed
to
go back and learn from the implementation to improve things.

To understand why, keep in mind it was put together as an experiment.
The idea was to have an environment which rather than being an SDK or
a
full build system, could be configured to be different steps in a
spectrum between those two positions. The goals included:

* using prebuild binary objects rather than building from source
* a prebuilt toolchain
* being able to generate customised images quickly
* could be updated to new versions of the underlying environment
* making commands like devtool available to the "SDK" user
* hiding the build system complexity from the "SDK" user where
possible
* allowing usage as per the current "standard SDK".
I would like to give some input from an application developer
perspective here. By this I mean using the eSDK like a "standard SDK",
developing a single application working directly with CMake, in an IDE
like VSCode.

For this use case it works well. What's missing from my perspective is
the following:

1. A way to update any dependency included in the eSDK and make it
available in the sysroots, so it can be consumed by CMake. This would
make it really easy for developers to test their change against a new
or patched version of a dependency.
2. A reliable way to update the eSDK to avoid re-downloading again.


The design decisions this resulted in were:

* being able to lock the sstate hashes to a known version
* using sstate as the binary object mechanism (so not package 
  management based)
* needing to capture a locked down version of the metadata (i.e. the 
  layers in use)

The eSDK as implemented did do many of the things it was designed to
do. As with most engineering, there were challenges that were
encountered, things that didn't work as expected and also
unanticipated
issues. It really needed someone to go back over it armed with that
knowledge for a v2 and that is still where we stand today as
development stopped.

The things which bother me about our current situation:

* there is no way to unlock the sstate hash of a specific component
  like a bblock/bbunlock command.
 
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.yoctoproject.org%2Fshow_bug.cgi%3Fid%3D13425&;data=05%7C01%7Cpascal.bach%40siemens.com%7C238052c120044493583408da31c88936%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637877034611563685%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=2Xu03DUqi7UCCVe40bg5QL1yigJUM84Z7pWiDAU60gQ%3D&amp;reserved=0
* we need to work out the layer setup and capture it into the eSDK.
If 
  we had a standard mechanism for layer setup and config this would
be 
  much easier. The current approach is very sensitive to different 
  layouts and lacks any tests for anything that isn't poky
* where sstate checksum mismatches occur, they are fiendishly 
  difficult to debug (a wider problem with bitbake but they affect 
  eSDK particularly badly)
* no mechanism to turn an existing build into an eSDK
* no mechanism to turn an eSDK install into a full build
* can't build eSDK within eSDK
At this I would also throw a completley different idea into the ring
for discussion. It is to get rid of the eSDK in it's curent form.

A bit of context: I have worked a lot with https://nixos.org/ recently
and I think it has some similarities to bitbake. The important
similarity here is that both are source based systems that can make use
of a cache to accelerate the build.

In nix the way you develop an application (e.g via CMake) is that you
open the .nix file (similar to a .bb recipe) via nix-shell. This will
drop you in a shell where all your dependencies, build tools and
envioronment variables are available, ready to build your software.

I think a similar approach could also be possible with bitbake/devtool
as it is pretty close to what we already have with the devshell.

So a potential workflow for an "sdkenv" would look as follows:

1. Download a preconfigured bitbake environment with a bundeled sstate.
Could be a tarball, installer or container => This gives you a starting
point as an application. Alternative to bundeling sstate, it could also
be provided from a mirror on demand
2. Run "devtool sdkenv <recipe>" => This assembles an environment based
on <recipes>s sysroot and drop you into a devshell like environment
where you can build your the <recipe> aplication, e.g via CMake.
3. To make use of a new version of the dependency, just update the
<dependency-recipe> and then run "devtool sdkenv <recipe>" again. This
will build <dependency-recipe> install it in <recipe> sysroot and drop
you in a new devshell. Now you can continue to build your application
but with the new dependency.


The advantages of this approach is:
1. The bitbake environment and the "sdkenv" are the same. The only
difference is how you use it (devtool or bitbake) and how much you
reley on the sstate cache.
2. Developers always have the full power of bitbake available. So it's
easy to update dependencies, crate new sdk and even do completely new
builds.
3. There is no need to crate a new eSDK installer as you can always
test it directly from within bitbake by calling devtool sdkenv <recipe>
4. Updates are simple as it just means updating the recipes and
providing an sstate-mirror.

What do you think? Is this feasible or even desired?


Pascal


Alexander Kanavin
 

So let me summarize where I think things should be heading:

1. Any new sdk work should be aiming to adjust the 'core yocto
workflow', e.g. a standard layer checkout, and not create or extend
any separate workflows with their own input artifacts, tests, and code
paths.

2. bblock/bbunlock as core, supported, testable and tested
functionality, and not an ad hoc addition to esdk.

3. devtool sdk [recipe] drops you into an environment identical to
that of the standard sdk, with source code for a component optionally
set up for builds (using component-level toolchain as in the sdk).

4. Layer and environment config: until we have that, kas +
TEMPLATECONF could be an interim way to set things up. The tool in the
core project should be bi-directional, and be able to both capture the
state of the configuration, and restore it from the captured state.
Sstate can be captured/restored together with layers and configs, or
it can be skipped with the assumption that it will be provided from a
server.

Thoughts? Anything about this that you think is unworkable, or
something obvious I completely missed?

Alex

On Wed, 18 May 2022 at 09:50, Pascal Bach <pascal.bach@...> wrote:

Hi Richard,

On Mon, 2022-05-09 at 15:30 +0100, Richard Purdie via
lists.openembedded.org wrote:
I've been asked about my thoughts on the eSDK. I'm going to try and
write down some of my ideas and thinking about it. I don't have a
fully
thought out plan to pull off the shelf but can at least share the
ideas.

In summary, I think many of the concepts in the eSDK are good but the
implementation isn't great, it is effectively a beta where we needed
to
go back and learn from the implementation to improve things.

To understand why, keep in mind it was put together as an experiment.
The idea was to have an environment which rather than being an SDK or
a
full build system, could be configured to be different steps in a
spectrum between those two positions. The goals included:

* using prebuild binary objects rather than building from source
* a prebuilt toolchain
* being able to generate customised images quickly
* could be updated to new versions of the underlying environment
* making commands like devtool available to the "SDK" user
* hiding the build system complexity from the "SDK" user where
possible
* allowing usage as per the current "standard SDK".
I would like to give some input from an application developer
perspective here. By this I mean using the eSDK like a "standard SDK",
developing a single application working directly with CMake, in an IDE
like VSCode.

For this use case it works well. What's missing from my perspective is
the following:

1. A way to update any dependency included in the eSDK and make it
available in the sysroots, so it can be consumed by CMake. This would
make it really easy for developers to test their change against a new
or patched version of a dependency.
2. A reliable way to update the eSDK to avoid re-downloading again.


The design decisions this resulted in were:

* being able to lock the sstate hashes to a known version
* using sstate as the binary object mechanism (so not package
management based)
* needing to capture a locked down version of the metadata (i.e. the
layers in use)

The eSDK as implemented did do many of the things it was designed to
do. As with most engineering, there were challenges that were
encountered, things that didn't work as expected and also
unanticipated
issues. It really needed someone to go back over it armed with that
knowledge for a v2 and that is still where we stand today as
development stopped.

The things which bother me about our current situation:

* there is no way to unlock the sstate hash of a specific component
like a bblock/bbunlock command.

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.yoctoproject.org%2Fshow_bug.cgi%3Fid%3D13425&;data=05%7C01%7Cpascal.bach%40siemens.com%7C238052c120044493583408da31c88936%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637877034611563685%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=2Xu03DUqi7UCCVe40bg5QL1yigJUM84Z7pWiDAU60gQ%3D&amp;reserved=0
* we need to work out the layer setup and capture it into the eSDK.
If
we had a standard mechanism for layer setup and config this would
be
much easier. The current approach is very sensitive to different
layouts and lacks any tests for anything that isn't poky
* where sstate checksum mismatches occur, they are fiendishly
difficult to debug (a wider problem with bitbake but they affect
eSDK particularly badly)
* no mechanism to turn an existing build into an eSDK
* no mechanism to turn an eSDK install into a full build
* can't build eSDK within eSDK
At this I would also throw a completley different idea into the ring
for discussion. It is to get rid of the eSDK in it's curent form.

A bit of context: I have worked a lot with https://nixos.org/ recently
and I think it has some similarities to bitbake. The important
similarity here is that both are source based systems that can make use
of a cache to accelerate the build.

In nix the way you develop an application (e.g via CMake) is that you
open the .nix file (similar to a .bb recipe) via nix-shell. This will
drop you in a shell where all your dependencies, build tools and
envioronment variables are available, ready to build your software.

I think a similar approach could also be possible with bitbake/devtool
as it is pretty close to what we already have with the devshell.

So a potential workflow for an "sdkenv" would look as follows:

1. Download a preconfigured bitbake environment with a bundeled sstate.
Could be a tarball, installer or container => This gives you a starting
point as an application. Alternative to bundeling sstate, it could also
be provided from a mirror on demand
2. Run "devtool sdkenv <recipe>" => This assembles an environment based
on <recipes>s sysroot and drop you into a devshell like environment
where you can build your the <recipe> aplication, e.g via CMake.
3. To make use of a new version of the dependency, just update the
<dependency-recipe> and then run "devtool sdkenv <recipe>" again. This
will build <dependency-recipe> install it in <recipe> sysroot and drop
you in a new devshell. Now you can continue to build your application
but with the new dependency.


The advantages of this approach is:
1. The bitbake environment and the "sdkenv" are the same. The only
difference is how you use it (devtool or bitbake) and how much you
reley on the sstate cache.
2. Developers always have the full power of bitbake available. So it's
easy to update dependencies, crate new sdk and even do completely new
builds.
3. There is no need to crate a new eSDK installer as you can always
test it directly from within bitbake by calling devtool sdkenv <recipe>
4. Updates are simple as it just means updating the recipes and
providing an sstate-mirror.

What do you think? Is this feasible or even desired?


Pascal




Alexander Kanavin
 

On Wed, 18 May 2022 at 09:50, Pascal Bach <pascal.bach@...> wrote:

The advantages of this approach is:
1. The bitbake environment and the "sdkenv" are the same. The only
difference is how you use it (devtool or bitbake) and how much you
reley on the sstate cache.
2. Developers always have the full power of bitbake available. So it's
easy to update dependencies, crate new sdk and even do completely new
builds.
3. There is no need to crate a new eSDK installer as you can always
test it directly from within bitbake by calling devtool sdkenv <recipe>
4. Updates are simple as it just means updating the recipes and
providing an sstate-mirror.

What do you think? Is this feasible or even desired?
Yes, it's feasible and I have a working proof of concept now:
https://lists.openembedded.org/g/openembedded-core/message/167226

Please take a look, try it out and let me know!

Alex


Pascal Bach
 

Hi Alex,

On Wed, 2022-06-22 at 12:38 +0200, Alexander Kanavin via
lists.openembedded.org wrote:
On Wed, 18 May 2022 at 09:50, Pascal Bach <pascal.bach@...>
wrote:

The advantages of this approach is:
1. The bitbake environment and the "sdkenv" are the same. The only
difference is how you use it (devtool or bitbake) and how much you
reley on the sstate cache.
2. Developers always have the full power of bitbake available. So
it's
easy to update dependencies, crate new sdk and even do completely
new
builds.
3. There is no need to crate a new eSDK installer as you can always
test it directly from within bitbake by calling devtool sdkenv
<recipe>
4. Updates are simple as it just means updating the recipes and
providing an sstate-mirror.

What do you think? Is this feasible or even desired?
Yes, it's feasible and I have a working proof of concept now:
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openembedded.org%2Fg%2Fopenembedded-core%2Fmessage%2F167226&;data=05%7C01%7Cpascal.bach%40siemens.com%7Ce78c057468254b68ea1f08da543b5dd9%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637914911194376193%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=68pCFUkCW%2FYN%2FBddnsZh6v1GQmZE2d1TK7lmeoVlQXo%3D&amp;reserved=0

Please take a look, try it out and let me know!
I tried out your patches and they do almost exactly what I described.
Thanks a lot for taking this up.

Being able to source the environment instead of just spawning a sub-
shell is even more convenient.

I also think it is sufficient to just run bitbake -c
prepare_recipe_sysroot <application> instead of the full
populate_sysroot to avoid building the full application, which might
take a long time.

What I tested is:

- Building an SDK for an application via:
bitbake meta-ide-support
bitbake -c prepare_recipe_sysroot <application>
bitbake build-sysroots

. build/tmp/environment-setup-aarch64-ccp-linux

"build <application> with cmake"

- Updating a dependency and using it in the sdk via:
"modify poco recipe"
bitbake poco
bitbake build-sysroots

"build <application> with cmake"

Both work and provide a very convenient way to deal with the
dependencies.

Using a local sstate the setup of the sdk only takes a few minutes. I
haven't tested the time using an sstate mirror.

Pascal


Alexander Kanavin
 

Cheers, I just want to clarify that populate_sysroot is used when you
want to place something into the unified sysroot as a build
dependency. Perhaps because you want to provide that dependency to an
aplication you're working on, as that application may not yet even
have a recipe of its own.

prepare_recipe_sysroot is when that application recipe exists, and you
just want to place all of its existing dependencies into the unified
sysroot with a single command.

I'm now prototyping the other missing piece for the 'direct SDK
workflow', which is layer management. Then we can look at
bblock/bbunlock.

Alex

On Wed, 29 Jun 2022 at 11:13, Bach, Pascal <pascal.bach@...> wrote:

Hi Alex,

On Wed, 2022-06-22 at 12:38 +0200, Alexander Kanavin via
lists.openembedded.org wrote:
On Wed, 18 May 2022 at 09:50, Pascal Bach <pascal.bach@...>
wrote:

The advantages of this approach is:
1. The bitbake environment and the "sdkenv" are the same. The only
difference is how you use it (devtool or bitbake) and how much you
reley on the sstate cache.
2. Developers always have the full power of bitbake available. So
it's
easy to update dependencies, crate new sdk and even do completely
new
builds.
3. There is no need to crate a new eSDK installer as you can always
test it directly from within bitbake by calling devtool sdkenv
<recipe>
4. Updates are simple as it just means updating the recipes and
providing an sstate-mirror.

What do you think? Is this feasible or even desired?
Yes, it's feasible and I have a working proof of concept now:
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openembedded.org%2Fg%2Fopenembedded-core%2Fmessage%2F167226&;data=05%7C01%7Cpascal.bach%40siemens.com%7Ce78c057468254b68ea1f08da543b5dd9%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637914911194376193%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=68pCFUkCW%2FYN%2FBddnsZh6v1GQmZE2d1TK7lmeoVlQXo%3D&amp;reserved=0

Please take a look, try it out and let me know!
I tried out your patches and they do almost exactly what I described.
Thanks a lot for taking this up.

Being able to source the environment instead of just spawning a sub-
shell is even more convenient.

I also think it is sufficient to just run bitbake -c
prepare_recipe_sysroot <application> instead of the full
populate_sysroot to avoid building the full application, which might
take a long time.

What I tested is:

- Building an SDK for an application via:
bitbake meta-ide-support
bitbake -c prepare_recipe_sysroot <application>
bitbake build-sysroots

. build/tmp/environment-setup-aarch64-ccp-linux

"build <application> with cmake"

- Updating a dependency and using it in the sdk via:
"modify poco recipe"
bitbake poco
bitbake build-sysroots

"build <application> with cmake"

Both work and provide a very convenient way to deal with the
dependencies.

Using a local sstate the setup of the sdk only takes a few minutes. I
haven't tested the time using an sstate mirror.

Pascal



Matthias Klein
 

I have not been working with the eSDK so far.

We are heavy users of the standard SDK, and use it daily in the team for application development.
We use it with QtCreator and CMake including remote debugging. During setup, we obtain a single time information from the environment to create the kit within QtCreator.
Later, when using the kit, the environment is no longer used. This has the advantage that you can use multiple SDKs for different devices and versions at the same time in QtCreator.

Please don't eliminate the standard SDK with its separate installer and independence to BitBake etc.
From the user perspective of an application developer it is perfect.

Best regards,
Matthias


Alexander Kanavin
 

The standard SDK is not going anywhere. Obviously we're not removing
something that has a lot of existing users.

However I think it's still worth for you to play with the 'direct SDK'
(the patches just landed in master) [1]. The built-in extensibility
and ability to trivially update everything is beneficial. But you do
need a bulletproof sstate infra if the developers run things on
underpowered laptops.

[1] https://git.yoctoproject.org/poky/commit/?id=9b3fcb0d91648ae3b53ec8ffcb31fb6eac9209dd

Alex

On Fri, 29 Jul 2022 at 08:37, Matthias Klein <matthias.klein@...> wrote:

I have not been working with the eSDK so far.

We are heavy users of the standard SDK, and use it daily in the team for application development.
We use it with QtCreator and CMake including remote debugging. During setup, we obtain a single time information from the environment to create the kit within QtCreator.
Later, when using the kit, the environment is no longer used. This has the advantage that you can use multiple SDKs for different devices and versions at the same time in QtCreator.

Please don't eliminate the standard SDK with its separate installer and independence to BitBake etc.
From the user perspective of an application developer it is perfect.

Best regards,
Matthias




Matthias Klein
 

I have not been working with the eSDK so far.

We are heavy users of the standard SDK, and use it daily in the team for application development.
We use it with QtCreator and CMake including remote debugging. During setup, we obtain a single time information from the environment to create the kit within QtCreator.
Later, when using the kit, the environment is no longer used. This has the advantage that you can use multiple SDKs for different devices and versions at the same time in QtCreator.

Please don't eliminate the standard SDK with its separate installer and independence to BitBake etc.
From the user perspective of an application developer it is perfect.

Best regards,
Matthias


Konrad Weihmann <kweihmann@...>
 



On 29 Jul 2022 10:15, Alexander Kanavin <alex.kanavin@...> wrote:

The standard SDK is not going anywhere. Obviously we're not removing
something that has a lot of existing users.

However I think it's still worth for you to play with the 'direct SDK'
(the patches just landed in master) [1]. The built-in extensibility
and ability to trivially update everything is beneficial. But you do
need a bulletproof sstate infra if the developers run things on
underpowered laptops.


This is the point I'm most worried about, keeping in mind that most of the users need to use 3rd party layers which aren't sstate stable at all, there might be a chance that this new approach doesn't really work well, even though I really like the idea of reusing the caches for this purpose.



[1] https://git.yoctoproject.org/poky/commit/?id=9b3fcb0d91648ae3b53ec8ffcb31fb6eac9209dd

Alex

On Fri, 29 Jul 2022 at 08:37, Matthias Klein <matthias.klein@...> wrote:
>
> I have not been working with the eSDK so far.
>
> We are heavy users of the standard SDK, and use it daily in the team for application development.
> We use it with QtCreator and CMake including remote debugging. During setup, we obtain a single time information from the environment to create the kit within QtCreator.
> Later, when using the kit, the environment is no longer used. This has the advantage that you can use multiple SDKs for different devices and versions at the same time in QtCreator.
>
> Please don't eliminate the standard SDK with its separate installer and independence to BitBake etc.
> From the user perspective of an application developer it is perfect.
>
> Best regards,
> Matthias
>
>
>
>