-----Original Message-----
From: openembedded-architecture@... <openembedded-
architecture@...> On Behalf Of Richard Purdie
Sent: den 16 juli 2021 12:09
To: Nicolas Dechesne <nicolas.dechesne@...>
Cc: openembedded-architecture <openembedded-
architecture@...>
Subject: Re: [Openembedded-architecture] Should we change variable
override formatting?

On Fri, 2021-07-16 at 10:12 +0200, Nicolas Dechesne wrote:

hey!

On Thu, Jul 15, 2021 at 3:56 PM Richard Purdie

<richard.purdie@...> wrote:

b) Some of the packaging code (at the very least) needs rewriting as

it accesses

   both RDEPENDS_${PN} and puts ${PN} in OVERRIDES and accesses

RDEPENDS. I'm not

   sure what else may be assuming this works.

Ouch.. I never realized that! I like Mark's suggestion to convert that

to variables. RDEPENDS is typically

used in users' layers, so that  way they wouldn't be impacted by an

override syntax change.

I have to admit I'm leaning the other way, make this explicit and hope
that
it actually helps users understand overrides a little better by making it
clear.

This change does buy us cleaner looking metadata and ultimately, a

faster

and cleaner internal bitbake that at least would use less memory. It

could set

the stage to allow the defval idea I mentioned in a previous mail to

happen.

It is a huge change and would need a lot of work to make it happen. Is

it worth

doing? Not sure. I'm putting it out there for discussion.

There is no doubt that the change is a good change and the project would

benefit

from it. However I think we must worry about our users, and even look

beyond the

layer maintainers that we know (e.g. the one on this list). There are

*way* more

layer maintainers that we don't know about in all the many companies who

are

successfully using YP to build their products. I am not worried about

all the main

layers at all, since we know all their maintainers, and we know they

will understand

why we make this change, and will make the effort to support it. But I

am very

worried about the hundreds (thousands??) of layer maintainers in all the

companies

using Yocto.

So if we do anything, I would really prefer if we find a way to do it in

a good way

*for them* , not *for us*. which means finding a way to support a soft

transition.

And of course we need to align this change with our LTS release cycle.

So perhaps

we can figure out how to support both syntaxes for some time (from one

LTS to the

next?), even if supporting both is impacting the performance. Or we can

have a layer

flag that indicates if it uses the old or the new syntax?

If we support both forms, firstly, few of the layers you're concerned
about will
change as it will break older compatibility and they have no incentive to
do so.

As I mentioned in another reply, there is a way some backwards
compatibility could
be added however I do worry about the horrible corner cases that may
result in.

Secondly, supporting both means we can't take advantage of any benefits
the change
brings to further improve for a period of time. Do we want to commit to
making this
change so that we can benefit from it in say four years time (a couple of
LTS cycles
you mention)?

I'm worried about the pain of transitioning, but I'm also very worried
that unless
we figure out how we can change something, we're not going to develop
beyond where
we are now.

I'm trying hard to find any way we can move forward with some of the
issues people
are raising and I'm not seeing many options. I'm not seeing any proposals
from anyone
else either, probably as there is so much inertia to overcome to make any
change :(.

Cheers,

Richard

This is a long answer, and responding to multiple subjects that have been
brought up in the different threads, so please bear with me.

I would love to see improvements to the bitbake syntax when it comes to
overrides. Unfortunately, I do not see how I would be able to ever update
to said version if it is done as a flag day release. The way we work is
to have our layers compatible with the version of bitbake we are using
(currently Hardknott). Then I (who basically do all work to build our
layers with master of Poky) have a special adaptation layer where I add
bbappends and some occasional backported bbclass to make our layers work
with Poky master. This work runs in parallel with the normal recipe updates
that the developers do so that when a new major release of Poky is ready,
so are we. However, if we would have to use a whole different version of
all our recipes, there is no way this process would work, and I can't stop
the normal development for a flag day upgrade of Poky. :(

The way I see it can be done would involve supporting both syntaxes in
one LTS-release. In the next non-LTS release, the support for the old
syntax can be dropped. That way there is always an intermediate release
that one can update to, update one's own recipes and then proceed to the
latest release. This of course means that the Yocto Project cannot reap
the benefits of the new and improved syntax immediately, but would have
to hold off for one release. On the other hand, this would mean the
transitioning path for all users of the Yocto Project would be trivial
(for some definition of trivial), compared to a major PITA.

Now, I don't know if supporting both syntaxes at once is feasible given
some of the comments given in the thread. However, most of them seem
related to being able to drop support for the old syntax and thereby
gaining the expected improvements to the internal data, which would not
be a goal for the release that actually has to support both syntaxes.
Its goal is compatibility and a way forward.


As to the actual syntax changes, one thing I would like to se is the
_append/_prepend/_remove operators actually turn into real operators.
I definitely do not want to see the same syntax for the operators as
for overrides (that is one of the most confusing things with the
current solution).

Here is one suggestion:

PACKAGECONFIG_append = "foo" => PACKAGECONFIG{} .= "foo"
PACKAGECONFIG_append = " foo" => PACKAGECONFIG{} += "foo"
PACKAGECONFIG_prepend = "foo " => PACKAGECONFIG{} =. "foo"
PACKAGECONFIG_prepend = " foo " => PACKAGECONFIG{} =+ "foo"
PACKAGECONFIG_remove = "foo " => PACKAGECONFIG{} -= "foo"

PACKAGECONFIG_append_bar = "foo" => PACKAGECONFIG{bar} .= "foo"
PACKAGECONFIG_append_bar = " foo" => PACKAGECONFIG{bar} += "foo"
PACKAGECONFIG_prepend_bar = "foo " => PACKAGECONFIG{bar} =. "foo"
PACKAGECONFIG_prepend_bar = " foo " => PACKAGECONFIG{bar} =+ "foo"
PACKAGECONFIG_remove_bar = "foo " => PACKAGECONFIG{bar} -= "foo"

PACKAGECONFIG_append_bar_foo = "foo" => PACKAGECONFIG{bar&foo} .= "foo"
PACKAGECONFIG_append_bar_foo = " foo" => PACKAGECONFIG{bar&foo} += "foo"
PACKAGECONFIG_prepend_bar_foo = "foo " => PACKAGECONFIG{bar&foo} =. "foo"
PACKAGECONFIG_prepend_bar_foo = " foo " => PACKAGECONFIG{bar&foo} =+ "foo"
PACKAGECONFIG_remove_bar_foo = "foo " => PACKAGECONFIG{bar&foo} -= "foo"

This would not have a corresponding solution for, e.g.,
PACKAGECONFIG_bar_foo_append, but I actually think that is good.

This would also allow for clearing out the pending appends for a
given override by doing:

PACKAGECONFIG{} = ""

or:

PACKAGECONFIG{bar} = ""

And a totally different solution (and probably more controversial) would
be to add a more programmatical way to handle overrides, e.g.:

PACKAGECONFIG += "foo" if overrides("foo bar")

or:

if override("foo") && override("bar"): PACKAGECONFIG += "foo"

It doesn't help with whether the update to the variable is immediate or
deferred though.


As to RDEPENDS_${PN} & co, wouldn't it make sense to use RDEPENDS_pn-${PN},
since "pn-<package name>" already exists as an override? On that note,
I would like to see overrides being prefixed more (maybe even as a
requirement). Some suggestions:

pn- - Package name
df- - Distro feature
mf- - Machine feature
pf- - Package feature (can we rename/alias PACKAGECONFIG to
PACKAGE_FEATURES while at it, for consistency?)
arch- - Architecture
dist- - Distribution

That way it would be more obvious what an override does. Especially the
use of the distro name as an override can have some unforeseen consequences
if the name is somewhat common).

//Peter