On Tue, Jun 15, 2021 at 10:59:40PM +0100, Richard Purdie wrote:

It certainly could be a solution to the problem but I worry that having
variables that users should touch and not touch and variables shadowing
other variables is going to get complex rather quickly, even when you just
consider how you document or "enforce" it. Do we want to encourage what
could become many more intermiediate variables? It does sound like we're 
effectively starting to write class "functions" with a list of variables
they operate on which may or may not be a good thing depending on how 
common it became.

Yes, that's a fair point. As you say, I kind of have the sense that our
classes are drifting towards being more like classes in a "real" programming
language, complete with an actual API, and possibly we ought to start
thinking about them more in those terms. As in, "if you include image.bbclass,
this is the set of variables it takes as input, and you need to give them
appropriate values".

From descriptions from others it sounds like people have creatively worked
around this kind of issue with stronger overrides which I'd not considered
and would also work, it is just a little ugly in a different way. I'd consider 
use of the forcevariable override in oe-core a metadata failing (much like 
I view _remove similarly in core).

I agree. Although it's true that _forcevariable probably would fix this
particular issue I think there's a fair chance that it will just end up creating
the opposite problem at some point in the future. I don't honestly think there
is any situation where using _forcevariable in oe-core itself would be the
right thing to do.

I suppose the other obvious tactical answer to the problem at hand is for
meta-zaurus to stop using an override at all for what it's doing. Although
possibly not quite so elegant, there's no reason that it couldn't do:

IMAGE_FSTYPES = "ext3"
# nb, include not require...
include conf/machine/zaurus/${MACHINE}-special.conf

and then in conf/machine/zaurus/collie-special.conf it could just set
IMAGE_FSTYPES = "squashfs"

directly without needing any kind of override. That said I don't entirely
understand why meta-zaurus doesn't already have a collie.conf (assuming
MACHINE=collie) where it could be doing this...

p.

Em qua., 16 de jun. de 2021 às 02:08, Chen Qi <Qi.Chen@...> escreveu:

Anyone proposed '==='? (I'm thinking about the 'always equal' sign in math.)

I really like the === proposal.

--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750

On Wed, Jun 16, 2021 at 2:22 AM Phil Blundell via
lists.openembedded.org <pb=pbcl.net@...> wrote:

On Tue, Jun 15, 2021 at 10:59:40PM +0100, Richard Purdie wrote:

It certainly could be a solution to the problem but I worry that having
variables that users should touch and not touch and variables shadowing
other variables is going to get complex rather quickly, even when you just
consider how you document or "enforce" it. Do we want to encourage what
could become many more intermiediate variables? It does sound like we're
effectively starting to write class "functions" with a list of variables
they operate on which may or may not be a good thing depending on how
common it became.

Yes, that's a fair point. As you say, I kind of have the sense that our
classes are drifting towards being more like classes in a "real" programming
language, complete with an actual API, and possibly we ought to start
thinking about them more in those terms. As in, "if you include image.bbclass,
this is the set of variables it takes as input, and you need to give them
appropriate values".

From descriptions from others it sounds like people have creatively worked
around this kind of issue with stronger overrides which I'd not considered
and would also work, it is just a little ugly in a different way. I'd consider
use of the forcevariable override in oe-core a metadata failing (much like
I view _remove similarly in core).

I agree. Although it's true that _forcevariable probably would fix this
particular issue I think there's a fair chance that it will just end up creating
the opposite problem at some point in the future. I don't honestly think there
is any situation where using _forcevariable in oe-core itself would be the
right thing to do.

I'm not sure I understand this. Why is using the _forcevariable
override somehow worse than inventing a new assignment operator to do
the same thing?

If we want to make a rule that nothing in oe-core should ever
forcefully override a setting provided by a BSP then that's OK... but
then isn't that an argument that a new assignment operator to
forcefully assign to variables shouldn't be used in oe-core?

I suppose the other obvious tactical answer to the problem at hand is for

meta-zaurus to stop using an override at all for what it's doing. Although
possibly not quite so elegant, there's no reason that it couldn't do:

IMAGE_FSTYPES = "ext3"
# nb, include not require...
include conf/machine/zaurus/${MACHINE}-special.conf

and then in conf/machine/zaurus/collie-special.conf it could just set
IMAGE_FSTYPES = "squashfs"

directly without needing any kind of override. That said I don't entirely
understand why meta-zaurus doesn't already have a collie.conf (assuming
MACHINE=collie) where it could be doing this...

Rewriting the BSP would work but it's sidestepping the issue and
doesn't really scale well. There will always be BSP layers which need
to support a collection of machines and the obvious way to do that is
to use default values and machine specific overrides within the BSP
layer. Making a rule that BSP layers can't use overrides for certain
variables will cause confusion and frustration.

I think the solution should be either that oe-core doesn't rely on
modifying variables set by the BSP (ie the initramfs images should be
able to signal to the image.bbclass that they want to use
INITRAMFS_FSTYPES via some other method than modifying IMAGE_FSTYPES)
or that we agree that oe-core can forcefully override the BSP
variables in certain well defined cases (and then the _forcevariable
override appears to be a good solution).

On Wed, Jun 16, 2021 at 11:05:06AM -0700, Andre McCurdy wrote:

I'm not sure I understand this. Why is using the _forcevariable
override somehow worse than inventing a new assignment operator to do
the same thing?

Because _forcevariable isn't positional. It means "no matter what
else appears in the metadata, either before or after this assignment,
I want the variable to end up with this value". I don't think that's
ever an appropriate assertion for something in oe-core to be making.

Obviously if you have two _forcevariable overrides on the same
variable then one of them is going to lose so to that extent it's
making an impossible promise. But other than that, it just doesn't
play very well with others, not least because its effects can't
be undone. Which is more or less the original problem that we
started with: all we've done is take it up a notch, and this is
exactly the kind of arms race that I mentioned in my first email
on this subject.

Richard's !=! or whatever we want to call it at least has the merit
that it overrides what has gone before but can itself be overridden
by what follows.

p.

On Wed, Jun 16, 2021 at 11:26 AM Phil Blundell <pb@...> wrote:

On Wed, Jun 16, 2021 at 11:05:06AM -0700, Andre McCurdy wrote:

I'm not sure I understand this. Why is using the _forcevariable
override somehow worse than inventing a new assignment operator to do
the same thing?

Because _forcevariable isn't positional. It means "no matter what
else appears in the metadata, either before or after this assignment,
I want the variable to end up with this value". I don't think that's
ever an appropriate assertion for something in oe-core to be making.

Obviously if you have two _forcevariable overrides on the same
variable then one of them is going to lose so to that extent it's
making an impossible promise. But other than that, it just doesn't
play very well with others, not least because its effects can't
be undone. Which is more or less the original problem that we
started with: all we've done is take it up a notch, and this is
exactly the kind of arms race that I mentioned in my first email
on this subject.

Richard's !=! or whatever we want to call it at least has the merit
that it overrides what has gone before but can itself be overridden
by what follows.

I don't think users generally have a good grasp of the concept of
"what has gone before". They expect = to override ?= and _myoverride
to override = and if the ordering of those assignments can change
their effect then it's a bug rather than a feature.

I'm sure the advanced users reading this thread will be able to dream
up situations where an assignment operator which forcefully overrides
all overrides but can then be overridden again with another assignment
can be used in some creative hack somewhere... but it's not making OE
any better for the average user.

On Wed, Jun 16, 2021 at 2:22 AM Phil Blundell via
lists.openembedded.org <pb=pbcl.net@...> wrote:

On Tue, Jun 15, 2021 at 10:59:40PM +0100, Richard Purdie wrote:

It certainly could be a solution to the problem but I worry that having
variables that users should touch and not touch and variables shadowing
other variables is going to get complex rather quickly, even when you just
consider how you document or "enforce" it. Do we want to encourage what
could become many more intermiediate variables? It does sound like we're
effectively starting to write class "functions" with a list of variables
they operate on which may or may not be a good thing depending on how
common it became.

Yes, that's a fair point.  As you say, I kind of have the sense that our
classes are drifting towards being more like classes in a "real" programming
language, complete with an actual API, and possibly we ought to start
thinking about them more in those terms.  As in, "if you include image.bbclass,
this is the set of variables it takes as input, and you need to give them
appropriate values".

From descriptions from others it sounds like people have creatively worked
around this kind of issue with stronger overrides which I'd not considered
and would also work, it is just a little ugly in a different way. I'd consider
use of the forcevariable override in oe-core a metadata failing (much like
I view _remove similarly in core).

I agree.  Although it's true that _forcevariable probably would fix this
particular issue I think there's a fair chance that it will just end up creating
the opposite problem at some point in the future.  I don't honestly think there
is any situation where using _forcevariable in oe-core itself would be the
right thing to do.

I'm not sure I understand this. Why is using the _forcevariable
override somehow worse than inventing a new assignment operator to do
the same thing?

If we want to make a rule that nothing in oe-core should ever
forcefully override a setting provided by a BSP then that's OK... but
then isn't that an argument that a new assignment operator to
forcefully assign to variables shouldn't be used in oe-core?

I suppose the other obvious tactical answer to the problem at hand is for

meta-zaurus to stop using an override at all for what it's doing.  Although
possibly not quite so elegant, there's no reason that it couldn't do:

IMAGE_FSTYPES = "ext3"
# nb, include not require...
include conf/machine/zaurus/${MACHINE}-special.conf

and then in conf/machine/zaurus/collie-special.conf it could just set
IMAGE_FSTYPES = "squashfs"

directly without needing any kind of override.  That said I don't entirely
understand why meta-zaurus doesn't already have a collie.conf (assuming
MACHINE=collie) where it could be doing this...

Rewriting the BSP would work but it's sidestepping the issue and
doesn't really scale well. There will always be BSP layers which need
to support a collection of machines and the obvious way to do that is
to use default values and machine specific overrides within the BSP
layer. Making a rule that BSP layers can't use overrides for certain
variables will cause confusion and frustration.

I think the solution should be either that oe-core doesn't rely on
modifying variables set by the BSP (ie the initramfs images should be
able to signal to the image.bbclass that they want to use
INITRAMFS_FSTYPES via some other method than modifying IMAGE_FSTYPES)
or that we agree that oe-core can forcefully override the BSP
variables in certain well defined cases (and then the _forcevariable
override appears to be a good solution).

I think the initramfs issue is just an example. For this particular issue, we can come up with several different ways to solve it.

The point is, this example demonstrates a general problem, that is, we currently lack the ability to let the recipe make the final call on variable setting. I've stated this in detail in another email. Instead of repeating the points there, I'd like to provide another way to look at this problem.

We all know that we have feature_check.bbclass, which allow us to do DISTRO_FEATURES checking, etc. Why is this bbclass working well? Because DISTRO_FEATURES must be set in conf files. Now we have a mechanism for recipes to say that if some distro feature is not satisfied, I don't build. What if a recipe wants to say if some variable value is not satisfied, I don't build? We can't do this is some var_check.bbclass, because we can set var in recipes and we can set it in many different ways. This could only be achieved at bitbake level. The '!=!' is a good way to do this.

The above reason also put a strong restriction on the new '!=!' operator, that is, it MUST be limited to be used in recipes only.

P.S.
We have to be clear that _forcevariable is also only an override, it works because it's the last one in OVERRIDES.
But _forcevariable does not beat _append, it also does not beat _remove. So even if we use it in recipes in oe-core, which we should not, it still does not solve the general problem.

Best Regards,
Chen Qi