I'm growing increasingly concerned about default value assignments in
OE. The basic problem is people don't understand the way default values
work and the mechanisms we do have don't let people do all the things
they want to do.

I'll pick on the example found by Ross/Jon in meta-arm. If you set:

BB_HASHBASE_WHITELIST += "X"

in layer.conf, the BB_HASHBASE_WHITELIST ?= "Y" from bitbake.conf is
lost as by then the variable already has a value. Another user reported
the same issue today in irc.

Another example would be whether recipes should set PACKAGECONFIG with
"=", "?=" or "??=". All work well for some scenarios and not for
others, often depending on whether the user wants to add to or remove
from the original value, or change it to something else entirely.

We then have a complete lack of standarisation of how bbclass files set
default values.

I don't have a fully thought out solution to all of this at this point.
I think it is clear out existing syntax isn't really capable of
expressing everything we need though and confusion current reigns.

I was partly responsible for introducing "??=" as a syntax for
"default" value. The problem with it is that "+=" doesn't modify it, a
deliberate (and necessary) choice at the time which makes it pretty
ineffective.

If bitbake started tracking whether an assignment has been used on a
given value vs. just += and =+ it would be possible to have an operator
which could have a behaviour more in line with what users expect?

Whether that could/should be some new assignment operator, or whether
we could "rescue" ??= I'm also not sure. A new operator would certainly
be safer.

Does anyone else:

a) Agree there is a usability issue here?
b) Think we should try and do something?

If so, what should we do, what is the behaviour we really need/want?

Cheers,

Richard

On Fri, Jul 03, 2020 at 03:31:33PM +0100, Richard Purdie wrote:

I'm growing increasingly concerned about default value assignments in
OE. The basic problem is people don't understand the way default values
work and the mechanisms we do have don't let people do all the things
they want to do.

I'll pick on the example found by Ross/Jon in meta-arm. If you set:

BB_HASHBASE_WHITELIST += "X"

in layer.conf, the BB_HASHBASE_WHITELIST ?= "Y" from bitbake.conf is
lost as by then the variable already has a value. Another user reported
the same issue today in irc.

I think DISTRO_FEATURES is (or at least was) behaving the same and
that's probably what more people often tend to change in local.conf
than BB_HASHBASE_WHITELIST. I always recommend to just use _append in
such cases and verify with "bitbake -e" when in doubt.

Whether that could/should be some new assignment operator, or whether
we could "rescue" ??= I'm also not sure. A new operator would certainly
be safer.

Does anyone else:

a) Agree there is a usability issue here?

I agree that people struggle with it, but luckily "bitbake -e" is
invaluable help to see what went wrong (and hopefully soon they will
learn to use "bitbake -e" before asking on ML).

People are also complaining that there are already too many
operators (4 assignments, 3 for append/prepend, remove and overrides) adding
another to the mix IMHO doesn't improve the overall usability.

Cheers,

I totally agree that there is an usability issue. I myself have been confused for a long time about all these options and how they work together (or don't) and having now a hard time telling other ppl to avoid some corner cases.

For me the difficulty arises from this multi-stage processing.
On the one hand we have "=", "+=", "=+", ".=", "=." which are solely working according to the flow of instructions - so it is directly depending on the order of files processed.

And then we have the 2nd and 3rd stage operator "_append", "_prepend" and "_remove", which only act on the outcome of the first stage (which is even now hard to predict in some cases).

And finally we have "unset" which is far more hard to understand as it's resets everything based on the instruction flow but also partly the 2nd/3rd stage modifier (IIRC)

If I could start from scratch I would only go with
- weak default " ?= "
- hard set " = "
- append and prepend operations " += ", " =+ "
and base them all on solely on the instruction flow, which would also would remove the part of another (hard to grasp) dimension: layer priority, as then only the order of the layer in BBLAYER would determine the final flow.

But as we can't, I would say

- " ??= " for defaults
- minimize the usage of 1st stage operator "+=", "=+", ".=", "=." and replace them by "_append" and "_prepend" (which would eliminate the need for .= and =. btw)

I know this might be controversial, but I've seen multiple examples of the scenario described below, where a weak default was silently overridden, which is hard to debug (and as Martin Jansa already wrote the only way to really check this is using bitbake -e).
Having a 5 or 10 level layer cake makes it even harder to check all the cases.

And finally if the core layer have adapted to this pattern
- retire the operators "+=", "=+", ".=", "=."
- maybe also " ?= " should be retired, as it should be covered by " ??= " if the above is really given

If there shouldn't be any changes one could also add some sanity check into the parser to at least warn ppl that there is something happening that they might not be aware of.

But certainly I don't see the need to introduce another operator

On Fri, 2020-07-03 at 14:08 -0700, Christopher Larson wrote:

I definitely think there’s a usability issue here. It’d be nice if
+=/=+/.=/=. were recorded operations against the current or default
value. Now that _append/_prepend/_remove are applied at getVar time,
not at the end of the parse, I could see the other operations working
the same way, just the non _ operators would be reset in certain
contexts, ex unset or assignment to a non-default value. I think this
would be the ideal way forward. If done the right way, I don’t think
this would necessarily have to break current semantics regarding
assignments to the current value undoing the previous +=/=+/.=/=.

I agree, having +=/=+ and friends operating against the default value
would in general be the behaviour most people want/expect.

The change in behaviour would be that "=" would no longer wipe out
previous +=/=+ ops. Its probably worth an experiment to see how much
that would break. I think if we could make that work, most ?= could
become ??= and ?= could be phased out as its += not working on ??= that
is the current pain point as I understand it.

It’s not enough to keep ??= and deprecate ?= due to their differing
behavior. The *last* ??= wins, so if a bbclass uses ??= to define a
default value, it’ll use that in preference to a defined ??= default
from the configuration metadata, which wouldn’t be what you’d want.
So either the configuration metadata has to never use ??=, a matter
of policy, or a bbclass has to use ?=, or define its fallback default
in its code rather than in a variable assignment. This is why I
always use ?= for bbclass defaults, they shouldn’t override default
configuration, even if it wasn’t user configuration.

I did try and standarise something for PACKAGECONFIG in recipes but
sadly with the current status quo, you can't keep everyone happy (those
that want to set their own value and those that want to define a set of
changes).

That is, a distro may want to define a default that’s used in
preference to the bbclass default, but not used in preference to the
value defined in local.conf. In that situation I could see the distro
using ??= and the bbclass using ?=. Of course, there are
alternatives. If we expect the user to either always use overrides in
local.conf, or always use explicitly provided hook variables, not the
ones actually used by the implementation, then it’d be a non-issue.

I'd say the "base" definition uses ??= and then all other reassignments
use "=", then ordering wins. The reason people don't do that today is
that +=/=+ and friends are lost.

There are obviously *many* use cases to consider here given how
widely the project is used and the differing operation orders amongst
the numerous variables we have. Any replacement would have to be
validated to not change any variable in any recipe from the current
behavior.

That can at least be validated more easily now! :)

Yes, I do agree, it needs a lot of careful thought and then testing. I
do think its something we should think about though.

Cheers,

Richard

On Fri, 2020-07-03 at 23:23 +0200, Alexander Kanavin wrote:

Maybe we could ask, what are the typical intentions behind those
additions and removals and assignments? E.g. for DISTRO_FEATURES I
can think of these:

1. Features A B C can be present in DISTRO_FEATURES if no other wish
is expressed
2. Features D E F must be present in DISTRO_FEATURES
3. Features G H I must not be present in DISTRO_FEATURES.

It would help if the following is guaranteed to happen (and currently
it does not):

- type 2 list never affects type 1 list (or vice versa), Several
lists of type 1 or type 2 produce a union of all items.

- if a type 3 list conflicts with a type 2 list, then bitbake stops
and prints an error.

Specific syntax is secondary to intentions that we want to make easy
to express.

Yes, I think we're talking about the same thing, its just a different
way of thinking about it. I'm adding the challenge of trying to figure
out what changes to existing behaviour we could consider to get us
where we need to be (or as close as is realistic) or if we have to
change some operator(s).

Cheers,

Richard

On Fri, 2020-07-03 at 20:44 +0200, Konrad Weihmann wrote:

I totally agree that there is an usability issue. I myself have been
confused for a long time about all these options and how they work
together (or don't) and having now a hard time telling other ppl to
avoid some corner cases.

For me the difficulty arises from this multi-stage processing.
On the one hand we have "=", "+=", "=+", ".=", "=." which are solely
working according to the flow of instructions - so it is directly
depending on the order of files processed.

And then we have the 2nd and 3rd stage operator "_append",
"_prepend" and "_remove", which only act on the outcome of the first
stage (which is even now hard to predict in some cases).

And finally we have "unset" which is far more hard to understand as
it's resets everything based on the instruction flow but also partly
the 2nd/3rd stage modifier (IIRC)

If I could start from scratch I would only go with
- weak default " ?= "

Presumable this would behave like ??= though (?= as implemented is
fairly flawed but so is ??= today, sadly).

- hard set " = "
- append and prepend operations " += ", " =+ "
and base them all on solely on the instruction flow, which would
also
would remove the part of another (hard to grasp) dimension: layer
priority, as then only the order of the layer in BBLAYER would
determine the final flow.

I have to admit if I was implementing layers again, I'd want to ditch a
load of the older "collections" bits we brought in for compatibility.

But as we can't, I would say

- " ??= " for defaults
- minimize the usage of 1st stage operator "+=", "=+", ".=", "=."
and replace them by "_append" and "_prepend" (which would eliminate
the need for .= and =. btw)

I'd much rather general _append fell out of use in favour of a += and
=+ that did what the user expected and the append syntax only remained
in general use for overrides. I guess in my discussion with Chris we're
effectively talking about mapping += and =+ to append.

I know this might be controversial, but I've seen multiple examples
of the scenario described below, where a weak default was silently
overridden, which is hard to debug (and as Martin Jansa already
wrote the only way to really check this is using bitbake -e).
Having a 5 or 10 level layer cake makes it even harder to check all
the cases.

And finally if the core layer have adapted to this pattern
- retire the operators "+=", "=+", ".=", "=."
- maybe also " ?= " should be retired, as it should be covered by "
??= " if the above is really given

If there shouldn't be any changes one could also add some sanity
check
into the parser to at least warn ppl that there is something
happening
that they might not be aware of.

But certainly I don't see the need to introduce another operator

It depends how we want to consider compatibility and migration but yes,
more operators probably isn't going to make people happy.

Cheers,

Richard

On Fri, Jul 03, 2020 at 08:44:03PM +0200, Konrad Weihmann wrote:

- " ??= " for defaults
- minimize the usage of 1st stage operator "+=", "=+", ".=", "=." and
replace them by "_append" and "_prepend" (which would eliminate the need for
.= and =. btw)

I agree that there is a bit of a usability problem here, but I think this
proposal is going in the wrong direction. It's really the _append/_prepend
operators that have the uninituitive semantics and that I think should be
avoided. In particular, it is very difficult to "kill" an _append once
it's been defined.

How much of the problem is really a tools issue, and how much is just
something that can be fixed with documentation/convention, is an interesting
question.

Part of the problem, at least, seems to stem from different opinions about
what variable settings are "required" in any given situation and what the
division of responsibility between layers, recipes, users and DISTROs
ought to be. I think that probably wants disentangling before we look
too closely at the exact syntax and semantics of our assignments.

p.

On 7/3/20 4:31 PM, Richard Purdie wrote:

I'm growing increasingly concerned about default value assignments in
OE. The basic problem is people don't understand the way default values
work and the mechanisms we do have don't let people do all the things
they want to do.
I'll pick on the example found by Ross/Jon in meta-arm. If you set:
BB_HASHBASE_WHITELIST += "X"
in layer.conf, the BB_HASHBASE_WHITELIST ?= "Y" from bitbake.conf is
lost as by then the variable already has a value. Another user reported
the same issue today in irc.
Another example would be whether recipes should set PACKAGECONFIG with
"=", "?=" or "??=". All work well for some scenarios and not for
others, often depending on whether the user wants to add to or remove
from the original value, or change it to something else entirely.
We then have a complete lack of standarisation of how bbclass files set
default values.
I don't have a fully thought out solution to all of this at this point.
I think it is clear out existing syntax isn't really capable of
expressing everything we need though and confusion current reigns.
I was partly responsible for introducing "??=" as a syntax for
"default" value. The problem with it is that "+=" doesn't modify it, a
deliberate (and necessary) choice at the time which makes it pretty
ineffective.
If bitbake started tracking whether an assignment has been used on a
given value vs. just += and =+ it would be possible to have an operator
which could have a behaviour more in line with what users expect?
Whether that could/should be some new assignment operator, or whether
we could "rescue" ??= I'm also not sure. A new operator would certainly
be safer.
Does anyone else:
a) Agree there is a usability issue here?

The example you mention with BB_HASHBASE_WHITELIST, it sounds to me like the underlying problem is that the order of file inclusion is not apparent to people, and not that there is any confusion about the ?=/+= operators ?

Jacob

On Sat, Jul 04, 2020 at 03:18:43PM +0200, Jacob Kroon wrote:

On 7/3/20 4:31 PM, Richard Purdie wrote:

I'm growing increasingly concerned about default value assignments in
OE. The basic problem is people don't understand the way default values
work and the mechanisms we do have don't let people do all the things
they want to do.

I'll pick on the example found by Ross/Jon in meta-arm. If you set:

BB_HASHBASE_WHITELIST += "X"

in layer.conf, the BB_HASHBASE_WHITELIST ?= "Y" from bitbake.conf is
lost as by then the variable already has a value. Another user reported
the same issue today in irc.

Another example would be whether recipes should set PACKAGECONFIG with
"=", "?=" or "??=". All work well for some scenarios and not for
others, often depending on whether the user wants to add to or remove
from the original value, or change it to something else entirely.

We then have a complete lack of standarisation of how bbclass files set
default values.

I don't have a fully thought out solution to all of this at this point.
I think it is clear out existing syntax isn't really capable of
expressing everything we need though and confusion current reigns.

I was partly responsible for introducing "??=" as a syntax for
"default" value. The problem with it is that "+=" doesn't modify it, a
deliberate (and necessary) choice at the time which makes it pretty
ineffective.

If bitbake started tracking whether an assignment has been used on a
given value vs. just += and =+ it would be possible to have an operator
which could have a behaviour more in line with what users expect?

Whether that could/should be some new assignment operator, or whether
we could "rescue" ??= I'm also not sure. A new operator would certainly
be safer.

Does anyone else:

a) Agree there is a usability issue here?

The example you mention with BB_HASHBASE_WHITELIST, it sounds to me
like the underlying problem is that the order of file inclusion is
not apparent to people, and not that there is any confusion about
the ?=/+= operators ?

I'd like to add my 2 cents.

While the complexity of operators may be somewhat confusing to new users or
developers of the project, alongside with the order of file inclusion and
different stages of parsing and processing variables...

But I tend to agree with Martin and Jacob here, that it is rather the matter
of education, experience and examining the environment, than it is a call
to action for adding yet another mechanism for setting variables and their
defaults. One example of past attempts to fix or workaround user's issues
like this was introduction of BACKFILL processing, which is even less
intuitive and more confusing, IMHO...

So, I would be very cautious about adding yet another operator.

--
Denys

On Sat, Jul 04, 2020 at 03:18:11PM -0400, Denys Dmytriyenko wrote:

One example of past attempts to fix or workaround user's issues
like this was introduction of BACKFILL processing, which is even less
intuitive and more confusing, IMHO...

That mechanism was introduced to solve a different and rather specific
problem. It might be true that it isn't entirely intuitive, and there
might be a bit of a lack of clear documentation describing how it's meant
to be used from the DISTRO maintainer's point of view. Possibly that's the
common thread that runs through all of this discussion.

p.

On 04.07.20 01:05, Phil Blundell wrote:

On Fri, Jul 03, 2020 at 08:44:03PM +0200, Konrad Weihmann wrote:

- " ??= " for defaults
- minimize the usage of 1st stage operator "+=", "=+", ".=", "=." and
replace them by "_append" and "_prepend" (which would eliminate the need for
.= and =. btw)

I agree that there is a bit of a usability problem here, but I think this
proposal is going in the wrong direction. It's really the _append/_prepend
operators that have the uninituitive semantics and that I think should be
avoided. In particular, it is very difficult to "kill" an _append once
it's been defined.

Absolutely agree, I just see that it is harder to get rid of the append/prepend mechanism than the other.

How much of the problem is really a tools issue, and how much is just
something that can be fixed with documentation/convention, is an interesting
question.

And I think it's not a tool issue per se - maybe it can be fixed by proper documentation - I'm not sure as most of the corner cases are far from being intuitive - so step one for me is to make them visible to the user.

Part of the problem, at least, seems to stem from different opinions about
what variable settings are "required" in any given situation and what the
division of responsibility between layers, recipes, users and DISTROs
ought to be. I think that probably wants disentangling before we look
too closely at the exact syntax and semantics of our assignments.
p.

Yeah that pretty much sums up the use cases - the question (and I guess tricky part) is about "no other wish is expressed" - and here, I think, something should be done, as it's not very intuitive for the user.

On 7/4/20 3:18 PM, Jacob Kroon via lists.openembedded.org wrote:

On 7/3/20 4:31 PM, Richard Purdie wrote:

I'm growing increasingly concerned about default value assignments in
OE. The basic problem is people don't understand the way default values
work and the mechanisms we do have don't let people do all the things
they want to do.

I'll pick on the example found by Ross/Jon in meta-arm. If you set:

BB_HASHBASE_WHITELIST += "X"

in layer.conf, the BB_HASHBASE_WHITELIST ?= "Y" from bitbake.conf is
lost as by then the variable already has a value. Another user reported
the same issue today in irc.

Another example would be whether recipes should set PACKAGECONFIG with
"=", "?=" or "??=". All work well for some scenarios and not for
others, often depending on whether the user wants to add to or remove
from the original value, or change it to something else entirely.

We then have a complete lack of standarisation of how bbclass files set
default values.

I don't have a fully thought out solution to all of this at this point.
I think it is clear out existing syntax isn't really capable of
expressing everything we need though and confusion current reigns.

I was partly responsible for introducing "??=" as a syntax for
"default" value. The problem with it is that "+=" doesn't modify it, a
deliberate (and necessary) choice at the time which makes it pretty
ineffective.

If bitbake started tracking whether an assignment has been used on a
given value vs. just += and =+ it would be possible to have an operator
which could have a behaviour more in line with what users expect?

Whether that could/should be some new assignment operator, or whether
we could "rescue" ??= I'm also not sure. A new operator would certainly
be safer.

Does anyone else:

a) Agree there is a usability issue here?

The example you mention with BB_HASHBASE_WHITELIST, it sounds to me like the underlying problem is that the order of file inclusion is not apparent to people, and not that there is any confusion about the ?=/+= operators ?

Here is a crazy idea: merge oe-core/meta/conf/bitbake.conf into oe-core/meta/conf/layer.conf, skip parsing a bitbake.conf altogether, and tell people to put the oe-core layer first in BBLAYERS. Then people can use += in their custom layer.conf's to modify oe-core configuration with expected(?) results. At least to me, that would be a more intuitive order of parsing the configuration files. I'm probably missing something obvious.

Jacob

On 7/7/20 9:15 AM, Jacob Kroon wrote:

On 7/4/20 3:18 PM, Jacob Kroon via lists.openembedded.org wrote:

On 7/3/20 4:31 PM, Richard Purdie wrote:

I'm growing increasingly concerned about default value assignments in
OE. The basic problem is people don't understand the way default values
work and the mechanisms we do have don't let people do all the things
they want to do.

I'll pick on the example found by Ross/Jon in meta-arm. If you set:

BB_HASHBASE_WHITELIST += "X"

in layer.conf, the BB_HASHBASE_WHITELIST ?= "Y" from bitbake.conf is
lost as by then the variable already has a value. Another user reported
the same issue today in irc.

Another example would be whether recipes should set PACKAGECONFIG with
"=", "?=" or "??=". All work well for some scenarios and not for
others, often depending on whether the user wants to add to or remove
from the original value, or change it to something else entirely.

We then have a complete lack of standarisation of how bbclass files set
default values.

I don't have a fully thought out solution to all of this at this point.
I think it is clear out existing syntax isn't really capable of
expressing everything we need though and confusion current reigns.

I was partly responsible for introducing "??=" as a syntax for
"default" value. The problem with it is that "+=" doesn't modify it, a
deliberate (and necessary) choice at the time which makes it pretty
ineffective.

If bitbake started tracking whether an assignment has been used on a
given value vs. just += and =+ it would be possible to have an operator
which could have a behaviour more in line with what users expect?

Whether that could/should be some new assignment operator, or whether
we could "rescue" ??= I'm also not sure. A new operator would certainly
be safer.

Does anyone else:

a) Agree there is a usability issue here?

The example you mention with BB_HASHBASE_WHITELIST, it sounds to me like the underlying problem is that the order of file inclusion is not apparent to people, and not that there is any confusion about the ?=/+= operators ?

Here is a crazy idea: merge oe-core/meta/conf/bitbake.conf into oe-core/meta/conf/layer.conf, skip parsing a bitbake.conf altogether, and tell people to put the oe-core layer first in BBLAYERS. Then people can use += in their custom layer.conf's to modify oe-core configuration with expected(?) results. At least to me, that would be a more intuitive order of parsing the configuration files. I'm probably missing something obvious.

I realise now that this will break for instance the inclusion of distro conf's from additional layers.

Jacob