On 7/16/21 11:56 AM, Richard Purdie wrote:

On Fri, 2021-07-16 at 11:44 -0500, Mark Hatle wrote:

On 7/16/21 4:22 AM, Richard Purdie wrote:

On Thu, 2021-07-15 at 09:26 -0500, Mark Hatle wrote:

On 7/15/21 8:56 AM, Richard Purdie wrote:

Breaking things down a bit, one thing I keep running into with our current 
codebase and metadata is that overrides are not clear. In my previous email,
the example was:

do_configure_class-native

A human can usually spot "class-native" is and "configure" or 
"configure_class-native" is not. Bitbake's parser struggles. It has huge 
internal lists including variables like x86_64 where it has to track 
whether "64" in an override.

One way of fixing this is to be explicit about overrides and use a different 
separator. See an example patch below I made to the quilt recipe using ":" 
instead to see how it looks. Personally, I think this looks like an improvement.

There are two challenges:

a) The work of migration. Do we migrate piecemeal or with a flag day? I'm not
   sure piecemeal is even possible unfortunately.

b) Some of the packaging code (at the very least) needs rewriting as it accesses
   both RDEPENDS_${PN} and puts ${PN} in OVERRIDES and accesses RDEPENDS. I'm not
   sure what else may be assuming this works.

Ya, this is all over the packaging code. Should be trivial to move to treating
it as a variable (and not override). I can help with this if you want.

I've been pondering this a bit more and my preference is to move this to use 
overrides more explicitly.

The part of the code I worked on, the override was used as an optimization to avoid:

d.getVar("RDEPENDS_%s" % package) or d.getVar("RDEPENDS")

(and similar for all of the other variables that could be package specific.)

Overrides (at least to me) are system (or at least recipe) wide settings that
allow you to 'override' key things. The package items, based on that definitely
really aren't overrides -- they're package specific variables. (This is also
why I suggested RDEPENDS[package] might actually be a better syntax.)

Overrides are one of the key things which make OE function as an architecture and 
how a lot of the "magic" in our metadata happens. There is a hierarchy of overrides,
some are system wide some are local to specific functional areas but using them for
packaging is a natural fit.

As such, I'm very strongly in favour of explicitly exposing that rather than hiding
it.

FWIW the image class code also does something similar for IMAGE_CMD and image types.

I'm all for making override usage more clear to people. But in this ONE case, I
don't think these are really overrides -- and worse they could cause confusion
because they are ONLY overrides when packaging. So if you use the override
chunk in something that doesn't affect a package specific setting it won't do
anything. It also won't do anything on the functions (I think), as those are
run before the override is loaded.

Specific code I'm talking about BTW is:

http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/package_rpm.bbclass

starting at line 287. (srcname is PN-src BTW)

then the loop starting at 328. Iterates over PACKAGES and sets the override one
at a time to the package name to get (conffiles, dirnames, summary/description,
pkgv, pkgr, pkge, license, section, description, rdepends, rrecommends,
rsuggests, rprovides, rreplaces, rconflicts, and I suspect a few others...)

Note, I could make the above changes pretty quickly and just submit this as an
implementation change in a few hours. I'd be happy to do that and look for
other 'in-place' usages of d.setVar("OVERRIDE"... to see if there are other
things like this.

As above, I'm very much of the opposite view, I think we want to promote this,
not remove it. Yes, you can emulate overrides with other forms of variable access
but that really isn't how the system was designed to be used.

The view I've also had and used to explain to others. There are two scopes of
variables in OE. global and recipe specific. Adding an override anywhere else,
really adds additional (hidden) scopes to the system.

I guess this metal model of scopes is influenced by 'bitbake -e'. I can run it
against the global system, or a specific recipe. But there is no way for me to
run it on a 'package' scope.

Additional mental issue I have with PACKAGE being a scope, is the limited nature
of the scope. The majority of variables in the system are not modified or used
when in this scope, only a VERY limited amount.. (Now this limited amount can
reference variables in the recipe scope.. But would you ever do:

PACKAGES = "${PN} ${PN}-dev ${PN}-src"

FILES_${PN} = "/foo/${MYFILE}"

MYFILE = "bar"
MYFILE_${PN} = "foo"

Current implementation, /foo/foo would be selected, only during package
processing -- but if you use bitbake -e recipe you can /foo/bar...


So back to what I offered before, from a quicker survey of poky the following
are the only non-global/recipe specific scopes that I was able to find.

'PACKAGE' specific overrides:
package_*.bbclass
update-rc.d.bbclass
insane.bbclass
buildhistory.bbclass

The variables allowed to work in the current form of VAR_pkg is very specific
and tailored.


uboot/syslinux/grub have a concept of 'labels', they each implement it slightly
differently but it's the same idea:
uboot-extlinux-config.bbclass
syslinux.bbclass
grub-efi-cfg.bbclass

Again, the list appears to be very specific and minimal from what I could tell.
Unlike the package side, I don't have a lot of experience here. (Variables are
specific to each of those three classes though, but still a limited set.)


'debugfs' usage for images. This appears to be how the debugfs stuff is
implemented:
image.bbclass

I'm not completely sure the range of variables used here. But again it looks
somewhat limited in scope... but this scope could quickly expand if you end up
nesting things variables, but I'm unsure that it would work based on the manual
processing of vardeps that I see.

Cheers,

Richard