Should we change variable override formatting?
Richard Purdie
Breaking things down a bit, one thing I keep running into with our current
codebase and metadata is that overrides are not clear. In my previous email, the example was: do_configure_class-native A human can usually spot "class-native" is and "configure" or "configure_class-native" is not. Bitbake's parser struggles. It has huge internal lists including variables like x86_64 where it has to track whether "64" in an override. One way of fixing this is to be explicit about overrides and use a different separator. See an example patch below I made to the quilt recipe using ":" instead to see how it looks. Personally, I think this looks like an improvement. There are two challenges: a) The work of migration. Do we migrate piecemeal or with a flag day? I'm not sure piecemeal is even possible unfortunately. b) Some of the packaging code (at the very least) needs rewriting as it accesses both RDEPENDS_${PN} and puts ${PN} in OVERRIDES and accesses RDEPENDS. I'm not sure what else may be assuming this works. This change does buy us cleaner looking metadata and ultimately, a faster and cleaner internal bitbake that at least would use less memory. It could set the stage to allow the defval idea I mentioned in a previous mail to happen. It is a huge change and would need a lot of work to make it happen. Is it worth doing? Not sure. I'm putting it out there for discussion. It is also going to be tempting that "if we're breaking things, lets break lots". I want to be mindful that tempting as that may be, if users can't convert clearly, it will be worlds of pain. The benefit of this change is that at least in the general case, the transition should be clear. Taking steps rather than changing the world may therefore be a better idea... Cheers, Richard diff --git a/meta/recipes-devtools/quilt/quilt.inc b/meta/recipes-devtools/quilt/quilt.inc index d7ecda7aaa6..f85de384d26 100644 --- a/meta/recipes-devtools/quilt/quilt.inc +++ b/meta/recipes-devtools/quilt/quilt.inc @@ -14,36 +14,36 @@ SRC_URI = "${SAVANNAH_GNU_MIRROR}/quilt/quilt-${PV}.tar.gz \ file://0001-tests-Allow-different-output-from-mv.patch \ " -SRC_URI_append_class-target = " file://gnu_patch_test_fix_target.patch" +SRC_URI:append:class-target = " file://gnu_patch_test_fix_target.patch" SRC_URI[md5sum] = "6800c2404a2c0598ab2eff92a636ba70" SRC_URI[sha256sum] = "314b319a6feb13bf9d0f9ffa7ce6683b06919e734a41275087ea457cc9dc6e07" inherit autotools-brokensep ptest -INHIBIT_AUTOTOOLS_DEPS_class-native = "1" -PATCHTOOL_class-native = "patch" +INHIBIT_AUTOTOOLS_DEPS:class-native = "1" +PATCHTOOL:class-native = "patch" CLEANBROKEN = "1" EXTRA_OECONF = "--with-perl='${USRBINPATH}/env perl' --with-patch=patch" -EXTRA_OECONF_append_class-native = " --disable-nls" +EXTRA_OECONF:append:class-native = " --disable-nls" EXTRA_AUTORECONF += "--exclude=aclocal" CACHED_CONFIGUREVARS += "ac_cv_path_BASH=/bin/bash ac_cv_path_COLUMN=column" # Make sure we don't have "-w" in shebang lines: it breaks using # "/usr/bin/env perl" as parser -do_configure_prepend () { +do_configure:prepend () { find ${S} -name "*.in" -exec sed -i -e "1s,^#\!.*@PERL@ -w$,#\! @PERL@\nuse warnings;," {} \; } # Don't setup symlinks to host utilities, we don't need them -do_configure_append () { +do_configure:append () { sed -e 's,^COMPAT_SYMLINKS.*:=.*,COMPAT_SYMLINKS :=,' -i ${S}/Makefile } -do_configure_class-native () { +do_configure:class-native () { oe_runconf } @@ -54,7 +54,7 @@ do_install () { rm -rf ${D}/${datadir}/emacs } -do_install_append_class-native () { +do_install:append:class-native () { # Dummy quiltrc file for patch.bbclass install -d ${D}${sysconfdir}/ touch ${D}${sysconfdir}/quiltrc @@ -75,16 +75,16 @@ do_install_ptest() { PACKAGES += "guards guards-doc" -FILES_${PN} = "${sysconfdir} ${datadir}/quilt \ +FILES:${PN} = "${sysconfdir} ${datadir}/quilt \ ${bindir}/quilt ${libdir}/quilt" -FILES_guards = "${bindir}/guards" -FILES_${PN}-doc = "${mandir}/man1/quilt.1 ${docdir}/${BPN}" -FILES_guards-doc = "${mandir}/man1/guards.1" +FILES:guards = "${bindir}/guards" +FILES:${PN}-doc = "${mandir}/man1/quilt.1 ${docdir}/${BPN}" +FILES:guards-doc = "${mandir}/man1/guards.1" -RDEPENDS_${PN} = "bash patch diffstat bzip2 util-linux less" -RDEPENDS_${PN}_class-native = "diffstat-native patch-native bzip2-native" +RDEPENDS:${PN} = "bash patch diffstat bzip2 util-linux less" +RDEPENDS:${PN}:class-native = "diffstat-native patch-native bzip2-native" -RDEPENDS_${PN}-ptest = "make file sed gawk diffutils findutils ed perl \ +RDEPENDS:${PN}-ptest = "make file sed gawk diffutils findutils ed perl \ perl-module-filehandle perl-module-getopt-std \ perl-module-posix perl-module-file-temp \ perl-module-text-parsewords perl-module-overloading \
|
|
Further thoughts on potential syntax changes
Richard Purdie
I've been quiet but have been thinking a lot about the syntax discussions.
I believe we do need to simplify things, at least so the majority of recipes have simpler assignment syntax that is understandable to new users. There was a recent question about whether it was preferred to use += or _append and it is hard to answer. My worries about setting PACKAGECONFIG with ??=, ?= or = are also still an open problem too. One potential approach is we'd just map += and =+ to append/prepend operations behind the scenes and all would be good. The trouble is there is a difference in behaviour. _append and _prepend apply to *any* variable value including an override value whilst += and =+ only apply to the "default" value and not any override value. This behaviour is taken advantage of by the metadata. The first example you find in a build is quilt-native: do_configure_prepend () { A } do_configure_append () { B } do_configure_class-native () { C } which wants it's prepend/appends always applied but wants to override the default value in the native case. As mentioned on IRC, I've played with the idea of a couple of new override levels, "defval" and "weakval" where: A = "X" would become A_defval within the datastore and then += and =+ can become prepends and appends to the defval. Sadly if we do that, we need to be able to know that do_configure_class-native is an override rather than creating do_configure_class-native_defval and we don't have a way to know that for example, "configure" isn't an override to the variable "do" yet class-native is. Working backwards, I think the question is what we want this syntax to look like. How do we say "append this to all variants of a variable" vs. "append this only to the default value" or "append this only to the overriden value"? Can we offer a way to clear a certain subset of appends? We have an immensely powerful syntax, the question is whether we can find a way to present it which makes sense to more people... If we did have clear marking of overrides, that could go some way to establishing a defval override level, which imports our current syntax into the overrides model which could then open up possibilities like allowing changing or clearing specific overrides explicitly and may allow dropping of the different assignment variants (ignoring flags :/). Not sure quite where I'm going with this but wanted to document it a little and share... Cheers, Richard
|
|
Re: Inclusive Language - wiki page
Michael Opdenacker
Greetings,
On 7/7/21 5:22 PM, Mark Hatle wrote: In the variables table:Instead of "includelist" and "excludelist", what about just "includes" and "excludes"? This terminology is already used in rsync's manual (for example), and it seems lighter and clear enough. For example, we could replace "CECC_USER_CLASS_BL" by "ICECC_USER_CLASS_EXCLUDES" and "ICECC_USER_PACKAGE_BL" by "ICECC_USER_PACKAGE_EXCLUDES". Any thoughts? I mentioned this on the wiki page (https://wiki.yoctoproject.org/wiki/Inclusive_language), but don't hesitate to revert my edits. Cheers, Michael. -- Michael Opdenacker, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
|
|
Re: Inclusive Language - wiki page
Andrea Adami
On Wed, Jul 7, 2021 at 7:16 PM Richard Purdie
<richard.purdie@...> wrote: +1 I find the whole issue almost unbelievable but hey... A.A.
|
|
Re: Inclusive Language - wiki page
Richard Purdie
On Wed, 2021-07-07 at 10:22 -0500, Mark Hatle wrote:
I'd note that it does actually raise "SkipRecipe" on the recipes in question so SKIP may actually be more consistent and help with understanding in other code. Cheers, Richard
|
|
Re: Inclusive Language - wiki page
Mark Hatle
On 7/7/21 7:30 AM, Armin Kuster wrote:
Hello all,In the variables table: PNBLACKLIST, I'd prefer PNEXCLUDELIST, or PNBLOCKLIST. (I think exclude better captures what it does.) Confused, where it says current name is whitelist, suggested rename is 'excludelist', isn't that the exact opposite? SSTATE_DUPWHITELIST, this is really a list of things to ignore. So my suggestion is SSTATE_DUPIGNORELIST. CVE_CHECK_PN_WHITELIST, I think there is a typo in the table for the replacement (missing an L). But like the above, I think ignore is a better word then skip here. So my suggestion would be CVE_CHECK_PN_IGNORELIST. --Mark see
|
|
Inclusive Language - wiki page
Hello all,
The Yocto Project TSC has created a wiki page to start making notes of offending names and possible replacements. At some point, this wiki page should include a plan on moving forward. Our hope is this will help streamline the process when folks send patches to replace offending language. Everyone is welcome to find and make note of any offending names in the tables on the wiki to help scope out this effort. https://wiki.yoctoproject.org/wiki/Inclusive_language see https://lists.openembedded.org/g/openembedded-architecture/topic/inclusive_language_summary/75821819 for background. regards, Armin On behalf of The Yocto Project TSC.
|
|
Re: New assignment operator?
Andrea Adami
Hello,
after a few days I've discovered the fallouts of my chat with RP. Just to be precise the layer is meta-handheld and the patch uncovering the issue is: https://git.openembedded.org/ Note that it was a lightheaded try to fix ubi build failures. Yes, because of the inflaction in the years nowadays only core-image-minimal fits in the few PEBs. Being there are pending pull requests I could just revert that patch (plus revert in meta-initramfs as well) and live in peace I am really tempted to do so :) Thanks to all for the reviews FWIW my 2 cents are that we have enough operators. Cheers A.A. P.S. I think the easier fix is to have IMAGE_FSTYPES defined in collie.conf (MACHINE.conf) but I am sure there are more BSP around grouping the settings in an .inc so no, it's not just about this special case.
|
|
Re: New assignment operator?
Chen Qi
On 06/17/2021 02:05 AM, Andre McCurdy
wrote:
On Wed, Jun 16, 2021 at 2:22 AM Phil Blundell via lists.openembedded.org <pb=pbcl.net@...> wrote:On Tue, Jun 15, 2021 at 10:59:40PM +0100, Richard Purdie wrote:It certainly could be a solution to the problem but I worry that having variables that users should touch and not touch and variables shadowing other variables is going to get complex rather quickly, even when you just consider how you document or "enforce" it. Do we want to encourage what could become many more intermiediate variables? It does sound like we're effectively starting to write class "functions" with a list of variables they operate on which may or may not be a good thing depending on how common it became.Yes, that's a fair point. As you say, I kind of have the sense that our classes are drifting towards being more like classes in a "real" programming language, complete with an actual API, and possibly we ought to start thinking about them more in those terms. As in, "if you include image.bbclass, this is the set of variables it takes as input, and you need to give them appropriate values".From descriptions from others it sounds like people have creatively worked around this kind of issue with stronger overrides which I'd not considered and would also work, it is just a little ugly in a different way. I'd consider use of the forcevariable override in oe-core a metadata failing (much like I view _remove similarly in core).I agree. Although it's true that _forcevariable probably would fix this particular issue I think there's a fair chance that it will just end up creating the opposite problem at some point in the future. I don't honestly think there is any situation where using _forcevariable in oe-core itself would be the right thing to do.I'm not sure I understand this. Why is using the _forcevariable override somehow worse than inventing a new assignment operator to do the same thing? If we want to make a rule that nothing in oe-core should ever forcefully override a setting provided by a BSP then that's OK... but then isn't that an argument that a new assignment operator to forcefully assign to variables shouldn't be used in oe-core?I suppose the other obvious tactical answer to the problem at hand is formeta-zaurus to stop using an override at all for what it's doing. Although possibly not quite so elegant, there's no reason that it couldn't do: IMAGE_FSTYPES = "ext3" # nb, include not require... include conf/machine/zaurus/${MACHINE}-special.conf and then in conf/machine/zaurus/collie-special.conf it could just set IMAGE_FSTYPES = "squashfs" directly without needing any kind of override. That said I don't entirely understand why meta-zaurus doesn't already have a collie.conf (assuming MACHINE=collie) where it could be doing this...Rewriting the BSP would work but it's sidestepping the issue and doesn't really scale well. There will always be BSP layers which need to support a collection of machines and the obvious way to do that is to use default values and machine specific overrides within the BSP layer. Making a rule that BSP layers can't use overrides for certain variables will cause confusion and frustration. I think the solution should be either that oe-core doesn't rely on modifying variables set by the BSP (ie the initramfs images should be able to signal to the image.bbclass that they want to use INITRAMFS_FSTYPES via some other method than modifying IMAGE_FSTYPES) or that we agree that oe-core can forcefully override the BSP variables in certain well defined cases (and then the _forcevariable override appears to be a good solution). I think the initramfs issue is just an example. For this particular issue, we can come up with several different ways to solve it. The point is, this example demonstrates a general problem, that is, we currently lack the ability to let the recipe make the final call on variable setting. I've stated this in detail in another email. Instead of repeating the points there, I'd like to provide another way to look at this problem. We all know that we have feature_check.bbclass, which allow us to do DISTRO_FEATURES checking, etc. Why is this bbclass working well? Because DISTRO_FEATURES must be set in conf files. Now we have a mechanism for recipes to say that if some distro feature is not satisfied, I don't build. What if a recipe wants to say if some variable value is not satisfied, I don't build? We can't do this is some var_check.bbclass, because we can set var in recipes and we can set it in many different ways. This could only be achieved at bitbake level. The '!=!' is a good way to do this. The above reason also put a strong restriction on the new '!=!' operator, that is, it MUST be limited to be used in recipes only. P.S. We have to be clear that _forcevariable is also only an override, it works because it's the last one in OVERRIDES. But _forcevariable does not beat _append, it also does not beat _remove. So even if we use it in recipes in oe-core, which we should not, it still does not solve the general problem. Best Regards, Chen Qi
|
|
Re: New assignment operator?
Andre McCurdy
On Wed, Jun 16, 2021 at 11:26 AM Phil Blundell <pb@...> wrote:
I don't think users generally have a good grasp of the concept of "what has gone before". They expect = to override ?= and _myoverride to override = and if the ordering of those assignments can change their effect then it's a bug rather than a feature. I'm sure the advanced users reading this thread will be able to dream up situations where an assignment operator which forcefully overrides all overrides but can then be overridden again with another assignment can be used in some creative hack somewhere... but it's not making OE any better for the average user.
|
|
Re: New assignment operator?
Phil Blundell
On Wed, Jun 16, 2021 at 11:05:06AM -0700, Andre McCurdy wrote:
I'm not sure I understand this. Why is using the _forcevariableBecause _forcevariable isn't positional. It means "no matter what else appears in the metadata, either before or after this assignment, I want the variable to end up with this value". I don't think that's ever an appropriate assertion for something in oe-core to be making. Obviously if you have two _forcevariable overrides on the same variable then one of them is going to lose so to that extent it's making an impossible promise. But other than that, it just doesn't play very well with others, not least because its effects can't be undone. Which is more or less the original problem that we started with: all we've done is take it up a notch, and this is exactly the kind of arms race that I mentioned in my first email on this subject. Richard's !=! or whatever we want to call it at least has the merit that it overrides what has gone before but can itself be overridden by what follows. p.
|
|
Re: New assignment operator?
Andre McCurdy
On Wed, Jun 16, 2021 at 2:22 AM Phil Blundell via
lists.openembedded.org <pb=pbcl.net@...> wrote: I'm not sure I understand this. Why is using the _forcevariable override somehow worse than inventing a new assignment operator to do the same thing? If we want to make a rule that nothing in oe-core should ever forcefully override a setting provided by a BSP then that's OK... but then isn't that an argument that a new assignment operator to forcefully assign to variables shouldn't be used in oe-core? Rewriting the BSP would work but it's sidestepping the issue andI suppose the other obvious tactical answer to the problem at hand is formeta-zaurus to stop using an override at all for what it's doing. Although doesn't really scale well. There will always be BSP layers which need to support a collection of machines and the obvious way to do that is to use default values and machine specific overrides within the BSP layer. Making a rule that BSP layers can't use overrides for certain variables will cause confusion and frustration. I think the solution should be either that oe-core doesn't rely on modifying variables set by the BSP (ie the initramfs images should be able to signal to the image.bbclass that they want to use INITRAMFS_FSTYPES via some other method than modifying IMAGE_FSTYPES) or that we agree that oe-core can forcefully override the BSP variables in certain well defined cases (and then the _forcevariable override appears to be a good solution).
|
|
Re: New assignment operator?
Otavio Salvador
Em qua., 16 de jun. de 2021 às 02:08, Chen Qi <Qi.Chen@...> escreveu:
Anyone proposed '==='? (I'm thinking about the 'always equal' sign in math.)I really like the === proposal. -- Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750
|
|
Re: New assignment operator?
Phil Blundell
On Tue, Jun 15, 2021 at 10:59:40PM +0100, Richard Purdie wrote:
It certainly could be a solution to the problem but I worry that havingYes, that's a fair point. As you say, I kind of have the sense that our classes are drifting towards being more like classes in a "real" programming language, complete with an actual API, and possibly we ought to start thinking about them more in those terms. As in, "if you include image.bbclass, this is the set of variables it takes as input, and you need to give them appropriate values". From descriptions from others it sounds like people have creatively workedI agree. Although it's true that _forcevariable probably would fix this particular issue I think there's a fair chance that it will just end up creating the opposite problem at some point in the future. I don't honestly think there is any situation where using _forcevariable in oe-core itself would be the right thing to do. I suppose the other obvious tactical answer to the problem at hand is for meta-zaurus to stop using an override at all for what it's doing. Although possibly not quite so elegant, there's no reason that it couldn't do: IMAGE_FSTYPES = "ext3" # nb, include not require... include conf/machine/zaurus/${MACHINE}-special.conf and then in conf/machine/zaurus/collie-special.conf it could just set IMAGE_FSTYPES = "squashfs" directly without needing any kind of override. That said I don't entirely understand why meta-zaurus doesn't already have a collie.conf (assuming MACHINE=collie) where it could be doing this... p.
|
|
Re: New assignment operator?
Chen Qi
On 06/15/2021 06:29 PM, Richard Purdie
wrote:
We have an interesting problem with initramfs images. They do something like: IMAGE_FSTYPES = "${INITRAMFS_FSTYPES}" which looks innocent enough until a BSP does: IMAGE_FSTYPES_collie = "xxx" where collie is MACHINE, hence an override which causes confusion as the value of INITRAMFS_FSTYPES is never used for that machine. This is a long standing issue where in standard metadata, there is no way to undo an override (or append/prepend/remove). I tried to be clever and did: python () { d.setVar("IMAGE_FSTYPES", d.getVar("INITRAMFS_FSTYPES")) } and if you do that before the image class inherit (since anon python is executed in the order it is defined), it nearly works. It doesn't work with live images as IMAGE_FSTYPES is expanded for class inherits before the anonymous python executes. I also wondered about: IMAGE_FSTYPES := "${INITRAMFS_FSTYPES}" but that doesn't clear overrides either. In fact I can't actually see a way to do it other than some horrible thing like: def myfunc(d): d.setVar("IMAGE_FSTYPES", d.getVar("INITRAMFS_FSTYPES")) DUMMYVAR := "${@myfunc(d)}" Yes, we could do that but I don't think we should. I wondered about making := clear existing overrides/appends/prepends/removes but then I wondered if we should have a new assignment operator which would do that (and also not force the value to be expanded). I mentioned this on irc and there were a few suggestions: IMAGE_FSTYPES =!! "${INITRAMFS_FSTYPES}" IMAGE_FSTYPES !=! "${INITRAMFS_FSTYPES}" IMAGE_FSTYPES *:= "${INITRAMFS_FSTYPES}" IMAGE_FSTYPES :=* "${INITRAMFS_FSTYPES}" The questions are a) is another assignment operator a good idea? I think so. 1) In the view of compatibility, it's acceptable. Before the change, we have: _remove > _append > OVERRIDES > normal assignments After the change, we have: !=! > _remove > _append > OVERRIDES > normal assignments So it does not break anything 2) In the view of scope, it's necessary. Recipe has smaller effective scope than conf file. Good design should be: it has higher priority than conf file. The general principle is: smaller scope, higher priority. For example, $HOME/.someconf beats /etc/someconf. The current problem with bitbake/yocto/oe is that recipes cannot have the final call. They have smaller scope than conf files, yet are of the same priority regarding variable settings. This problem can only be solved at the bitbake level. So adding a new assignment operator to let the recipe be able to make the final call is necessary. b) if so, which form? Implementation shouldn't be hard in bitbake, the APIs are already there to do this kind of operation. I am kind of preferring !=!. Any form is OK. '!=!' seems a very natural choice for vi users. However, for users like me, it looks like 'never equals to' at a first glance. Anyone proposed '==='? (I'm thinking about the 'always equal' sign in math.) f(x) === C means no matter what x is, f always equals to C. VAR(history) === C means no matter what the history is, the VAR always equals to C. To clarify, I used 'VAR(history)' above to show that in bitbake a VAR's final value depends on its setting history. Best Regards, Chen Qi Cheers, Richard
|
|
Re: New assignment operator?
Richard Purdie
On Tue, 2021-06-15 at 17:10 +0200, Phil Blundell wrote:
On Tue, Jun 15, 2021 at 02:29:16PM +0100, Richard Purdie wrote:That is a good summary, yes.For that reason I would like to change the initramfs recipe somehow toThanks for the background, I understand the issue a bit better now. So, the simplest way of fixing the immediate problem seems to be to separateIt certainly could be a solution to the problem but I worry that having variables that users should touch and not touch and variables shadowing other variables is going to get complex rather quickly, even when you just consider how you document or "enforce" it. Do we want to encourage what could become many more intermiediate variables? It does sound like we're effectively starting to write class "functions" with a list of variables they operate on which may or may not be a good thing depending on how common it became. From descriptions from others it sounds like people have creatively worked around this kind of issue with stronger overrides which I'd not considered and would also work, it is just a little ugly in a different way. I'd consider use of the forcevariable override in oe-core a metadata failing (much like I view _remove similarly in core). Which does bring us back to whether adding something allowing the initramfs recipes to do the right thing would be a better solution? An initramfs class to abstract it whatever we do may also be worth considering but is tangential. Cheers, Richard
|
|
Re: New assignment operator?
Andre McCurdy
On Tue, Jun 15, 2021 at 8:48 AM Mark Hatle
<mark.hatle@...> wrote: On 6/15/21 10:10 AM, Phil Blundell via lists.openembedded.org wrote:I skimmed some of the thread so may have missed a detail... but if theOn Tue, Jun 15, 2021 at 02:29:16PM +0100, Richard Purdie wrote:There is more then once I thought it would be good to add a flag for some sortFor that reason I would like to change the initramfs recipe somehow toThanks for the background, I understand the issue a bit better now. initramfs recipe wants to forcefully over-ride a value of IMAGE_FSTYPES set by a BSP, then isn't the existing _forcevariable over-ride the way to do that?
|
|
Re: New assignment operator?
Mark Hatle
On 6/15/21 10:10 AM, Phil Blundell via lists.openembedded.org wrote:
On Tue, Jun 15, 2021 at 02:29:16PM +0100, Richard Purdie wrote:There is more then once I thought it would be good to add a flag for some sortFor that reason I would like to change the initramfs recipe somehow toThanks for the background, I understand the issue a bit better now. of variable type. Specific for this reason. "This variable is a MACHINE variable, this is a DISTRO variable, this is a recipe variable, this is a class variable, etc." Of course identifying what we're doing and warning may not be so easy, but it could help. Along with this identify where an override is being used, if it's a MACHINE override it probably shouldn't be in a DISTRO file for instance. The case where I was curious if we could do something like this very different. I saw MACHINE files setting what are effectively distribution features, and distro.conf files having machine overrides in them. All tend to lead to problems like this in the long term, variables that get set and can't easily be "unset". --Mark p.
|
|
Re: New assignment operator?
Phil Blundell
On Tue, Jun 15, 2021 at 02:29:16PM +0100, Richard Purdie wrote:
For that reason I would like to change the initramfs recipe somehow toThanks for the background, I understand the issue a bit better now. It seems fairly apparent that what's wrong here isn't fundamentally an issue with OVERRIDES, it's more that the semantic layering (in the generic sense of the word, not meaning OE layers) has just gone wrong with IMAGE_FSTYPES. So you have this one variable that, on one hand, is a user-tweakable knob that they can set to say "this is the type of filesystem image I want to generate for my particular machine", and on the other hand is part of this shadow interface contract between the initramfs code and the image-construction backend. meta-zaurus is trying to change the first one and accidentally clobbering the second one and I agree it can't really do much better with the current infrastructure. So, the simplest way of fixing the immediate problem seems to be to separate the two things out: make image.bbclass look at some new variable (IMAGE_GENERATE_FSTYPES or something), teach the initramfs code to set that variable rather than IMAGE_FSTYPES, and provide a soft default of IMAGE_GENERATE_FSTYPES ??= ${IMAGE_FSTYPES}. In the more general case it seems like any given variable ought to be classifiable as something that's either settable by the user configuration (including the BSP in that) or that is set by some other class in oe-core, but I think we ought to try to avoid having variables that fall into both groups. p.
|
|
Re: New assignment operator?
Tom Rini
On Tue, Jun 15, 2021 at 11:29:01AM +0100, Richard Purdie wrote:
We have an interesting problem with initramfs images.I'm going to intentionally avoid using "image" as gets overloaded I think. This is because an initramfs can be more than one of: - The only filesystem container we create for a MACHINE - A filesystem container that gets bundled inside of the kernel vmlinux. - A filesystem container that gets bundled inside of a container along side a kernel (zImage, Image, etc), one or more device trees and perhaps other blobs. - A filesystem container that gets placed inside of another filesystem via wic. And once you start to add in the why for those use cases, things can get racey too. They do something like:And this starts to be a lot more common of a problem than it might seem, once you start having multiple cases from the list I made above in use. When you aren't dealing with public layers, the fix is "easy" and it's to adjust the machine confs and image recipes to not do what they were doing. What I want to suggest is that the problem is with how we treat initramfs like it's another image, when it's not. What if for this problem we treat INITRAMFS_FSTYPES separate from IMAGE_FSTYPES? Make them "inherit initramfs" which in turn generates filesystem containers based only on what INITRAMFS_FSTYPES evaluates out to. And soft-assign that to cpio.gz by default. This is a long standing issue where in standard metadata, there is no wayThis is a separate problem. Is there some way to make remove be the last operation? One of: FOO_remove = "bar" FOO_forcevariable_remove = "baz" really ought to do what I mean and take bar or baz out of FOO. Even if some part of OVERRIDE prior to forcevariable puts it in. -- Tom
|
|