On 9/9/21 9:09 AM, Richard Purdie wrote:

Hi All,

We have a decision facing us with 3.5. There are a number of invasive issues
looming on the horizon and I'm not sure exactly what the best thing to do with
them is.

a) Inclusive language

A lot of variables potentially need renaming with varying options for backwards
compatibility. Do we add compatibility for all cases.

By backward compatibility do you mean allowing a layer to use one of the
offending variables? if so, what is the point?

How much do we help users
with migration?

You mean like in a script?

Is there wide support for the changes?

You mean this LF project? https://inclusivenaming.org ( Think its overly
complication things)

or OE/YP folks getting involved?

Do we change the master branch to something else? I personally have a preference
for "devel" over "main" regardless of what others are doing as it matches what
it is in our case. Changing that alone is days of work for me trying to get all
our automation to deal with it.

If the master branch gets renamed, what about yocto-buildhistory,
yocto-buildstats and yocto-testresults? Would master branches or tags
get renamed as well? Is this part of the day's worth of work you mentioned?

Would the Yocto Project enforce a renaming in all the repos they host?

What about layer-index?

- armin

I am frustrated that after lots of people saying this was important and the TSC
putting a process/plan in place, the work hasn't been done. This does imply that
if "we" do it, I'll get to do a lot of work on it which I have mixed feelings
about. Not doing it also looks bad for the project though.

b) layer.conf changes

Part of the reason the LAYER* variables didn't change in the overrides changes
is that not only are they not overrides in any way they're used but also that we
could probably do with dropping a lot of the legacy ways layers work and adopt
something easier for people to understand (e.g. the term collections is old and
we have too many ways of handling priorities).

I think a deeper cleanup of layer configuration is possible and desirable but it
would mean dropping "features" some people are using. I don't know how much
support there would be for that. Would people accept a standard layer priority
for example?

c) data store operational changes

One of the things the new overrides changes lets us do is know definitively what
is a variable and what is an override. Currently when you call d.keys(), you'd
get A and A:qemux86. We could decide that only A should be shown and this could
speed up certain operations due to the simplified key list.

d) potential syntax removal

Another option from the overides changes would be to disallow certain operator
combinations. One often commented on example would be A:append:b += "c", there
are others depending on how far we wanted to go with that. These should be
possible to at least show errors to the user.

e) potential data store internals rewrite

Yet another possible move from the overrides syntax change is that it would
allow us to remove the "overridedata" element of the data store into the
specific variables. I have a proof of concept of this but it turns out to be
really ugly code, unless we accept some user visible changes to the behaviour of
the datastore (and/or remove some forms of syntax). At present I'm experimenting
without making user visible changes but we do have potential decisions to make
here too.

f) f-strings in python code

We now have a compatible python version but I'm reluctant to start changing
large chunks of the code base with the risks of breaking in transitions. The
more of this we take in, the bigger the risks of something being backported to
older releases which causes issues.

FWIW, I'm seeing signs other layers are going to start using this regardless of
what the architecture list or TSCs say. I really don't want to get into a
position where we have multiple "standards" for things as the current situation
is bad enough. I'm therefore worried about this one a lot. Part of me wonders if
we should just start allowing them and watch things unravel for older releases.

g) bitbake internal changes

Other things in bitbake are showing their age. The client/server model it uses
really needs reworking to use async apis. tinfoil needs to adapt to handle
parallel parsing if it is to scale in the future. The logging in bitbake needs a
total overhaul and previous work completing, with potential user visible
changes.

h) merge of some currently optional functionality into the core

Merging bits of reproducible builds and maybe some license/sbom handling into
the core project workflows as not optional may be desirable to reduce the test
matrix and the chances of people not testing codepaths with changes.

i) dropping/cleaning up some core code

Chunks of the license class code come to mind, there are probably other areas
which could do with cleanup.


There are probably more of these, these are just the handful of issues that came
to mind right now. I can't help think there were some others that were discussed
in the tech call on Tuesday.

I'm conscious we don't want to do what python did with the 2->3 transition.
Equally, I know there are a lot of things we could do with changing, not a lot
of people interested in doing the work and an LTS "looming" which we may or may
not want to get some of these changes into.

Thoughts on how much we should be considering for 3.5 and the changes in general
welcome...

Cheers,

Richard

Hi All,

We have a decision facing us with 3.5. There are a number of invasive issues
looming on the horizon and I'm not sure exactly what the best thing to do with
them is.

a) Inclusive language

A lot of variables potentially need renaming with varying options for backwards
compatibility. Do we add compatibility for all cases. How much do we help users
with migration? Is there wide support for the changes?

Do we change the master branch to something else? I personally have a preference
for "devel" over "main" regardless of what others are doing as it matches what
it is in our case. Changing that alone is days of work for me trying to get all
our automation to deal with it.

I am frustrated that after lots of people saying this was important and the TSC
putting a process/plan in place, the work hasn't been done. This does imply that
if "we" do it, I'll get to do a lot of work on it which I have mixed feelings
about. Not doing it also looks bad for the project though.

b) layer.conf changes

Part of the reason the LAYER* variables didn't change in the overrides changes
is that not only are they not overrides in any way they're used but also that we
could probably do with dropping a lot of the legacy ways layers work and adopt
something easier for people to understand (e.g. the term collections is old and
we have too many ways of handling priorities).

I think a deeper cleanup of layer configuration is possible and desirable but it
would mean dropping "features" some people are using. I don't know how much
support there would be for that. Would people accept a standard layer priority
for example?

c) data store operational changes

One of the things the new overrides changes lets us do is know definitively what
is a variable and what is an override. Currently when you call d.keys(), you'd
get A and A:qemux86. We could decide that only A should be shown and this could
speed up certain operations due to the simplified key list.

d) potential syntax removal

Another option from the overides changes would be to disallow certain operator
combinations. One often commented on example would be A:append:b += "c", there
are others depending on how far we wanted to go with that. These should be
possible to at least show errors to the user.

e) potential data store internals rewrite

Yet another possible move from the overrides syntax change is that it would
allow us to remove the "overridedata" element of the data store into the
specific variables. I have a proof of concept of this but it turns out to be
really ugly code, unless we accept some user visible changes to the behaviour of
the datastore (and/or remove some forms of syntax). At present I'm experimenting
without making user visible changes but we do have potential decisions to make
here too.

f) f-strings in python code

We now have a compatible python version but I'm reluctant to start changing
large chunks of the code base with the risks of breaking in transitions. The
more of this we take in, the bigger the risks of something being backported to
older releases which causes issues.

FWIW, I'm seeing signs other layers are going to start using this regardless of
what the architecture list or TSCs say. I really don't want to get into a
position where we have multiple "standards" for things as the current situation
is bad enough. I'm therefore worried about this one a lot. Part of me wonders if
we should just start allowing them and watch things unravel for older releases.

g) bitbake internal changes

Other things in bitbake are showing their age. The client/server model it uses
really needs reworking to use async apis. tinfoil needs to adapt to handle
parallel parsing if it is to scale in the future. The logging in bitbake needs a
total overhaul and previous work completing, with potential user visible
changes.

h) merge of some currently optional functionality into the core

Merging bits of reproducible builds and maybe some license/sbom handling into
the core project workflows as not optional may be desirable to reduce the test
matrix and the chances of people not testing codepaths with changes.

i) dropping/cleaning up some core code

Chunks of the license class code come to mind, there are probably other areas
which could do with cleanup.


There are probably more of these, these are just the handful of issues that came
to mind right now. I can't help think there were some others that were discussed
in the tech call on Tuesday.

I'm conscious we don't want to do what python did with the 2->3 transition.
Equally, I know there are a lot of things we could do with changing, not a lot
of people interested in doing the work and an LTS "looming" which we may or may
not want to get some of these changes into.

Thoughts on how much we should be considering for 3.5 and the changes in general
welcome...

Cheers,

Richard

Hi,

We will be having an OpenEmbedded Happy Hour on Auguest 25 for Asia/Pacific
timezones @ 2100/9pm UTC (5pm ET / 2pm PT):

https://www.openembedded.org/wiki/Calendar
https://www.timeanddate.com/worldclock/fixedtime.html?msg=OpenEmbedded+Happy+Hour+August+25&iso=20210825T21

--
Regards,
Denys Dmytriyenko <denis@...>
PGP: 0x420902729A92C964 - https://denix.org/0x420902729A92C964
Fingerprint: 25FC E4A5 8A72 2F69 1186 6D76 4209 0272 9A92 C964

On 31/07/21 23:09, Richard Purdie wrote:

On Sat, 2021-07-31 at 15:29 +0000, Peter Kjellerstedt wrote:

-----Original Message-----
From: openembedded-architecture@... <openembedded-
architecture@...> On Behalf Of Richard Purdie
Sent: den 30 juli 2021 15:40
To: openembedded-architecture <openembedded-
architecture@...>
Subject: Re: [Openembedded-architecture] Overrides conversion plan

On Wed, 2021-07-28 at 16:43 +0100, Richard Purdie via
lists.openembedded.org wrote:


I have continued to work on this and I now think we're as ready as we'll
ever
be with the core. I have:

* submitted a section for the migration guide documenting the conversion
process
* increased the minimum bitbake version for OE-Core
* bumped the local.conf version to require new versions of the config file
* added an error to bitbake if it sees "_append"|"_prepend"|"_remove" in
  variable names which would suggest an unconverted layer. If you use
those
  in function names in the datastore that was never a good idea and is no
  longer supported.
* merged compatibility changes back to bitbake 1.50, 1.48 and 1.46
* merged the conversion script to OE-Core
* merged submitted tweaks to the conversion script (thanks Martin!)
* made OE-Core honister only, no longer supporting hardknott
* updated converted layers to be honister compatible
* converted autobuilder-helper to use the new syntax
* ensured all of a-quick builds on the autobuilder
* submitted patches for meta-yocto, meta-gplv2, meta-mingw and bitbake

I plan to merge these various things on Monday (2nd August). After that
time, unconverted layers will no longer work with master.

Cheers,

Richard

[ I am on summer vacation so I just happened to see a tweet about this. ]

I am sorry to say, but I think you are going too fast. AFAICT, the support
for using : instead of _ in overrides is not yet in any Dunfell or Hardknott
release, only on the branches. I can't speak for others of course, but at
least we will not pick it up until it is included in an actual release.

There are releases of hardknott and dunfell currently in QA so this won't
make it in until the ones after that.
I understand the concern however I don't think it is reasonable to wait that
long. You do have the options of pulling in the changes earlier, or
back porting them and if that isn't an option, it shouldn't be that long until
the next releases happen.
I am trying hard to find ways to allow things to operate over multiple releases
and I think we have come up with a good plan there for this change. This is not
something we have ever planned or committed to supporting though. I think it
is in fact dangerous as it is effectively tying the project into never changing
anything, or if it does, planning changes in multiple years time which isn't in
my view reasonable. I worry that the bar to making any change like this is so
high as to put off anyone from ever doing it. The master branch is a development
branch at the end of the day.
Cheers,
Richard

Hi Richard,
please apologize my question but I am not sure I have a clear vision of the global impact of this new feature across the branches because the answers above are confusing me.

From what I understood so far and I've see in the poky repo, the 'master' branch already has the switch done on 02/08/21 [1]

I don't understand if it is planned to do the same in 'dunfell' LTS or other branches (how and when).


[1] https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=2abf8a699edd513405befbd1a0eafc8f55d6b514


Thank you
--
Marco

On Sat, 2021-07-31 at 15:29 +0000, Peter Kjellerstedt wrote:

-----Original Message-----
From: openembedded-architecture@... <openembedded-
architecture@...> On Behalf Of Richard Purdie
Sent: den 30 juli 2021 15:40
To: openembedded-architecture <openembedded-
architecture@...>
Subject: Re: [Openembedded-architecture] Overrides conversion plan

On Wed, 2021-07-28 at 16:43 +0100, Richard Purdie via
lists.openembedded.org wrote:


I have continued to work on this and I now think we're as ready as we'll
ever
be with the core. I have:

* submitted a section for the migration guide documenting the conversion
process
* increased the minimum bitbake version for OE-Core
* bumped the local.conf version to require new versions of the config file
* added an error to bitbake if it sees "_append"|"_prepend"|"_remove" in
  variable names which would suggest an unconverted layer. If you use
those
  in function names in the datastore that was never a good idea and is no
  longer supported.
* merged compatibility changes back to bitbake 1.50, 1.48 and 1.46
* merged the conversion script to OE-Core
* merged submitted tweaks to the conversion script (thanks Martin!)
* made OE-Core honister only, no longer supporting hardknott
* updated converted layers to be honister compatible
* converted autobuilder-helper to use the new syntax
* ensured all of a-quick builds on the autobuilder
* submitted patches for meta-yocto, meta-gplv2, meta-mingw and bitbake

I plan to merge these various things on Monday (2nd August). After that
time, unconverted layers will no longer work with master.

Cheers,

Richard

[ I am on summer vacation so I just happened to see a tweet about this. ]

I am sorry to say, but I think you are going too fast. AFAICT, the support
for using : instead of _ in overrides is not yet in any Dunfell or Hardknott
release, only on the branches. I can't speak for others of course, but at
least we will not pick it up until it is included in an actual release.

There are releases of hardknott and dunfell currently in QA so this won't
make it in until the ones after that.

I understand the concern however I don't think it is reasonable to wait that
long. You do have the options of pulling in the changes earlier, or 
back porting them and if that isn't an option, it shouldn't be that long until
the next releases happen.

I am trying hard to find ways to allow things to operate over multiple releases
and I think we have come up with a good plan there for this change. This is not 
something we have ever planned or committed to supporting though. I think it
is in fact dangerous as it is effectively tying the project into never changing
anything, or if it does, planning changes in multiple years time which isn't in
my view reasonable. I worry that the bar to making any change like this is so
high as to put off anyone from ever doing it. The master branch is a development
branch at the end of the day.

Cheers,

Richard

On Sat, Jul 31, 2021 at 5:29 AM Peter Kjellerstedt
<peter.kjellerstedt@...> wrote:

-----Original Message-----
From: openembedded-architecture@... <openembedded-
architecture@...> On Behalf Of Richard Purdie
Sent: den 30 juli 2021 15:40
To: openembedded-architecture <openembedded-
architecture@...>
Subject: Re: [Openembedded-architecture] Overrides conversion plan

On Wed, 2021-07-28 at 16:43 +0100, Richard Purdie via
lists.openembedded.org wrote:

I think the challenge is going to be the flag day issue for master

branches.

For example, there is code in devtool and other places which knows about

the

override character. If we allow mixing the different syntax for master

then

those tools need to complicate things by referencing both characters. To

try

and preserve what is left of my sanity, I'm starting to think we just

require

layers to migrate to the new syntax to continue to work with master. The

good

news is that those converted layers should work with dunfell and older

releases

where the layer already does that with the backported bitbake syntax

update.

If we accept that we need to have a flag day for master use, the

question is

when. We could pick some data well in the future or even post 3.4

however I'm

not sure this buys much and we probably may as well get on and do it.

Given these things, I therefore propose that we should start these

changes

and require it for master, probably relatively quickly within a couple

of

weeks?

I have continued to work on this and I now think we're as ready as we'll
ever
be with the core. I have:

* submitted a section for the migration guide documenting the conversion
process
* increased the minimum bitbake version for OE-Core
* bumped the local.conf version to require new versions of the config file
* added an error to bitbake if it sees "_append"|"_prepend"|"_remove" in
variable names which would suggest an unconverted layer. If you use
those
in function names in the datastore that was never a good idea and is no
longer supported.
* merged compatibility changes back to bitbake 1.50, 1.48 and 1.46
* merged the conversion script to OE-Core
* merged submitted tweaks to the conversion script (thanks Martin!)
* made OE-Core honister only, no longer supporting hardknott
* updated converted layers to be honister compatible
* converted autobuilder-helper to use the new syntax
* ensured all of a-quick builds on the autobuilder
* submitted patches for meta-yocto, meta-gplv2, meta-mingw and bitbake

I plan to merge these various things on Monday (2nd August). After that
time, unconverted layers will no longer work with master.

Cheers,

Richard

[ I am on summer vacation so I just happened to see a tweet about this. ]

I am sorry to say, but I think you are going too fast. AFAICT, the support
for using : instead of _ in overrides is not yet in any Dunfell or Hardknott
release, only on the branches. I can't speak for others of course, but at
least we will not pick it up until it is included in an actual release.

I am currently testing the change in dunfell and so far so good. If
no issues are found support will be present in the 3.1.11 release
currently scheduled for 2021/9/24

Steve

On Fri, Jul 30, 2021 at 3:40 PM Richard Purdie
<richard.purdie@...> wrote:

On Wed, 2021-07-28 at 16:43 +0100, Richard Purdie via lists.openembedded.org wrote:

I think the challenge is going to be the flag day issue for master branches.
For example, there is code in devtool and other places which knows about the
override character. If we allow mixing the different syntax for master then
those tools need to complicate things by referencing both characters. To try
and preserve what is left of my sanity, I'm starting to think we just require
layers to migrate to the new syntax to continue to work with master. The good
news is that those converted layers should work with dunfell and older releases
where the layer already does that with the backported bitbake syntax update.

If we accept that we need to have a flag day for master use, the question is
when. We could pick some data well in the future or even post 3.4 however I'm
not sure this buys much and we probably may as well get on and do it.

Given these things, I therefore propose that we should start these changes
and require it for master, probably relatively quickly within a couple of
weeks?

I have continued to work on this and I now think we're as ready as we'll ever
be with the core. I have:

* submitted a section for the migration guide documenting the conversion process
* increased the minimum bitbake version for OE-Core
* bumped the local.conf version to require new versions of the config file
* added an error to bitbake if it sees "_append"|"_prepend"|"_remove" in
variable names which would suggest an unconverted layer. If you use those
in function names in the datastore that was never a good idea and is no
longer supported.
* merged compatibility changes back to bitbake 1.50, 1.48 and 1.46
* merged the conversion script to OE-Core
* merged submitted tweaks to the conversion script (thanks Martin!)
* made OE-Core honister only, no longer supporting hardknott
* updated converted layers to be honister compatible
* converted autobuilder-helper to use the new syntax
* ensured all of a-quick builds on the autobuilder
* submitted patches for meta-yocto, meta-gplv2, meta-mingw and bitbake

I plan to merge these various things on Monday (2nd August). After that time,
unconverted layers will no longer work with master.

Cheers,

Richard

Thank you very much Martin and Richard.
Cheers
A.A.

On Wed, 2021-07-28 at 16:43 +0100, Richard Purdie via lists.openembedded.org wrote:

I think the challenge is going to be the flag day issue for master branches.
For example, there is code in devtool and other places which knows about the
override character. If we allow mixing the different syntax for master then
those tools need to complicate things by referencing both characters. To try
and preserve what is left of my sanity, I'm starting to think we just require
layers to migrate to the new syntax to continue to work with master. The good
news is that those converted layers should work with dunfell and older releases
where the layer already does that with the backported bitbake syntax update.

If we accept that we need to have a flag day for master use, the question is 
when. We could pick some data well in the future or even post 3.4 however I'm
not sure this buys much and we probably may as well get on and do it.

Given these things, I therefore propose that we should start these changes 
and require it for master, probably relatively quickly within a couple of 
weeks?

I have continued to work on this and I now think we're as ready as we'll ever
be with the core. I have:

* submitted a section for the migration guide documenting the conversion process
* increased the minimum bitbake version for OE-Core
* bumped the local.conf version to require new versions of the config file
* added an error to bitbake if it sees "_append"|"_prepend"|"_remove" in 
variable names which would suggest an unconverted layer. If you use those 
in function names in the datastore that was never a good idea and is no 
longer supported.
* merged compatibility changes back to bitbake 1.50, 1.48 and 1.46
* merged the conversion script to OE-Core
* merged submitted tweaks to the conversion script (thanks Martin!)
* made OE-Core honister only, no longer supporting hardknott
* updated converted layers to be honister compatible
* converted autobuilder-helper to use the new syntax
* ensured all of a-quick builds on the autobuilder
* submitted patches for meta-yocto, meta-gplv2, meta-mingw and bitbake

I plan to merge these various things on Monday (2nd August). After that time,
unconverted layers will no longer work with master.

Cheers,

Richard

Hi Marco,

On Thu, 2021-07-29 at 08:50 +0200, Marco Cavallini wrote:

IMHO it is important to produce a clear documentation about the new
features maybe providing some practical cases.

I agree it is important this is documented. I've sent a patch to the manuals 
to add a section to the 3.4 migration guide:

https://lists.yoctoproject.org/g/docs/message/1589

Does that help?

Hopefully that should start to document things and I'd welcome help in knowing
what other information we need to add there. It is a little more verbose
than some other migration guide sections since this one is something a lot
of people will run into.

Cheers,

Richard

On 7/28/21 10:43 AM, Richard Purdie wrote:

Hi All,

I've spent quite a bit of time seeing how to progress things. I now
have a script which can convert layers and seems to have reasonable success
on OE-Core, bitbake, yocto-docs, meta-yocto, meta-gplv2 and meta-mingw. I've
proposed this for OE-Core along with an example conversion patch for OE-Core.

The current patch status is I have conversions (including to bitbake's core)
in master-next branches. This nearly has q-quick builds passing on the 
autobuilder, there are a handful of oe-selftest cases which don't pass yet.

Things do get a little more complex than I'd hoped. We use OVERRIDES
in a number of interesting places, including within multilib configurations
for the tune-XXX variable suffixes. I did try making the code do "fallback"
variable lookups instead of expansion for tune- but I quickly gave up due
to the nesting. My view is that they are being used as overrides and we should 
therefore recognise them as such. Over time I think the tune code can be
improved to take explicit advantage of that.

As such, I'm proposing we make the package variables and tune variables
explicit overrides and the conversion scripts and patches above assume this.

I think the challenge is going to be the flag day issue for master branches.
For example, there is code in devtool and other places which knows about the
override character. If we allow mixing the different syntax for master then
those tools need to complicate things by referencing both characters. To try
and preserve what is left of my sanity, I'm starting to think we just require
layers to migrate to the new syntax to continue to work with master. The good
news is that those converted layers should work with dunfell and older releases
where the layer already does that with the backported bitbake syntax update.

If we accept that we need to have a flag day for master use, the question is 
when. We could pick some data well in the future or even post 3.4 however I'm
not sure this buys much and we probably may as well get on and do it.

Given these things, I therefore propose that we should start these changes 
and require it for master, probably relatively quickly within a couple of 
weeks?

Sooner the better is my view...

Thanks for all of this, I know it's going to make things better longer term no
matter when it switches over to the new format. (Also making the tune-XXX a
first class override makes sense!)

--Mark

Cheers,

Richard

On 28/07/21 17:43, Richard Purdie wrote:

Hi All,
I've spent quite a bit of time seeing how to progress things. I now
have a script which can convert layers and seems to have reasonable success
on OE-Core, bitbake, yocto-docs, meta-yocto, meta-gplv2 and meta-mingw. I've
proposed this for OE-Core along with an example conversion patch for OE-Core.
The current patch status is I have conversions (including to bitbake's core)
in master-next branches. This nearly has q-quick builds passing on the
autobuilder, there are a handful of oe-selftest cases which don't pass yet.
Things do get a little more complex than I'd hoped. We use OVERRIDES
in a number of interesting places, including within multilib configurations
for the tune-XXX variable suffixes. I did try making the code do "fallback"
variable lookups instead of expansion for tune- but I quickly gave up due
to the nesting. My view is that they are being used as overrides and we should
therefore recognise them as such. Over time I think the tune code can be
improved to take explicit advantage of that.
As such, I'm proposing we make the package variables and tune variables
explicit overrides and the conversion scripts and patches above assume this.
I think the challenge is going to be the flag day issue for master branches.
For example, there is code in devtool and other places which knows about the
override character. If we allow mixing the different syntax for master then
those tools need to complicate things by referencing both characters. To try
and preserve what is left of my sanity, I'm starting to think we just require
layers to migrate to the new syntax to continue to work with master. The good
news is that those converted layers should work with dunfell and older releases
where the layer already does that with the backported bitbake syntax update.
If we accept that we need to have a flag day for master use, the question is
when. We could pick some data well in the future or even post 3.4 however I'm
not sure this buys much and we probably may as well get on and do it.
Given these things, I therefore propose that we should start these changes
and require it for master, probably relatively quickly within a couple of
weeks?
Cheers,
Richard

Hi Richard,
thank you for your precious work.

IMHO it is important to produce a clear documentation about the new features maybe providing some practical cases.

Just my 2 cents,

Cheers,
--
Marco

Em qua., 28 de jul. de 2021 às 12:43, Richard Purdie
<richard.purdie@...> escreveu:

Given these things, I therefore propose that we should start these changes
and require it for master, probably relatively quickly within a couple of
weeks?

I believe your plan is good and given the new layers will work down to
dunfell I see no big problems in requiring it to work on master. In
fact, I think it is the most logical move as it allows master to
optimize for its new format.

--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9 9981-7854 Mobile: +1 (347) 903-9750

Hi All,

I've spent quite a bit of time seeing how to progress things. I now
have a script which can convert layers and seems to have reasonable success
on OE-Core, bitbake, yocto-docs, meta-yocto, meta-gplv2 and meta-mingw. I've
proposed this for OE-Core along with an example conversion patch for OE-Core.

The current patch status is I have conversions (including to bitbake's core)
in master-next branches. This nearly has q-quick builds passing on the 
autobuilder, there are a handful of oe-selftest cases which don't pass yet.

Things do get a little more complex than I'd hoped. We use OVERRIDES
in a number of interesting places, including within multilib configurations
for the tune-XXX variable suffixes. I did try making the code do "fallback"
variable lookups instead of expansion for tune- but I quickly gave up due
to the nesting. My view is that they are being used as overrides and we should 
therefore recognise them as such. Over time I think the tune code can be
improved to take explicit advantage of that.

As such, I'm proposing we make the package variables and tune variables
explicit overrides and the conversion scripts and patches above assume this.

I think the challenge is going to be the flag day issue for master branches.
For example, there is code in devtool and other places which knows about the
override character. If we allow mixing the different syntax for master then
those tools need to complicate things by referencing both characters. To try
and preserve what is left of my sanity, I'm starting to think we just require
layers to migrate to the new syntax to continue to work with master. The good
news is that those converted layers should work with dunfell and older releases
where the layer already does that with the backported bitbake syntax update.

If we accept that we need to have a flag day for master use, the question is 
when. We could pick some data well in the future or even post 3.4 however I'm
not sure this buys much and we probably may as well get on and do it.

Given these things, I therefore propose that we should start these changes 
and require it for master, probably relatively quickly within a couple of 
weeks?

Cheers,

Richard

On Tue, Jul 27, 2021 at 05:40:32PM +0100, Richard Purdie wrote:

On Tue, 2021-07-27 at 12:38 -0400, Denys Dmytriyenko wrote:

On Tue, Jul 27, 2021 at 08:18:13AM +0100, Richard Purdie wrote:

On Mon, 2021-07-26 at 23:51 -0400, Denys Dmytriyenko wrote:

All,

The layer has been updated and moved to git.yoctoproject.org under a central
mixin repo called "meta-lts-mixins", branch "dunfell/kernel":
http://git.yoctoproject.org/cgit/cgit.cgi/meta-lts-mixins/

The naming is in accordance with the LTS mixin layer guidelines:
https://wiki.yoctoproject.org/wiki/Stable_Release_and_LTS#LTS_.E2.80.9CMixin.E2.80.9D_repositories

Other mixin layers are also welcome!

Should the layer be called kernel-5.10 since there potentially may be other
versions?

Either that, or it can hold multiple versions...

I think that defeats the idea of specific purpose layers?

Not necessarily. Providing latest LTS kernels for Dunfell also seems very
specific. For every extra year of official support for Dunfell, this can
offer a new LTS kernel version, that can be selected by PREFERRED_VERSION.
Creating a new branch for each kernel version seems wasteful. But I can do
that as well, if needed.

--
Regards,
Denys Dmytriyenko <denis@...>
PGP: 0x420902729A92C964 - https://denix.org/0x420902729A92C964
Fingerprint: 25FC E4A5 8A72 2F69 1186 6D76 4209 0272 9A92 C964

On Tue, 2021-07-27 at 12:38 -0400, Denys Dmytriyenko wrote:

On Tue, Jul 27, 2021 at 08:18:13AM +0100, Richard Purdie wrote:

On Mon, 2021-07-26 at 23:51 -0400, Denys Dmytriyenko wrote:

All,

The layer has been updated and moved to git.yoctoproject.org under a central
mixin repo called "meta-lts-mixins", branch "dunfell/kernel":
http://git.yoctoproject.org/cgit/cgit.cgi/meta-lts-mixins/

The naming is in accordance with the LTS mixin layer guidelines:
https://wiki.yoctoproject.org/wiki/Stable_Release_and_LTS#LTS_.E2.80.9CMixin.E2.80.9D_repositories

Other mixin layers are also welcome!

Should the layer be called kernel-5.10 since there potentially may be other
versions?

Either that, or it can hold multiple versions...

I think that defeats the idea of specific purpose layers?

Cheers,

Richard