Date   

Re: layers.openembedded.org - Layer Policy Proposal (revised)

Armin Kuster
 



On 12/20/19 5:37 PM, Mark Hatle wrote:
I am proposing the following as a new policy for the layers in the layer index.
Recently I have been working with the layer index and it's become clear that
'master' is pretty much unusable as it stands.  Below is my proposal to clean
this up and keep it clean.  In addition to the proposal, at the end, I will
include the ramifications of implementing this proposal.  (All day is current as
of yesterday.)

This proposal has been updated based on the feedback received from the original
thread with the subject: layers.openembedded.org - suggestion to remove obsolete
layers

Layer Index Proposal
--------------------

Background:

The layer index (layers.openembedded.org) provides a convenient method to index
existing layers, as well as track the creation of new supported branches.
Currently the layer index indexes Yocto Project release branches (and associated
OpenEmbedded and layers) starting with Danny 1.3 through Zeus 3.0, as well as
the master branch.

The layer index is mostly used by people connecting their web browser and
reviewing things in a human readable form.  However, we have automated processes
that use this same information to help construct and manage projects.  For
instance, ‘bitbake-layers layerindex-fetch <layer>’ can be used to query the
layer index, and download the layer and it’s layer dependencies into an existing
project.

Two problems have been identified:

1. As URLs change or git projects/servers go away, the layer index is not
updated to reflect a change or remove obsolete — inaccessible items.  At a
minimum we need to identify layers no longer have valid URLs.

2. Layers that are obsolete and no longer maintained are still present and can
cause confusion and make it harder for users to identify useful components.

There is value in carrying a layer in the index, even if it is not currently
valid, as this can induce users to fix and contribute changes to those layers.
However, after the layer has been invalid for a significant period of time it
can be detrimental to the project as it suggests to people that OpenEmbedded and
Yocto Project layers are unmaintained, out of date and/or contain insecure software.

The following policy is being proposed for the layer index.  The policy items
can be accomplished though automation, as described below.

Policy:

For all branches, the following criteria will be evaluated on a regular basis:

- Is the layer URL still valid?

If the layer URL has not been valid for 30 days or more, an email will be sent
to the registered layer maintainers and a note will be added to indicate this
layer appears to no longer be available, and will be removed unless the URL is
update.

If the URL is updated, or becomes valid the note will be cleared once the URL is
valid.

After 90 days, the layer will be removed from the index as no longer available.


- Does the layer claim to support (LAYERSERIES_COMPAT) that is compatible with
oe-core in that release branch?  (Only applies to sumo and newer branches)

If the layer does not claim to be compatible with the branch after 30 days, an
email will be sent to the registered layer maintainers and a note will be added
to indicate the layer is not listed as compatible with oe-core on the specific
branch.

If the LAYERSERIES_COMPAT is updated, the note will be removed.

Further for the _master_ branch only, the following will be evaluated:

- If the LAYERSERIES_COMPAT is not listed as compatible (see the previous item),
has the layer’s “master related” branch been updated within the last 18 months?

If the layer is not compatible, and the branch has not been updated within the
last 18 months the layer will be removed from the “master branch” index as
inactive.  Note: release branches are not expected to be affected by this change!

The above items will processed via automation.  (See the attachment as to what
changes would occur if the above policy proposal is approved.)

Thanks for taking the time to put this together.


Additional Request
------------------

During the initial proposal that was sent, another item was identified.  Below
covers this suggested change to the layer index itself.

It was request that we should add a delete or ‘unpublish’ button accessible to
the owners of the layer.  It is my opinion that actually deleting the layer
should be reserved for layer index administrators to prevent bad actors from
removing layers that are otherwise still valid.  Unpublishing a layer could
still cause temporary inconvenience, but would be very easy to revert.

Well, I requested the removal of one of my layers over IRC or maybe it was an email. How do you know it was Me?




Attachment
----------

The attachments are tables per branch.  A few of the indications are explained here:

!URL - The URL is invalid

!layer - the layer itself isn't valid, but the URL worked

!Compat - The layer does not indicate it is compatible

How is a layer identified that  passes the yocto-check-layer or has been met the "Yocto Compatible" badge? Are we overloading "Compatible" here?

Note - A note needs to be added to this layer in the index

Email - The maintainer of the layer would be contacted prior (not yet slated for
removal)

Remove - the layer should be removed from this branch's index

(master only)

LAST - number of days since the last commit, if !COMPAT is detected.


Stats:

(in all cases the number of layers with bad urls exceeds the 90 days.)

Sumo - Remove 3 layers
     - Add compat notes to 11 layers

Thud - Remove 1 layer
     - Add compat notes to 4 layers

Warrior - Remove 5 layers
        - Add compat notes to 2 layers

Zeus - (no layers to remove)
     - Add compat notes to 3 layers

Master - Remove 129 layers
       - Add compat notes to 103 layers

_______________________________________________
Openembedded-architecture mailing list
Openembedded-architecture@...
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture


layers.openembedded.org - Layer Policy Proposal (revised)

Mark Hatle
 

I am proposing the following as a new policy for the layers in the layer index.
Recently I have been working with the layer index and it's become clear that
'master' is pretty much unusable as it stands. Below is my proposal to clean
this up and keep it clean. In addition to the proposal, at the end, I will
include the ramifications of implementing this proposal. (All day is current as
of yesterday.)

This proposal has been updated based on the feedback received from the original
thread with the subject: layers.openembedded.org - suggestion to remove obsolete
layers

Layer Index Proposal
--------------------

Background:

The layer index (layers.openembedded.org) provides a convenient method to index
existing layers, as well as track the creation of new supported branches.
Currently the layer index indexes Yocto Project release branches (and associated
OpenEmbedded and layers) starting with Danny 1.3 through Zeus 3.0, as well as
the master branch.

The layer index is mostly used by people connecting their web browser and
reviewing things in a human readable form. However, we have automated processes
that use this same information to help construct and manage projects. For
instance, ‘bitbake-layers layerindex-fetch <layer>’ can be used to query the
layer index, and download the layer and it’s layer dependencies into an existing
project.

Two problems have been identified:

1. As URLs change or git projects/servers go away, the layer index is not
updated to reflect a change or remove obsolete — inaccessible items. At a
minimum we need to identify layers no longer have valid URLs.

2. Layers that are obsolete and no longer maintained are still present and can
cause confusion and make it harder for users to identify useful components.

There is value in carrying a layer in the index, even if it is not currently
valid, as this can induce users to fix and contribute changes to those layers.
However, after the layer has been invalid for a significant period of time it
can be detrimental to the project as it suggests to people that OpenEmbedded and
Yocto Project layers are unmaintained, out of date and/or contain insecure software.

The following policy is being proposed for the layer index. The policy items
can be accomplished though automation, as described below.

Policy:

For all branches, the following criteria will be evaluated on a regular basis:

- Is the layer URL still valid?

If the layer URL has not been valid for 30 days or more, an email will be sent
to the registered layer maintainers and a note will be added to indicate this
layer appears to no longer be available, and will be removed unless the URL is
update.

If the URL is updated, or becomes valid the note will be cleared once the URL is
valid.

After 90 days, the layer will be removed from the index as no longer available.


- Does the layer claim to support (LAYERSERIES_COMPAT) that is compatible with
oe-core in that release branch? (Only applies to sumo and newer branches)

If the layer does not claim to be compatible with the branch after 30 days, an
email will be sent to the registered layer maintainers and a note will be added
to indicate the layer is not listed as compatible with oe-core on the specific
branch.

If the LAYERSERIES_COMPAT is updated, the note will be removed.

Further for the _master_ branch only, the following will be evaluated:

- If the LAYERSERIES_COMPAT is not listed as compatible (see the previous item),
has the layer’s “master related” branch been updated within the last 18 months?

If the layer is not compatible, and the branch has not been updated within the
last 18 months the layer will be removed from the “master branch” index as
inactive. Note: release branches are not expected to be affected by this change!

The above items will processed via automation. (See the attachment as to what
changes would occur if the above policy proposal is approved.)


Additional Request
------------------

During the initial proposal that was sent, another item was identified. Below
covers this suggested change to the layer index itself.

It was request that we should add a delete or ‘unpublish’ button accessible to
the owners of the layer. It is my opinion that actually deleting the layer
should be reserved for layer index administrators to prevent bad actors from
removing layers that are otherwise still valid. Unpublishing a layer could
still cause temporary inconvenience, but would be very easy to revert.


Attachment
----------

The attachments are tables per branch. A few of the indications are explained here:

!URL - The URL is invalid

!layer - the layer itself isn't valid, but the URL worked

!Compat - The layer does not indicate it is compatible

Note - A note needs to be added to this layer in the index

Email - The maintainer of the layer would be contacted prior (not yet slated for
removal)

Remove - the layer should be removed from this branch's index

(master only)

LAST - number of days since the last commit, if !COMPAT is detected.


Stats:

(in all cases the number of layers with bad urls exceeds the 90 days.)

Sumo - Remove 3 layers
- Add compat notes to 11 layers

Thud - Remove 1 layer
- Add compat notes to 4 layers

Warrior - Remove 5 layers
- Add compat notes to 2 layers

Zeus - (no layers to remove)
- Add compat notes to 3 layers

Master - Remove 129 layers
- Add compat notes to 103 layers


Yocto Project stable release changes

Richard Purdie
 

There has been a lot of discussion about stable releases and how the
project might handle them going forward. The TSC has put together some
process/policy information on the project wiki:

https://wiki.yoctoproject.org/wiki/LTS

As part of this we've included information about how an LTS could work
and how it fits with other stable releases. At this point this is not a
commitment to an LTS, its documentation about how the TSC believes one
should work.

Any commitment to an LTS release is something the governing board would
have to make and is being actively discussed.

The aim here is to have a consistent process for how stable releases
work and are maintained, and also how at the end of this there is an
option for community support.

This does change a few things from how the project currently operates
but given resources (human and automated), the TSC believes this should
given the best outcome for supporting the community and ensuring
project quality given the resources we have.

Cheers,

Richard
(on behalf of the YP TSC)


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Armin Kuster
 

Hello,


On 12/12/19 5:13 PM, Randy MacLeod wrote:
On 12/12/19 6:53 PM, Randy MacLeod wrote:
On 12/10/19 2:34 PM, Khem Raj wrote:
On Tue, Dec 10, 2019 at 9:23 AM Randy MacLeod
<randy.macleod@...> wrote:

In the YP technical meeting today, Richard suggested that we stop
support for CentOS-7. Is there any objection to doing so
before 3.1-M2?

CentOS-7 is just too old and there is significant work to support it.
I am in support of it, but then I also fear that many corporate
policies might still
be using it until 2024 when security updates end so perhaps would
like to hear
centos7 users here. if no one speaks up then we can safely retire it
before 3.1

I confirmed that some of our customers still use RHEL-7 for
many of their build machines. Some have only recently gone through
the pain
of upgrading from RHEL-6. Such an upgrade is relatively simple for
individual
developers but for large organizations it can take many if not 10s of
person-years
of work. It would therefore be difficult, costly, and painful for
them to migrate
again so soon. As Khem mentioned in another thread, supporting using
SCL:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Developer_Guide/scl-utils.html

may not be viable for such organizations either.


At some point, be it in 2020 or 2021, such organizations will have to
either
update their OSes, use containers/VMs, or arrange for custom support
for older distros
but many users would prefer for that day be delayed until after
oe-core-3.2 to
coincide with the end of support for new hardware installs.
FYI, for CentOS-7, the end of new hardware support/ 'Full Updates' is
Q4 2020:
   https://wiki.centos.org/About/Product
There seems to be interest in solving this issue. I strongly suggest
those interested should get involved.  I have opened a defect to track
the issue to keep Centos7 in the AB.

https://bugzilla.yoctoproject.org/show_bug.cgi?id=13714

The YP TSC is putting a time limit  of  1/20/2020 (3.1-M2) when we will
revisit this topic and determine where we go from there.

Kind regards,
Armin
On behave of the Yocto Project TSC.


../Randy


../Randy


Richard said that many if not most of the package upgrades that he
deals with fail for CentOS-7 and he has to either fix them himself
or get the person who submitted the work to do so. Newer distributions
are not nearly so problematic.

While the CentOS-7 distro is still a supported by it's provider,
the toolchain is very old:
   - gcc-4.8
   - glibc-2.17
   - binutils-2.27

One could add a newer toolchain to the buildtools tarball to address
some of the CentOS-7 support problems. So far, we have only use
the host's toolchain and it seems best to continue to do so.

Release and support dates for CentOS-6,7,8 are here:
    https://wiki.centos.org/About/Product
Note that 'Full Updates' or new hardware support for
CentOS-7 stops in Q3 2020.

-- 
# Randy MacLeod
# Wind River Linux
_______________________________________________
Openembedded-architecture mailing list
Openembedded-architecture@...
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Diego Santa Cruz
 

-----Original Message-----
From: Khem Raj <raj.khem@...>
Sent: 12 December 2019 20:38
To: Diego Santa Cruz <Diego.SantaCruz@...>
Cc: Randy MacLeod <randy.macleod@...>; openembedded-
architecture <openembedded-architecture@...>
Subject: Re: [Openembedded-architecture] Yocto support for Centos-7 (RHEL-7):
drop in early 2020?

On Tue, Dec 10, 2019 at 11:14 PM Diego Santa Cruz
<Diego.SantaCruz@...> wrote:

-----Original Message-----
From: openembedded-architecture-bounces@...
<openembedded-architecture-bounces@...> On
Behalf Of
Khem Raj
Sent: 10 December 2019 20:35
To: Randy MacLeod <randy.macleod@...>
Cc: openembedded-architecture <openembedded-
architecture@...>
Subject: Re: [Openembedded-architecture] Yocto support for Centos-7
(RHEL-7):
drop in early 2020?

On Tue, Dec 10, 2019 at 9:23 AM Randy MacLeod
<randy.macleod@...> wrote:


In the YP technical meeting today, Richard suggested that we stop
support for CentOS-7. Is there any objection to doing so before
3.1-M2?

CentOS-7 is just too old and there is significant work to support it.
I am in support of it, but then I also fear that many corporate policies might
still
be using it until 2024 when security updates end so perhaps would like to
hear
centos7 users here. if no one speaks up then we can safely retire it before
3.1
While I see what the motivation to remove CentOS-7 is, dropping it will
probably create issues for people. CentOS 8 was released not so long ago. In our
case we have older products (among which Yocto based ones) which do not
necessarily work on CentOS 8. CentOS 8 is relatively recent, so we have not had
time to test how old products work on it.
it will start with 3.1 which means that this impacts when user is
upgrading beyong 3.1, so there still is time to plan upgrade of build
infra if one plans to.

Isn't it possible to require things like devtoolset-8 (gcc 8.3, binutils-2.30) on
CentOS 7 to keep it going?
It certainly is, and perhaps a good writeup enabling scl to use
updated toolchain might be a good contribution to yocto projects "How
do I" page [1]
however, in many cases IT folks dont let users install packages as
they wish, so it still would require some process change on top of
prerequisites that
yocto already asks for.
The setup is not difficult, I could manage to write something up in Feb., before that it will be difficult to set time aside.

But if the system's glibc on CentOS 7 causes problems then that's another story, as to the best of my knowledge there is no scl (nor any equivalent concept) for that.

In our case we are using devtoolset-8 with Yocto thud on CentOS 7 with
success, as some layers require a recent host gcc to build.
it certainly means extra work for setup etc and scl is interactive, so
perhaps if its described clearly how one can use it that might help
end users.
Well, scl is not necessary interactive, although it is generally presented like that. But in any case it remains easier than updating existing systems to CentOS 8 ;-)


[1] https://wiki.yoctoproject.org/wiki/How_do_I
--
Diego Santa Cruz, PhD
Technology Architect
spinetix.com


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Josef Holzmayr <holzmayr@...>
 

On Thu, Dec 12, 2019 at 06:53:01PM -0500, Randy MacLeod wrote:
On 12/10/19 2:34 PM, Khem Raj wrote:
On Tue, Dec 10, 2019 at 9:23 AM Randy MacLeod
<randy.macleod@...> wrote:

In the YP technical meeting today, Richard suggested that we stop
support for CentOS-7. Is there any objection to doing so
before 3.1-M2?

CentOS-7 is just too old and there is significant work to support it.
I am in support of it, but then I also fear that many corporate
policies might still
be using it until 2024 when security updates end so perhaps would like to hear
centos7 users here. if no one speaks up then we can safely retire it before 3.1

I confirmed that some of our customers still use RHEL-7 for
many of their build machines. Some have only recently gone through the pain
of upgrading from RHEL-6. Such an upgrade is relatively simple for individual
developers but for large organizations it can take many if not 10s of person-years
of work.
Actually if a company is so large that it takes man-years for a
migration, then it is for sure cheaper to allocate developer time to OE,
making sure the project suits their needs.

Because that sounds, at least to my very limited point of view, like:
"We are super huge and important, thou shall do what we need." instead of
"We are super huge and important, hence we can archieve things that
others cannot."

My $.02

--
———————————————
Josef Holzmayr
Software Developer Embedded Systems

Tel: +49 8444 9204-48
Fax: +49 8444 9204-50

R-S-I Elektrotechnik GmbH & Co. KG
Woelkestrasse 11
D-85301 Schweitenkirchen
www.rsi-elektrotechnik.de
———————————————
Amtsgericht Ingolstadt – GmbH: HRB 191328 – KG: HRA 170393
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
Ust-IdNr: DE 128592548

_____________________________________________________________
Amtsgericht Ingolstadt - GmbH: HRB 191328 - KG: HRA 170363
Geschäftsführer: Dr.-Ing. Michael Sorg, Dipl.-Ing. Franz Sorg
USt-IdNr.: DE 128592548


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Randy MacLeod
 

On 12/12/19 6:53 PM, Randy MacLeod wrote:
On 12/10/19 2:34 PM, Khem Raj wrote:
On Tue, Dec 10, 2019 at 9:23 AM Randy MacLeod
<randy.macleod@...> wrote:

In the YP technical meeting today, Richard suggested that we stop
support for CentOS-7. Is there any objection to doing so
before 3.1-M2?

CentOS-7 is just too old and there is significant work to support it.
I am in support of it, but then I also fear that many corporate
policies might still
be using it until 2024 when security updates end so perhaps would like to hear
centos7 users here. if no one speaks up then we can safely retire it before 3.1

I confirmed that some of our customers still use RHEL-7 for
many of their build machines. Some have only recently gone through the pain
of upgrading from RHEL-6. Such an upgrade is relatively simple for individual
developers but for large organizations it can take many if not 10s of person-years
of work. It would therefore be difficult, costly, and painful for them to migrate
again so soon. As Khem mentioned in another thread, supporting using SCL:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Developer_Guide/scl-utils.html
may not be viable for such organizations either.


At some point, be it in 2020 or 2021, such organizations will have to either
update their OSes, use containers/VMs, or arrange for custom support for older distros
but many users would prefer for that day be delayed until after oe-core-3.2 to
coincide with the end of support for new hardware installs.
FYI, for CentOS-7, the end of new hardware support/ 'Full Updates' is Q4 2020:
   https://wiki.centos.org/About/Product

../Randy


../Randy


Richard said that many if not most of the package upgrades that he
deals with fail for CentOS-7 and he has to either fix them himself
or get the person who submitted the work to do so. Newer distributions
are not nearly so problematic.

While the CentOS-7 distro is still a supported by it's provider,
the toolchain is very old:
   - gcc-4.8
   - glibc-2.17
   - binutils-2.27

One could add a newer toolchain to the buildtools tarball to address
some of the CentOS-7 support problems. So far, we have only use
the host's toolchain and it seems best to continue to do so.

Release and support dates for CentOS-6,7,8 are here:
    https://wiki.centos.org/About/Product
Note that 'Full Updates' or new hardware support for
CentOS-7 stops in Q3 2020.

--
# Randy MacLeod
# Wind River Linux
_______________________________________________
Openembedded-architecture mailing list
Openembedded-architecture@...
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture
--
# Randy MacLeod
# Wind River Linux


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Randy MacLeod
 

On 12/10/19 2:34 PM, Khem Raj wrote:
On Tue, Dec 10, 2019 at 9:23 AM Randy MacLeod
<randy.macleod@...> wrote:

In the YP technical meeting today, Richard suggested that we stop
support for CentOS-7. Is there any objection to doing so
before 3.1-M2?

CentOS-7 is just too old and there is significant work to support it.
I am in support of it, but then I also fear that many corporate
policies might still
be using it until 2024 when security updates end so perhaps would like to hear
centos7 users here. if no one speaks up then we can safely retire it before 3.1

I confirmed that some of our customers still use RHEL-7 for
many of their build machines. Some have only recently gone through the pain
of upgrading from RHEL-6. Such an upgrade is relatively simple for individual
developers but for large organizations it can take many if not 10s of person-years
of work. It would therefore be difficult, costly, and painful for them to migrate
again so soon. As Khem mentioned in another thread, supporting using SCL:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Developer_Guide/scl-utils.html
may not be viable for such organizations either.


At some point, be it in 2020 or 2021, such organizations will have to either
update their OSes, use containers/VMs, or arrange for custom support for older distros
but many users would prefer for that day be delayed until after oe-core-3.2 to
coincide with the end of support for new hardware installs.

../Randy


Richard said that many if not most of the package upgrades that he
deals with fail for CentOS-7 and he has to either fix them himself
or get the person who submitted the work to do so. Newer distributions
are not nearly so problematic.

While the CentOS-7 distro is still a supported by it's provider,
the toolchain is very old:
- gcc-4.8
- glibc-2.17
- binutils-2.27

One could add a newer toolchain to the buildtools tarball to address
some of the CentOS-7 support problems. So far, we have only use
the host's toolchain and it seems best to continue to do so.

Release and support dates for CentOS-6,7,8 are here:
https://wiki.centos.org/About/Product
Note that 'Full Updates' or new hardware support for
CentOS-7 stops in Q3 2020.

--
# Randy MacLeod
# Wind River Linux
_______________________________________________
Openembedded-architecture mailing list
Openembedded-architecture@...
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture

--
# Randy MacLeod
# Wind River Linux


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Khem Raj
 

On Tue, Dec 10, 2019 at 11:14 PM Diego Santa Cruz
<Diego.SantaCruz@...> wrote:

-----Original Message-----
From: openembedded-architecture-bounces@...
<openembedded-architecture-bounces@...> On Behalf Of
Khem Raj
Sent: 10 December 2019 20:35
To: Randy MacLeod <randy.macleod@...>
Cc: openembedded-architecture <openembedded-
architecture@...>
Subject: Re: [Openembedded-architecture] Yocto support for Centos-7 (RHEL-7):
drop in early 2020?

On Tue, Dec 10, 2019 at 9:23 AM Randy MacLeod
<randy.macleod@...> wrote:


In the YP technical meeting today, Richard suggested that we stop
support for CentOS-7. Is there any objection to doing so before
3.1-M2?

CentOS-7 is just too old and there is significant work to support it.
I am in support of it, but then I also fear that many corporate policies might still
be using it until 2024 when security updates end so perhaps would like to hear
centos7 users here. if no one speaks up then we can safely retire it before 3.1
While I see what the motivation to remove CentOS-7 is, dropping it will probably create issues for people. CentOS 8 was released not so long ago. In our case we have older products (among which Yocto based ones) which do not necessarily work on CentOS 8. CentOS 8 is relatively recent, so we have not had time to test how old products work on it.
it will start with 3.1 which means that this impacts when user is
upgrading beyong 3.1, so there still is time to plan upgrade of build
infra if one plans to.

Isn't it possible to require things like devtoolset-8 (gcc 8.3, binutils-2.30) on CentOS 7 to keep it going?
It certainly is, and perhaps a good writeup enabling scl to use
updated toolchain might be a good contribution to yocto projects "How
do I" page [1]
however, in many cases IT folks dont let users install packages as
they wish, so it still would require some process change on top of
prerequisites that
yocto already asks for.

In our case we are using devtoolset-8 with Yocto thud on CentOS 7 with success, as some layers require a recent host gcc to build.
it certainly means extra work for setup etc and scl is interactive, so
perhaps if its described clearly how one can use it that might help
end users.

[1] https://wiki.yoctoproject.org/wiki/How_do_I


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Andre McCurdy
 

On Wed, Dec 11, 2019 at 9:55 PM Adrian Bunk <bunk@...> wrote:
On Wed, Dec 11, 2019 at 08:48:04PM -0800, Andre McCurdy wrote:
On Wed, Dec 11, 2019 at 7:38 PM Adrian Bunk <bunk@...> wrote:

There is no need to question whether niche features like gcc or GNOME
for the target is a useful way of spending resources since the work is
done by the people interested in the feature.
Yes, that's probably expected. Developers work on stuff which
interests them. That's precisely why there's a danger that effort and
resources don't always get spent on the work which would benefit the
widest range of users.
This is not a danger, this is how open source projects work.
Yes, it's how open source projects work. It's also a model where
there's a risk that the best interests of the wider user base don't
get much consideration. There's no conflict between those two
statements.

Yocto is not a company where all developers are paid resources that can
be told what to spend their time on.

Developers spend effort on what is useful for their local paid work,
or what is fun to do as a hobby.

...
In some cases this even creates new bugs for everyone else,
like the breakage caused by an incorrect CentOS 7 workaround in nettle
that needed a last-minute fixup before the release of Yocto 2.7.
That was an embarrassing mistake, which you've brought up before.
Not exactly embarrassing, the problem is obvious only in hindsight.

It happens sometimes. Dropping support for CentOS 7 or musl isn't going
to stop embarrassing mistakes happening again.
...
It avoids bugs being introduced by workarounds.

Trying to build and run 2020 software on a 2013 distribution is
problematic, and will generate more and more problems noone else
has seen or is interested in fixing.
In general yes, but build environments are somewhat of a special in
that one of their key functions is to isolate the code being built
from any specific details of the host. OE already does a very good job
of doing that (by building -native versions of many of the tools
traditionally provided by the host in legacy build environment, etc)
but if there are ways to improve that isolation even further then I
don't think we should dismiss efforts to work on them just because key
developers are happy with the level of isolation we have already.

Bumping the baseline by 3 years to Ubuntu 16.04 would make both existing
and future OE-specific workarounds for ancient hosts unnecessary.

cu
Adrian


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Adrian Bunk
 

On Wed, Dec 11, 2019 at 08:48:04PM -0800, Andre McCurdy wrote:
On Wed, Dec 11, 2019 at 7:38 PM Adrian Bunk <bunk@...> wrote:

There is no need to question whether niche features like gcc or GNOME
for the target is a useful way of spending resources since the work is
done by the people interested in the feature.
Yes, that's probably expected. Developers work on stuff which
interests them. That's precisely why there's a danger that effort and
resources don't always get spent on the work which would benefit the
widest range of users.
This is not a danger, this is how open source projects work.

Yocto is not a company where all developers are paid resources that can
be told what to spend their time on.

Developers spend effort on what is useful for their local paid work,
or what is fun to do as a hobby.

...
In some cases this even creates new bugs for everyone else,
like the breakage caused by an incorrect CentOS 7 workaround in nettle
that needed a last-minute fixup before the release of Yocto 2.7.
That was an embarrassing mistake, which you've brought up before.
Not exactly embarrassing, the problem is obvious only in hindsight.

It happens sometimes. Dropping support for CentOS 7 or musl isn't going
to stop embarrassing mistakes happening again.
...
It avoids bugs being introduced by workarounds.

Trying to build and run 2020 software on a 2013 distribution is
problematic, and will generate more and more problems noone else
has seen or is interested in fixing.

Bumping the baseline by 3 years to Ubuntu 16.04 would make both existing
and future OE-specific workarounds for ancient hosts unnecessary.

cu
Adrian


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Andre McCurdy
 

On Wed, Dec 11, 2019 at 7:38 PM Adrian Bunk <bunk@...> wrote:
On Wed, Dec 11, 2019 at 06:31:30PM -0800, Andre McCurdy wrote:
On Wed, Dec 11, 2019 at 5:13 PM Denys Dmytriyenko <denis@...> wrote:
On Wed, Dec 11, 2019 at 01:30:30PM -0500, Philip Balister wrote:

From my point of view, the people that need CentOS 7 support need to
step up and do the work. The project can't allocate critical development
resources to support older distros.
I agree with Philip here - we should not be spending already scarce
engineering resources trying to shoehorn CentOS 7 support in.
We already spend scarce engineering resources on stuff like (just as
an example) gcc for the target, which is a far more "niche" use case
than CentOS 7 support.
It does matter a lot how the resulting work is distributed.

There is no need to question whether niche features like gcc or GNOME
for the target is a useful way of spending resources since the work is
done by the people interested in the feature.
Yes, that's probably expected. Developers work on stuff which
interests them. That's precisely why there's a danger that effort and
resources don't always get spent on the work which would benefit the
widest range of users.

Niche features like CentOS 7 support or musl place a burden on everyone
contributing to OE, since these are common cases where completely
unrelated contributions require extra work.
True. Continually updating all packages in oe-core and supporting
multiple CPU architecture places a burden on everyone contributing to
OE too. There's always a trade off between burdening developers and
making the project interesting / useful / usable for users.

In some cases this even creates new bugs for everyone else,
like the breakage caused by an incorrect CentOS 7 workaround in nettle
that needed a last-minute fixup before the release of Yocto 2.7.
That was an embarrassing mistake, which you've brought up before. It
happens sometimes. Dropping support for CentOS 7 or musl isn't going
to stop embarrassing mistakes happening again. The good news though is
that if mistakes are fixed relatively quickly most users are either
completely unaware that anything happened or they just update and move
on. ie I don't think mistakes like that cause any real damage at all.

You could also ask the question why do we have scarce engineering
resources? One factor is that OE is not user friendly and putting up
more barriers (ie stricter requirements on the host OS) isn't going to
help with that. Effort spent to increase the chances that OE "just
works" by automatically providing more of its dependencies and relying
less on the host may be a good investment in terms of alienating less
users, who may then persevere long enough to eventually become
contributors.
The problem is not lack of users, the problem is converting existing
users into upstream contributors.

Effort spent on getting more users is wasted effort if the goal is to
improve the scarce engineering resources.
If the goal is to increase the amount of engineering resource then
attracting / retaining users and converting them to contributors are
both important.

And existing contributors might stop contributing when a large part of
the contributing effort ends up having to be spent on niche features
without interest to the contributor.
That's true. It also works the other way around though - I've largely
given up contributing because the particular niche features which
interest me don't fit well upstream. Kicking niche features out of the
project can reduce contributions too.


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Adrian Bunk
 

On Wed, Dec 11, 2019 at 06:31:30PM -0800, Andre McCurdy wrote:
On Wed, Dec 11, 2019 at 5:13 PM Denys Dmytriyenko <denis@...> wrote:
On Wed, Dec 11, 2019 at 01:30:30PM -0500, Philip Balister wrote:

From my point of view, the people that need CentOS 7 support need to
step up and do the work. The project can't allocate critical development
resources to support older distros.
I agree with Philip here - we should not be spending already scarce
engineering resources trying to shoehorn CentOS 7 support in.
We already spend scarce engineering resources on stuff like (just as
an example) gcc for the target, which is a far more "niche" use case
than CentOS 7 support.
It does matter a lot how the resulting work is distributed.

There is no need to question whether niche features like gcc or GNOME
for the target is a useful way of spending resources since the work is
done by the people interested in the feature.

Niche features like CentOS 7 support or musl place a burden on everyone
contributing to OE, since these are common cases where completely
unrelated contributions require extra work.

In some cases this even creates new bugs for everyone else,
like the breakage caused by an incorrect CentOS 7 workaround in nettle
that needed a last-minute fixup before the release of Yocto 2.7.

You could also ask the question why do we have scarce engineering
resources? One factor is that OE is not user friendly and putting up
more barriers (ie stricter requirements on the host OS) isn't going to
help with that. Effort spent to increase the chances that OE "just
works" by automatically providing more of its dependencies and relying
less on the host may be a good investment in terms of alienating less
users, who may then persevere long enough to eventually become
contributors.
The problem is not lack of users, the problem is converting existing
users into upstream contributors.

Effort spent on getting more users is wasted effort if the goal is to
improve the scarce engineering resources.

And existing contributors might stop contributing when a large part of
the contributing effort ends up having to be spent on niche features
without interest to the contributor.

cu
Adrian


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Andre McCurdy
 

On Wed, Dec 11, 2019 at 5:13 PM Denys Dmytriyenko <denis@...> wrote:
On Wed, Dec 11, 2019 at 01:30:30PM -0500, Philip Balister wrote:
On 12/11/19 5:29 AM, Richard Purdie wrote:
On Wed, 2019-12-11 at 11:18 +0100, Andreas Müller wrote:
On Wed, Dec 11, 2019 at 8:15 AM Diego Santa Cruz via
Openembedded-architecture
<openembedded-architecture@...> wrote:
-----Original Message-----
From: openembedded-architecture-bounces@...
<openembedded-architecture-bounces@...> On
Behalf Of
Khem Raj
Sent: 10 December 2019 20:35
To: Randy MacLeod <randy.macleod@...>
Cc: openembedded-architecture <openembedded-
architecture@...>
Subject: Re: [Openembedded-architecture] Yocto support for
Centos-7 (RHEL-7):
drop in early 2020?

On Tue, Dec 10, 2019 at 9:23 AM Randy MacLeod
<randy.macleod@...> wrote:

In the YP technical meeting today, Richard suggested that we
stop
support for CentOS-7. Is there any objection to doing so before
3.1-M2?

CentOS-7 is just too old and there is significant work to
support it.
I am in support of it, but then I also fear that many corporate
policies might still
be using it until 2024 when security updates end so perhaps would
like to hear
centos7 users here. if no one speaks up then we can safely retire
it before 3.1
While I see what the motivation to remove CentOS-7 is, dropping it
will probably create issues for people. CentOS 8 was released not
so long ago. In our case we have older products (among which Yocto
based ones) which do not necessarily work on CentOS 8. CentOS 8 is
relatively recent, so we have not had time to test how old products
work on it.

Isn't it possible to require things like devtoolset-8 (gcc 8.3,
binutils-2.30) on CentOS 7 to keep it going?

In our case we are using devtoolset-8 with Yocto thud on CentOS 7
with success, as some layers require a recent host gcc to build.
How about extending uninative with more gcc bits?
uninative isn't the right place. What we'd need is a nativesdk-gcc
recipe and then to add nativesdk-gcc to buildtools-tarball.

We could then add some mechanism to auto-install buildtools tarball up
front on systems that need it, in a similar way to what uninative does.

Definitely possible and would solve certain problems, we'd just need
someone to implement it...
From my point of view, the people that need CentOS 7 support need to
step up and do the work. The project can't allocate critical development
resources to support older distros.
I agree with Philip here - we should not be spending already scarce
engineering resources trying to shoehorn CentOS 7 support in.
We already spend scarce engineering resources on stuff like (just as
an example) gcc for the target, which is a far more "niche" use case
than CentOS 7 support.

You could also ask the question why do we have scarce engineering
resources? One factor is that OE is not user friendly and putting up
more barriers (ie stricter requirements on the host OS) isn't going to
help with that. Effort spent to increase the chances that OE "just
works" by automatically providing more of its dependencies and relying
less on the host may be a good investment in terms of alienating less
users, who may then persevere long enough to eventually become
contributors.

I realize this is mostly about OE-Core, but on a related note, I very
recently had to install a more modern gcc8 compiler on Ubuntu 16.04 on some
of our build servers, as 16.04 was lacking recent C++ standards support
in its default 5.4 compiler. The point is that we cannot keep on patching
out every use of newer API to make it work with old OSes...

--
Denys
_______________________________________________
Openembedded-architecture mailing list
Openembedded-architecture@...
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Denys Dmytriyenko
 

On Wed, Dec 11, 2019 at 01:30:30PM -0500, Philip Balister wrote:
On 12/11/19 5:29 AM, Richard Purdie wrote:
On Wed, 2019-12-11 at 11:18 +0100, Andreas Müller wrote:
On Wed, Dec 11, 2019 at 8:15 AM Diego Santa Cruz via
Openembedded-architecture
<openembedded-architecture@...> wrote:
-----Original Message-----
From: openembedded-architecture-bounces@...
<openembedded-architecture-bounces@...> On
Behalf Of
Khem Raj
Sent: 10 December 2019 20:35
To: Randy MacLeod <randy.macleod@...>
Cc: openembedded-architecture <openembedded-
architecture@...>
Subject: Re: [Openembedded-architecture] Yocto support for
Centos-7 (RHEL-7):
drop in early 2020?

On Tue, Dec 10, 2019 at 9:23 AM Randy MacLeod
<randy.macleod@...> wrote:

In the YP technical meeting today, Richard suggested that we
stop
support for CentOS-7. Is there any objection to doing so before
3.1-M2?

CentOS-7 is just too old and there is significant work to
support it.
I am in support of it, but then I also fear that many corporate
policies might still
be using it until 2024 when security updates end so perhaps would
like to hear
centos7 users here. if no one speaks up then we can safely retire
it before 3.1
While I see what the motivation to remove CentOS-7 is, dropping it
will probably create issues for people. CentOS 8 was released not
so long ago. In our case we have older products (among which Yocto
based ones) which do not necessarily work on CentOS 8. CentOS 8 is
relatively recent, so we have not had time to test how old products
work on it.

Isn't it possible to require things like devtoolset-8 (gcc 8.3,
binutils-2.30) on CentOS 7 to keep it going?

In our case we are using devtoolset-8 with Yocto thud on CentOS 7
with success, as some layers require a recent host gcc to build.
How about extending uninative with more gcc bits?
uninative isn't the right place. What we'd need is a nativesdk-gcc
recipe and then to add nativesdk-gcc to buildtools-tarball.

We could then add some mechanism to auto-install buildtools tarball up
front on systems that need it, in a similar way to what uninative does.

Definitely possible and would solve certain problems, we'd just need
someone to implement it...
From my point of view, the people that need CentOS 7 support need to
step up and do the work. The project can't allocate critical development
resources to support older distros.
I agree with Philip here - we should not be spending already scarce
engineering resources trying to shoehorn CentOS 7 support in.

I realize this is mostly about OE-Core, but on a related note, I very
recently had to install a more modern gcc8 compiler on Ubuntu 16.04 on some
of our build servers, as 16.04 was lacking recent C++ standards support
in its default 5.4 compiler. The point is that we cannot keep on patching
out every use of newer API to make it work with old OSes...

--
Denys


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Andre McCurdy
 

On Wed, Dec 11, 2019 at 4:14 AM Adrian Bunk <bunk@...> wrote:
On Wed, Dec 11, 2019 at 10:29:27AM +0000, Richard Purdie wrote:
On Wed, 2019-12-11 at 11:18 +0100, Andreas Müller wrote:
On Wed, Dec 11, 2019 at 8:15 AM Diego Santa Cruz via
Openembedded-architecture
<openembedded-architecture@...> wrote:
While I see what the motivation to remove CentOS-7 is, dropping it
will probably create issues for people. CentOS 8 was released not
so long ago. In our case we have older products (among which Yocto
based ones) which do not necessarily work on CentOS 8. CentOS 8 is
relatively recent, so we have not had time to test how old products
work on it.

Isn't it possible to require things like devtoolset-8 (gcc 8.3,
binutils-2.30) on CentOS 7 to keep it going?

In our case we are using devtoolset-8 with Yocto thud on CentOS 7
with success, as some layers require a recent host gcc to build.
How about extending uninative with more gcc bits?
uninative isn't the right place. What we'd need is a nativesdk-gcc
recipe and then to add nativesdk-gcc to buildtools-tarball.

We could then add some mechanism to auto-install buildtools tarball up
front on systems that need it, in a similar way to what uninative does.

Definitely possible and would solve certain problems, we'd just need
someone to implement it...
What you describe sounds like a manual and painful way to create some
kind of container - once you have toolchain/chrpath/tar/... of the host
replaced there is not much left that is actually used from the host.

The simple and sane solution would be to provide or require a container
with a supported distribution instead.
The downside of that is that it causes problems for anyone who is
already building in a container, e.g. anyone using a Docker image to
run Jenkins, etc.

https://hub.docker.com/r/jenkins/jenkins/


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Andreas Müller
 

On Wed, Dec 11, 2019 at 11:29 AM Richard Purdie
<richard.purdie@...> wrote:
How about extending uninative with more gcc bits?
uninative isn't the right place. What we'd need is a nativesdk-gcc
recipe and then to add nativesdk-gcc to buildtools-tarball.
Maybe - but from user's perspective it is perfect: Enter a line in
local.conf and let it happen. No manual download/install / container
setup.

Did that to make our very old build server work with zeus images. We
have to keep it old because there are products sitting on VERY old
releases (I think it was daisy). And yes I know that this completely
irresponsible but we are a 'team' of two for the whole stack
(applications inclusive) - but that is another story...

Andreas

We could then add some mechanism to auto-install buildtools tarball up
front on systems that need it, in a similar way to what uninative does.

Definitely possible and would solve certain problems, we'd just need
someone to implement it...

Cheers,

Richard


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Philip Balister
 

On 12/11/19 5:29 AM, Richard Purdie wrote:
On Wed, 2019-12-11 at 11:18 +0100, Andreas Müller wrote:
On Wed, Dec 11, 2019 at 8:15 AM Diego Santa Cruz via
Openembedded-architecture
<openembedded-architecture@...> wrote:
-----Original Message-----
From: openembedded-architecture-bounces@...
<openembedded-architecture-bounces@...> On
Behalf Of
Khem Raj
Sent: 10 December 2019 20:35
To: Randy MacLeod <randy.macleod@...>
Cc: openembedded-architecture <openembedded-
architecture@...>
Subject: Re: [Openembedded-architecture] Yocto support for
Centos-7 (RHEL-7):
drop in early 2020?

On Tue, Dec 10, 2019 at 9:23 AM Randy MacLeod
<randy.macleod@...> wrote:

In the YP technical meeting today, Richard suggested that we
stop
support for CentOS-7. Is there any objection to doing so before
3.1-M2?

CentOS-7 is just too old and there is significant work to
support it.
I am in support of it, but then I also fear that many corporate
policies might still
be using it until 2024 when security updates end so perhaps would
like to hear
centos7 users here. if no one speaks up then we can safely retire
it before 3.1
While I see what the motivation to remove CentOS-7 is, dropping it
will probably create issues for people. CentOS 8 was released not
so long ago. In our case we have older products (among which Yocto
based ones) which do not necessarily work on CentOS 8. CentOS 8 is
relatively recent, so we have not had time to test how old products
work on it.

Isn't it possible to require things like devtoolset-8 (gcc 8.3,
binutils-2.30) on CentOS 7 to keep it going?

In our case we are using devtoolset-8 with Yocto thud on CentOS 7
with success, as some layers require a recent host gcc to build.
How about extending uninative with more gcc bits?
uninative isn't the right place. What we'd need is a nativesdk-gcc
recipe and then to add nativesdk-gcc to buildtools-tarball.

We could then add some mechanism to auto-install buildtools tarball up
front on systems that need it, in a similar way to what uninative does.

Definitely possible and would solve certain problems, we'd just need
someone to implement it...
From my point of view, the people that need CentOS 7 support need to
step up and do the work. The project can't allocate critical development
resources to support older distros.

Philip


Cheers,

Richard


_______________________________________________
Openembedded-architecture mailing list
Openembedded-architecture@...
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Alexander Kanavin
 

On Wed, 11 Dec 2019 at 13:58, Adrian Bunk <bunk@...> wrote:
>...
> Creating containers, whilst easier than it ever used to be, does
> usually require elevated permissions and can cause problems with things
> like qemu acceleration and networking. If you've already dealt with
> that, fine but some setups wouldn't have.
>...

Status quo is that qemu might be broken when the host has a more recent
libdrm than the Yocto release to be built.

Like the qemu virgl issues with warrior on Fedora 30.

I wouldn't say that qemu is broken when virgl isn't working. Virgl support is entirely optional, and you have to explicitly request it by passing an option to runqemu. It's also not working on some very old distros like centos 7, but rather than take them off the supported distro list, we simply skip the automated test there, as there is no hope to make it work.

Ideas on how to better decouple virgl support from the host GL stack including libdrm (without drastically increasing build times) are welcome.

Alex


Re: Yocto support for Centos-7 (RHEL-7): drop in early 2020?

Adrian Bunk
 

On Wed, Dec 11, 2019 at 12:33:45PM +0000, Richard Purdie wrote:
...
Creating containers, whilst easier than it ever used to be, does
usually require elevated permissions and can cause problems with things
like qemu acceleration and networking. If you've already dealt with
that, fine but some setups wouldn't have.
...
Status quo is that qemu might be broken when the host has a more recent
libdrm than the Yocto release to be built.

Like the qemu virgl issues with warrior on Fedora 30.

Cheers,

Richard
cu
Adrian

701 - 720 of 1685