On Tue, 2 Nov 2021, at 11:52, Martin Jansa wrote:

On Tue, Nov 2, 2021 at 12:46 PM Richard Purdie
<richard.purdie@...> wrote:

On Tue, 2021-11-02 at 11:32 +0100, Martin Jansa wrote:
 > There is even bigger issue with git repos from github.com now:
 >
 >
https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git
 >
 > bitbake git fetcher uses git:// protocol by default and as of today you
can
 > experience "short brownouts" and on January 11 it will all fail to
fetch (and
 > only fully populated PREMIRRORS can save you for a while, until SRCREV
is
 > updated).
 >
 > Short statistics from current oe-core/master:
 > martin@jama:/OE/openembedded-core$ git grep git://github.* | grep -v
protocol=
 > | wc -l
 > 52
 > martin@jama:/OE/openembedded-core$ git grep
git://github.*protocol=https | wc
 > -l
 > 20
 > martin@jama:/OE/openembedded-core$ git grep git://github.*protocol=git
| wc -l
 > 2
 >
 > 54 from 74 recipes will fail to fetch in oe-core only.
 
 Thanks for reporting this, it helps to know this is happening as we'll
probably
 start seeing odd error reports for the brownouts.

The brownouts are already happening, got 20+ failed jenkins jobs over night,
because they failed to fetch various metadata layers over git:// from
github. And hopefully my understanding of the announcement is correct and
git:// brownouts are planned only for today.
 

I've updated the conversion script I mentioned earlier in this thread to
handle
 remapping the github.com urls too and also fixed the few corner cases I
found
 after the first conversion. I've sent those patches to OE-Core.

Thanks!, looks good to me.

For the older releases, rather than trying to rewrite all the urls, I
think we
 may want to patch bitbake to correctly handle the github urls
specifically.

Considering how many people I've seen complaining about new overrides syntax
breaking their just updated oe-core/dunfell build, just because they don't
update bitbake revision it might be safer to do both (so that at least the
maintained layers get the explicit protocol=https in SRC_URIs and the not-
so-well-maintained layers could be saved by git fetcher changing the
protocol automagically).

I totally agree with that. I still think we should also warn out so we don't
have to maintain this magic quirk forever.