Re: Default branch names in git urls


Martin Jansa
 

On Tue, Nov 2, 2021 at 12:46 PM Richard Purdie <richard.purdie@...> wrote:
On Tue, 2021-11-02 at 11:32 +0100, Martin Jansa wrote:
> There is even bigger issue with git repos from github.com now:
>
> https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git
>
> bitbake git fetcher uses git:// protocol by default and as of today you can
> experience "short brownouts" and on January 11 it will all fail to fetch (and
> only fully populated PREMIRRORS can save you for a while, until SRCREV is
> updated).
>
> Short statistics from current oe-core/master:
> martin@jama:/OE/openembedded-core$ git grep git://github.* | grep -v protocol=
> | wc -l
> 52
> martin@jama:/OE/openembedded-core$ git grep git://github.*protocol=https | wc
> -l
> 20
> martin@jama:/OE/openembedded-core$ git grep git://github.*protocol=git | wc -l
> 2
>
> 54 from 74 recipes will fail to fetch in oe-core only.

Thanks for reporting this, it helps to know this is happening as we'll probably
start seeing odd error reports for the brownouts.

The brownouts are already happening, got 20+ failed jenkins jobs over night, because they failed to fetch various metadata layers over git:// from github. And hopefully my understanding of the announcement is correct and git:// brownouts are planned only for today.
 
I've updated the conversion script I mentioned earlier in this thread to handle
remapping the github.com urls too and also fixed the few corner cases I found
after the first conversion. I've sent those patches to OE-Core.

Thanks!, looks good to me.

For the older releases, rather than trying to rewrite all the urls, I think we
may want to patch bitbake to correctly handle the github urls specifically.

Considering how many people I've seen complaining about new overrides syntax breaking their just updated oe-core/dunfell build, just because they don't update bitbake revision it might be safer to do both (so that at least the maintained layers get the explicit protocol=https in SRC_URIs and the not-so-well-maintained layers could be saved by git fetcher changing the protocol automagically).
 
Regards,

Join {openembedded-architecture@lists.openembedded.org to automatically receive all group messages.