Can we change bitbake fetcher to default to https instead git
anonymous protocol as fallback? this will be good security measure
too.

On Tue, Nov 2, 2021 at 5:46 AM Richard Purdie
<richard.purdie@...> wrote:

On Tue, 2021-11-02 at 12:32 +0000, Richard Purdie via lists.openembedded.org
wrote:

On Tue, 2021-11-02 at 11:56 +0000, Andrei Gherzan wrote:

On Tue, 2 Nov 2021, at 11:52, Martin Jansa wrote:

On Tue, Nov 2, 2021 at 12:46 PM Richard Purdie
<richard.purdie@...> wrote:

On Tue, 2021-11-02 at 11:32 +0100, Martin Jansa wrote:
> There is even bigger issue with git repos from github.com now:
>
>
https://github.blog/2021-09-01-improving-git-protocol-security-github/#no-more-unauthenticated-git
>
> bitbake git fetcher uses git:// protocol by default and as of today you
can
> experience "short brownouts" and on January 11 it will all fail to
fetch (and
> only fully populated PREMIRRORS can save you for a while, until SRCREV
is
> updated).
>
> Short statistics from current oe-core/master:
> martin@jama:/OE/openembedded-core$ git grep git://github.* | grep -v
protocol=
> | wc -l
> 52
> martin@jama:/OE/openembedded-core$ git grep
git://github.*protocol=https | wc
> -l
> 20
> martin@jama:/OE/openembedded-core$ git grep git://github.*protocol=git

wc -l

> 2
>
> 54 from 74 recipes will fail to fetch in oe-core only.

Thanks for reporting this, it helps to know this is happening as we'll
probably
start seeing odd error reports for the brownouts.

The brownouts are already happening, got 20+ failed jenkins jobs over night,
because they failed to fetch various metadata layers over git:// from
github. And hopefully my understanding of the announcement is correct and
git:// brownouts are planned only for today.

I've updated the conversion script I mentioned earlier in this thread to
handle
remapping the github.com urls too and also fixed the few corner cases I
found
after the first conversion. I've sent those patches to OE-Core.

Thanks!, looks good to me.

For the older releases, rather than trying to rewrite all the urls, I
think we
may want to patch bitbake to correctly handle the github urls
specifically.

Considering how many people I've seen complaining about new overrides syntax
breaking their just updated oe-core/dunfell build, just because they don't
update bitbake revision it might be safer to do both (so that at least the
maintained layers get the explicit protocol=https in SRC_URIs and the not-
so-well-maintained layers could be saved by git fetcher changing the
protocol automagically).

I totally agree with that. I still think we should also warn out so we don't
have to maintain this magic quirk forever.

I think we put a warning on master and forwards but not older bitbakes.

I've sent out a couple of patches for bitbake, one which does the remapping and
a second which adds the warning. Testing would be appreciated before I merge
them (I need to focus on master first).

Cheers,

Richard