On Thu, 2022-03-31 at 15:45 +0000, Peter Kjellerstedt wrote:

-----Original Message-----
From: Richard Purdie <richard.purdie@...>
Sent: den 31 mars 2022 16:16
To: Peter Kjellerstedt <peter.kjellerstedt@...>; openembedded-
core@...
Subject: Re: [OE-core] [PATCH RFC] mirrors: Switch glibc and binutils to
use shallow mirror tarballs

On Thu, 2022-03-31 at 14:09 +0000, Peter Kjellerstedt wrote:

-----Original Message-----
From: openembedded-core@...
<openembedded-core@...> On Behalf Of Richard Purdie
Sent: den 30 mars 2022 14:24
To: openembedded-core@...
Subject: [OE-core] [PATCH RFC] mirrors: Switch glibc and binutils to use
shallow mirror tarballs

These two repositories are large and overload our downloads server as a
premirror but the recipes are easier to maintain as git urls. Compromise
and use shallow clones for them.

In order to be effective, we need premirror entries on where to find
the shallow mirror tarballs.

Signed-off-by: Richard Purdie <richard.purdie@...>
---
 meta/classes/mirrors.bbclass | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/meta/classes/mirrors.bbclass b/meta/classes/mirrors.bbclass
index 37dc449ade7..ffdccff5fb4 100644
--- a/meta/classes/mirrors.bbclass
+++ b/meta/classes/mirrors.bbclass
@@ -76,3 +76,14 @@ git://git.gnome.org/.*
git://gitlab.gnome.org/GNOME/PATH;protocol=https \
 git://.*/.* git://HOST/PATH;protocol=https \
 git://.*/.* git://HOST/git/PATH;protocol=https \
 "
+
+# Switch glibc and binutils recipes to use shallow clones as they're
large and this
+# improves user experience whilst allowing the flexibility of git urls
in the recipes
+BB_GIT_SHALLOW:pn-binutils = "1"
+BB_GIT_SHALLOW:pn-binutils-cross-${TARGET_ARCH} = "1"
+BB_GIT_SHALLOW:pn-binutils-cross-canadian-${TRANSLATED_TARGET_ARCH} =
"1"
+BB_GIT_SHALLOW:pn-binutils-cross-testsuite = "1"
+BB_GIT_SHALLOW:pn-binutils-crosssdk-${SDK_SYS} = "1"
+BB_GIT_SHALLOW:pn-glibc = "1"
+PREMIRRORS += "git://sourceware.org/git/glibc.git
https://downloads.yoctoproject.org/mirror/sources/ \
+ git://sourceware.org/git/binutils-gdb.git
https://downloads.yoctoproject.org/mirror/sources/"

Please use PREMIRRORS:append here, or you will wipe away any
default configuration of PREMIRRORS defined using "?=".

No, I'm not using append here since I do want users to be able to reset
this.

That can easily be achieved by instead doing something like:

SHALLOW_PREMIRRORS = " \
    git://sourceware.org/git/glibc.git https://downloads.yoctoproject.org/mirror/sources/ \
    git://sourceware.org/git/binutils-gdb.git https://downloads.yoctoproject.org/mirror/sources/ \
"
PREMIRRORS:append = " ${SHALLOW_PREMIRRORS}"

This implies that any variable alteration should always use indirection via
another variable. I am not going to support moves in that direction as it is
horrible. If our syntax is causing issues we need to resolve that but I'm not
going to compound it with moves like the above, sorry.

I'd also note this matches how MIRRORS is being handled.

which I think is a far better solution anyway since it is very
hard to reset something in PREMIRRORS as one cannot use :remove.
With this solution it is possible to set SHALLOW_PREMIRRORS = ""
in case one do not want the mirrors OE-Core adds by default.

Otherwise you are effectively saying that one can no longer define
a default for PREMIRRORS using ?=, which was possible up until the
above change. And unless I can persuade you to switch to :append,
then this needs to be mentioned in the migration notes for 4.0.

Lets release note it then.

Cheers,

Richard