[PATCH 1/3] go: drop patch to make content based hash generation less pedan


Jose Quaresma
 

With patch in place I have see some build issues mainly in meta-virtualization
with ti am62xx-evm, am64xx-evm and riscv64 machines.

All the mahines fails to build the following recipes:

| Summary: 5 tasks failed:
| /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/runc/runc-opencontainers_git.bb:do_compile
| /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/skopeo/skopeo_git.bb:do_compile
| /srv/oe/build/conf/../../layers/meta-lmp/meta-lmp-base/recipes-support/fioconfig/fioconfig_git.bb:do_compile
| /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/docker/docker-ce_git.bb:do_compile
| /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/containerd/containerd-opencontainers_git.bb:do_compile
| Summary: There were 8 ERROR messages, returning a non-zero exit code.

And all of them fails with the same issue:

| panic: runtime error: index out of range [0] with length 0
|
| goroutine 6688 [running]:
| cmd/go/internal/work.(*Builder).buildActionID(0xc0000030e0, 0xc000172a00)
| /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:281 +0x2425
| cmd/go/internal/work.(*Builder).build(0xc0000030e0, {0xc000000238?, 0x0?}, 0xc000172a00)
| /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:478 +0x199
| cmd/go/internal/work.(*Builder).Do.func2({0xb2e4f0, 0xc000000238}, 0xc000172a00)
| /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:139 +0x5fc
| cmd/go/internal/work.(*Builder).Do.func3()
| /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:201 +0xb9
| created by cmd/go/internal/work.(*Builder).Do
| /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:187 +0x3f5
| make: *** [Makefile:244: bin/containerd] Error 2
| make: *** Waiting for unfinished jobs....

Without this patch everithing look good.

Signed-off-by: Jose Quaresma <jose.quaresma@...>
---
meta/recipes-devtools/go/go-1.19.inc | 1 -
1 file changed, 1 deletion(-)

diff --git a/meta/recipes-devtools/go/go-1.19.inc b/meta/recipes-devtools/go/go-1.19.inc
index f733a807b4..355a82eb9f 100644
--- a/meta/recipes-devtools/go/go-1.19.inc
+++ b/meta/recipes-devtools/go/go-1.19.inc
@@ -5,7 +5,6 @@ FILESEXTRAPATHS:prepend := "${FILE_DIRNAME}/go:"
LIC_FILES_CHKSUM = "file://LICENSE;md5=5d4950ecb7b26d2c5e4e7b4e0dd74707"

SRC_URI += "\
- file://0001-cmd-go-make-content-based-hash-generation-less-pedan.patch \
file://0003-allow-GOTOOLDIR-to-be-overridden-in-the-environment.patch \
file://0004-ld-add-soname-to-shareable-objects.patch \
file://0005-make.bash-override-CC-when-building-dist-and-go_boot.patch \
--
2.34.1


Alexander Kanavin
 

You need to remove the actual patch, and run the reproducibility test.

Alex

On Fri, 30 Sept 2022 at 19:55, Jose Quaresma <quaresma.jose@...> wrote:

With patch in place I have see some build issues mainly in meta-virtualization
with ti am62xx-evm, am64xx-evm and riscv64 machines.

All the mahines fails to build the following recipes:

| Summary: 5 tasks failed:
| /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/runc/runc-opencontainers_git.bb:do_compile
| /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/skopeo/skopeo_git.bb:do_compile
| /srv/oe/build/conf/../../layers/meta-lmp/meta-lmp-base/recipes-support/fioconfig/fioconfig_git.bb:do_compile
| /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/docker/docker-ce_git.bb:do_compile
| /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/containerd/containerd-opencontainers_git.bb:do_compile
| Summary: There were 8 ERROR messages, returning a non-zero exit code.

And all of them fails with the same issue:

| panic: runtime error: index out of range [0] with length 0
|
| goroutine 6688 [running]:
| cmd/go/internal/work.(*Builder).buildActionID(0xc0000030e0, 0xc000172a00)
| /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:281 +0x2425
| cmd/go/internal/work.(*Builder).build(0xc0000030e0, {0xc000000238?, 0x0?}, 0xc000172a00)
| /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:478 +0x199
| cmd/go/internal/work.(*Builder).Do.func2({0xb2e4f0, 0xc000000238}, 0xc000172a00)
| /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:139 +0x5fc
| cmd/go/internal/work.(*Builder).Do.func3()
| /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:201 +0xb9
| created by cmd/go/internal/work.(*Builder).Do
| /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:187 +0x3f5
| make: *** [Makefile:244: bin/containerd] Error 2
| make: *** Waiting for unfinished jobs....

Without this patch everithing look good.

Signed-off-by: Jose Quaresma <jose.quaresma@...>
---
meta/recipes-devtools/go/go-1.19.inc | 1 -
1 file changed, 1 deletion(-)

diff --git a/meta/recipes-devtools/go/go-1.19.inc b/meta/recipes-devtools/go/go-1.19.inc
index f733a807b4..355a82eb9f 100644
--- a/meta/recipes-devtools/go/go-1.19.inc
+++ b/meta/recipes-devtools/go/go-1.19.inc
@@ -5,7 +5,6 @@ FILESEXTRAPATHS:prepend := "${FILE_DIRNAME}/go:"
LIC_FILES_CHKSUM = "file://LICENSE;md5=5d4950ecb7b26d2c5e4e7b4e0dd74707"

SRC_URI += "\
- file://0001-cmd-go-make-content-based-hash-generation-less-pedan.patch \
file://0003-allow-GOTOOLDIR-to-be-overridden-in-the-environment.patch \
file://0004-ld-add-soname-to-shareable-objects.patch \
file://0005-make.bash-override-CC-when-building-dist-and-go_boot.patch \
--
2.34.1




Jose Quaresma
 

Hi Alex,

Alexander Kanavin <alex.kanavin@...> escreveu no dia sexta, 30/09/2022 à(s) 18:57:
You need to remove the actual patch, and run the reproducibility test.

Forgot to remove the patch! Will send a v2 but I can't run the reproducibility now.

Jose


Alex

On Fri, 30 Sept 2022 at 19:55, Jose Quaresma <quaresma.jose@...> wrote:
>
> With patch in place I have see some build issues mainly in meta-virtualization
> with ti am62xx-evm, am64xx-evm and riscv64 machines.
>
> All the mahines fails to build the following recipes:
>
> | Summary: 5 tasks failed:
> |  /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/runc/runc-opencontainers_git.bb:do_compile
> |  /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/skopeo/skopeo_git.bb:do_compile
> |  /srv/oe/build/conf/../../layers/meta-lmp/meta-lmp-base/recipes-support/fioconfig/fioconfig_git.bb:do_compile
> |  /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/docker/docker-ce_git.bb:do_compile
> |  /srv/oe/build/conf/../../layers/meta-virtualization/recipes-containers/containerd/containerd-opencontainers_git.bb:do_compile
> | Summary: There were 8 ERROR messages, returning a non-zero exit code.
>
> And all of them fails with the same issue:
>
> | panic: runtime error: index out of range [0] with length 0
> |
> | goroutine 6688 [running]:
> | cmd/go/internal/work.(*Builder).buildActionID(0xc0000030e0, 0xc000172a00)
> |       /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:281 +0x2425
> | cmd/go/internal/work.(*Builder).build(0xc0000030e0, {0xc000000238?, 0x0?}, 0xc000172a00)
> |       /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:478 +0x199
> | cmd/go/internal/work.(*Builder).Do.func2({0xb2e4f0, 0xc000000238}, 0xc000172a00)
> |       /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:139 +0x5fc
> | cmd/go/internal/work.(*Builder).Do.func3()
> |       /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:201 +0xb9
> | created by cmd/go/internal/work.(*Builder).Do
> |       /srv/oe/build/tmp-lmp/work/x86_64-linux/go-cross-riscv64/1.19-r0/recipe-sysroot-native/usr/lib/riscv64-lmp-linux/go/src/cmd/go/internal/work/exec.go:187 +0x3f5
> | make: *** [Makefile:244: bin/containerd] Error 2
> | make: *** Waiting for unfinished jobs....
>
> Without this patch everithing look good.
>
> Signed-off-by: Jose Quaresma <jose.quaresma@...>
> ---
>  meta/recipes-devtools/go/go-1.19.inc | 1 -
>  1 file changed, 1 deletion(-)
>
> diff --git a/meta/recipes-devtools/go/go-1.19.inc b/meta/recipes-devtools/go/go-1.19.inc
> index f733a807b4..355a82eb9f 100644
> --- a/meta/recipes-devtools/go/go-1.19.inc
> +++ b/meta/recipes-devtools/go/go-1.19.inc
> @@ -5,7 +5,6 @@ FILESEXTRAPATHS:prepend := "${FILE_DIRNAME}/go:"
>  LIC_FILES_CHKSUM = "file://LICENSE;md5=5d4950ecb7b26d2c5e4e7b4e0dd74707"
>
>  SRC_URI += "\
> -    file://0001-cmd-go-make-content-based-hash-generation-less-pedan.patch \
>      file://0003-allow-GOTOOLDIR-to-be-overridden-in-the-environment.patch \
>      file://0004-ld-add-soname-to-shareable-objects.patch \
>      file://0005-make.bash-override-CC-when-building-dist-and-go_boot.patch \
> --
> 2.34.1
>
>
>
>


--
Best regards,

José Quaresma


Alexander Kanavin
 

On Fri, 30 Sept 2022 at 20:01, Jose Quaresma <quaresma.jose@...> wrote:
Forgot to remove the patch! Will send a v2 but I can't run the reproducibility now.
You should, I believe the patch was added in the first place to fix
the reproducibility issues, and won't be accepted if its removal
breaks that.

Alex


Jose Quaresma
 



Alexander Kanavin <alex.kanavin@...> escreveu no dia sexta, 30/09/2022 à(s) 19:02:
On Fri, 30 Sept 2022 at 20:01, Jose Quaresma <quaresma.jose@...> wrote:
> Forgot to remove the patch! Will send a v2 but I can't run the reproducibility now.

You should, I believe the patch was added in the first place to fix
the reproducibility issues, and won't be accepted if its removal
breaks that.

Sure, I can do it but only next week.
I tried to understand and come up with a way to fix the issue but don't understand the real problem behind it.

Jose


Alex


--
Best regards,

José Quaresma


Jose Quaresma
 



Jose Quaresma via lists.openembedded.org <quaresma.jose=gmail.com@...> escreveu no dia sexta, 30/09/2022 à(s) 19:10:


Alexander Kanavin <alex.kanavin@...> escreveu no dia sexta, 30/09/2022 à(s) 19:02:
On Fri, 30 Sept 2022 at 20:01, Jose Quaresma <quaresma.jose@...> wrote:
> Forgot to remove the patch! Will send a v2 but I can't run the reproducibility now.

You should, I believe the patch was added in the first place to fix
the reproducibility issues, and won't be accepted if its removal
breaks that.

Sure, I can do it but only next week.
I tried to understand and come up with a way to fix the issue but don't understand the real problem behind it.

This patch fixes some reproducibility issues as referred by Alex so with the removal it will come again.
I attach the diffoscope output (diff-html.tar.zst) of the reproducibility test if anyone is interested.

The affect content is on go-runtime-dev and go-runtime-staticdev:

reproducibleB/tmp/deploy/ipk/core2-64/go-runtime-dev_1.19-r0_core2-64.ipk
./usr/lib/go/pkg/linux_amd64_dynlink/net.a
./usr/lib/go/pkg/linux_amd64_dynlink/os/signal/internal/pty.a
./usr/lib/go/pkg/linux_amd64_dynlink/os/user.a
./usr/lib/go/pkg/linux_amd64_dynlink/plugin.a
./usr/lib/go/pkg/linux_amd64_dynlink/runtime/cgo.a

reproducibleB/tmp/deploy/ipk/core2-64/go-runtime-staticdev_1.19-r0_core2-64.ipk
./usr/lib/go/pkg/linux_amd64/net.a
./usr/lib/go/pkg/linux_amd64/os/signal/internal/pty.a
./usr/lib/go/pkg/linux_amd64/os/user.a
./usr/lib/go/pkg/linux_amd64/plugin.a
./usr/lib/go/pkg/linux_amd64/runtime/cgo.a

So I think we can't remove that patch as it breaks the reproducibility.
Maybe it is better to open a bug because the go is broken in some machines.

I can try to understand better what's going on and try to solve the problem but it's a little out of my knowledge,
so any help will be much appreciated.

Jose
 

Jose


Alex


--
Best regards,

José Quaresma





--
Best regards,

José Quaresma