[PATCH] selftest: incompatible_lic.py: run bitbake -c testimage separately for each image


Martin Jansa
 

* with 1 preconfigured tap device you cannot run testimage on multiple images
at the same time

in my case have /tmp/qemu-tap-locks/tap0.skip (tap0 is used by openvpn on builder)
and /etc/runqemu-nosudo as my build user doesn't have sudo powers

but then when 2 do_testimage tasks are executed at the same time it fails like this:

NOTE: recipe core-image-weston-1.0-r0: task do_testimage: Started
NOTE: recipe core-image-full-cmdline-1.0-r0: task do_testimage: Started
QMP Available for connection at /OE/build/poky/build-st/tmp/.c2d3k3bh
QMP Available for connection at /OE/build/poky/build-st/tmp/._4agpi1v
ERROR: core-image-full-cmdline-1.0-r0 do_testimage: runqemu exited with code 1
ERROR: core-image-full-cmdline-1.0-r0 do_testimage: Output from runqemu:
runqemu - INFO - Continuing with the following parameters:
KERNEL: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/bzImage]
MACHINE: [qemux86-64]
FSTYPE: [ext4]
ROOTFS: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/core-image-full-cmdline-qemux86-64.ext4]
CONFFILE: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/core-image-full-cmdline-qemux86-64.qemuboot.conf]

runqemu - INFO - Found /tmp/qemu-tap-locks/tap0.skip, skipping tap0
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap1.lock failed: [Errno 11] Resource temporarily unavailable
runqemu - ERROR - Error: There are no available tap devices to use for networking,
runqemu - ERROR - and I see /etc/runqemu-nosudo exists, so I am not going to try creating
runqemu - ERROR - a new one with sudo.
runqemu - INFO - Cleaning up
runqemu - INFO - Host uptime: 67474.59

tput: No value for $TERM and no -T specified

WARNING: core-image-full-cmdline-1.0-r0 do_testimage: Qemu ended unexpectedly, dump data from host is in /OE/build/poky/build-st/tmp/log/runtime-hostdump/202303131614_qemu
WARNING: core-image-full-cmdline-1.0-r0 do_testimage: runqemu during is_alive() test
ERROR: core-image-full-cmdline-1.0-r0 do_testimage: Error executing a python function in exec_func_python() autogenerated:

which is very reasonable error message, but 600 lines later it also shows:

RESULTS - xorg.XorgTest.test_xorg_running: SKIPPED (0.00s)
SUMMARY:
core-image-weston () - Ran 71 tests in 34.073s
core-image-weston - OK - All required tests passed (successes=14, skipped=57, failures=0, errors=0)
NOTE: recipe core-image-weston-1.0-r0: task do_testimage: Succeeded
NOTE: Tasks Summary: Attempted 1141 tasks of which 1139 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
/OE/build/poky/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimag

and I didn't notice that there are actually 2 different images core-image-weston
and core-image-full-cmdline and spent most time trying to figure out how it was
able to run all those ptest when qemu failed to start at the beginning.

It's possible to create 2nd tap with runqemu-ifup before running this test.
Then it works, maybe some check like in DevtoolTests to error out early if
there isn't enough tap devices? But the implementation in:
https://git.openembedded.org/openembedded-core/commit/?id=2258345e19efff7717fe19a5026ec55f1b6f90b6
has its own limitations (e.g. not checking /tmp/qemu-tap-locks/tap0.skip)
and would probably need to be re-implemented as some generic decorator
as other tests need this as well.

It's a bit slower when serialized:
INFO - RESULTS - incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: PASSED (145.96s)
without this change:
INFO - RESULTS - incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: PASSED (104.72s)

I don't mind if this is rejected, sending it mostly to document this behavior.

Signed-off-by: Martin Jansa <Martin.Jansa@...>
---
meta/lib/oeqa/selftest/cases/incompatible_lic.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/cases/incompatible_lic.py b/meta/lib/oeqa/selftest/cases/incompatible_lic.py
index 4edf60fc55..b2da25e612 100644
--- a/meta/lib/oeqa/selftest/cases/incompatible_lic.py
+++ b/meta/lib/oeqa/selftest/cases/incompatible_lic.py
@@ -158,5 +158,6 @@ CORE_IMAGE_EXTRA_INSTALL:pn-core-image-weston += "rpm"
CORE_IMAGE_BASE_INSTALL:remove:pn-core-image-weston = "matchbox-terminal"
""")
bitbake('core-image-full-cmdline core-image-weston')
- bitbake('-c testimage core-image-full-cmdline core-image-weston')
+ bitbake('-c testimage core-image-full-cmdline')
+ bitbake('-c testimage core-image-weston')

--
2.39.2


Richard Purdie
 

On Mon, 2023-03-13 at 16:56 +0100, Martin Jansa wrote:
* with 1 preconfigured tap device you cannot run testimage on multiple images
at the same time

in my case have /tmp/qemu-tap-locks/tap0.skip (tap0 is used by openvpn on builder)
and /etc/runqemu-nosudo as my build user doesn't have sudo powers

but then when 2 do_testimage tasks are executed at the same time it fails like this:

NOTE: recipe core-image-weston-1.0-r0: task do_testimage: Started
NOTE: recipe core-image-full-cmdline-1.0-r0: task do_testimage: Started
QMP Available for connection at /OE/build/poky/build-st/tmp/.c2d3k3bh
QMP Available for connection at /OE/build/poky/build-st/tmp/._4agpi1v
ERROR: core-image-full-cmdline-1.0-r0 do_testimage: runqemu exited with code 1
ERROR: core-image-full-cmdline-1.0-r0 do_testimage: Output from runqemu:
runqemu - INFO - Continuing with the following parameters:
KERNEL: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/bzImage]
MACHINE: [qemux86-64]
FSTYPE: [ext4]
ROOTFS: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/core-image-full-cmdline-qemux86-64.ext4]
CONFFILE: [/OE/build/poky/build-st/tmp/deploy/images/qemux86-64/core-image-full-cmdline-qemux86-64.qemuboot.conf]

runqemu - INFO - Found /tmp/qemu-tap-locks/tap0.skip, skipping tap0
runqemu - INFO - Acquiring lockfile /tmp/qemu-tap-locks/tap1.lock failed: [Errno 11] Resource temporarily unavailable
runqemu - ERROR - Error: There are no available tap devices to use for networking,
runqemu - ERROR - and I see /etc/runqemu-nosudo exists, so I am not going to try creating
runqemu - ERROR - a new one with sudo.
runqemu - INFO - Cleaning up
runqemu - INFO - Host uptime: 67474.59

tput: No value for $TERM and no -T specified

WARNING: core-image-full-cmdline-1.0-r0 do_testimage: Qemu ended unexpectedly, dump data from host is in /OE/build/poky/build-st/tmp/log/runtime-hostdump/202303131614_qemu
WARNING: core-image-full-cmdline-1.0-r0 do_testimage: runqemu during is_alive() test
ERROR: core-image-full-cmdline-1.0-r0 do_testimage: Error executing a python function in exec_func_python() autogenerated:

which is very reasonable error message, but 600 lines later it also shows:

RESULTS - xorg.XorgTest.test_xorg_running: SKIPPED (0.00s)
SUMMARY:
core-image-weston () - Ran 71 tests in 34.073s
core-image-weston - OK - All required tests passed (successes=14, skipped=57, failures=0, errors=0)
NOTE: recipe core-image-weston-1.0-r0: task do_testimage: Succeeded
NOTE: Tasks Summary: Attempted 1141 tasks of which 1139 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
/OE/build/poky/meta/recipes-extended/images/core-image-full-cmdline.bb:do_testimag

and I didn't notice that there are actually 2 different images core-image-weston
and core-image-full-cmdline and spent most time trying to figure out how it was
able to run all those ptest when qemu failed to start at the beginning.

It's possible to create 2nd tap with runqemu-ifup before running this test.
Then it works, maybe some check like in DevtoolTests to error out early if
there isn't enough tap devices? But the implementation in:
https://git.openembedded.org/openembedded-core/commit/?id=2258345e19efff7717fe19a5026ec55f1b6f90b6
has its own limitations (e.g. not checking /tmp/qemu-tap-locks/tap0.skip)
and would probably need to be re-implemented as some generic decorator
as other tests need this as well.

It's a bit slower when serialized:
INFO - RESULTS - incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: PASSED (145.96s)
without this change:
INFO - RESULTS - incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: PASSED (104.72s)

I don't mind if this is rejected, sending it mostly to document this behavior.

Signed-off-by: Martin Jansa <Martin.Jansa@...>
---
meta/lib/oeqa/selftest/cases/incompatible_lic.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/cases/incompatible_lic.py b/meta/lib/oeqa/selftest/cases/incompatible_lic.py
index 4edf60fc55..b2da25e612 100644
--- a/meta/lib/oeqa/selftest/cases/incompatible_lic.py
+++ b/meta/lib/oeqa/selftest/cases/incompatible_lic.py
@@ -158,5 +158,6 @@ CORE_IMAGE_EXTRA_INSTALL:pn-core-image-weston += "rpm"
CORE_IMAGE_BASE_INSTALL:remove:pn-core-image-weston = "matchbox-terminal"
""")
bitbake('core-image-full-cmdline core-image-weston')
- bitbake('-c testimage core-image-full-cmdline core-image-weston')
+ bitbake('-c testimage core-image-full-cmdline')
+ bitbake('-c testimage core-image-weston')
It is expected that this should work and that multiple tap/tun devices
would be available in general when using selftest so I don't plan to
merge this. We could use all the parallelism we can find!

Cheers,

Richard