Re: [PATCH v3 0/6] scripts/resulttool/regression: add metadata filtering


Richard Purdie
 

I'll try and stop poking at this but it is all rather interesting and I
think we have spotted our first nasty regression. The quilt ptests did
really stop running properly and reporting test results!

Looking at a recent master report:

https://autobuilder.yocto.io/pub/non-release/20230224-14/testresults/testresult-report.txt

you can see the quilt ptest count is still zero as it was in M2 but not
in M1.

I'm thinking Ross might have been responsible with:

https://git.yoctoproject.org/poky/commit/?id=61bb4d8e75dfaaf980c32fbe992d34f794b7c537

!

The reason there are python3 and python3-cryptography changes are that
there are 35,000 python3 ptests and about 250 changes so that isn't
really unexpected for what was probably a version change. I'd guess
python3-cryptography made changes to their test suite. Overall the
counts of pass rates increased and no failures so those overall counts
may be good to have in the report too.

We might want to log the version of the recipe in the data somewhere so
we can show if this was a python3 version change.

Cheers,

Richard

Join openembedded-core@lists.openembedded.org to automatically receive all group messages.