Re: [PATCH v3 0/6] scripts/resulttool/regression: add metadata filtering
Richard Purdie
I'll try and stop poking at this but it is all rather interesting and I
think we have spotted our first nasty regression. The quilt ptests did really stop running properly and reporting test results! Looking at a recent master report: https://autobuilder.yocto.io/pub/non-release/20230224-14/testresults/testresult-report.txt you can see the quilt ptest count is still zero as it was in M2 but not in M1. I'm thinking Ross might have been responsible with: https://git.yoctoproject.org/poky/commit/?id=61bb4d8e75dfaaf980c32fbe992d34f794b7c537 ! The reason there are python3 and python3-cryptography changes are that there are 35,000 python3 ptests and about 250 changes so that isn't really unexpected for what was probably a version change. I'd guess python3-cryptography made changes to their test suite. Overall the counts of pass rates increased and no failures so those overall counts may be good to have in the report too. We might want to log the version of the recipe in the data somewhere so we can show if this was a python3 version change. Cheers, Richard |
|