Re: [PATCH 1/2] sstate: don't limit the thread pool size when checking mirrors

Richard Purdie

On Thu, 2021-12-02 at 22:05 +0000, Jose Quaresma wrote:
This improves the time needed to check the mirrors,
reducing it from 50s to 8s when building the core-image-minimal
using the yocto upstream sstate and hashequivlance servers
on a machine with 8 cpu cores.

Tested with:

SSTATE_MIRRORS = "file://.*;downloadfilename=PATH"
bitbake core-image-minimal

Signed-off-by: Jose Quaresma <quaresma.jose@...>
meta/classes/sstate.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 0326d27c74..5e404d7cd8 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -1005,7 +1005,7 @@ def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True,
tasklist.append((tid, sstatefile))

if tasklist:
- nproc = min(int(d.getVar("BB_NUMBER_THREADS")), len(tasklist))
+ nproc = len(tasklist)

progress = len(tasklist) >= 100
if progress:
No, we're not doing that. That opens a ton (thousands) of parallel connections
to the servers and isn't fair on the systems hosting the data. In many cases
that would look like a DoS attack.



Join to automatically receive all group messages.