[RFC][PATCH v2 1/4] sstate: add a LockedSet class as python set() is not thread safe


Jose Quaresma
 

With this LockedSet python class we can call the 'add' and 'remove'
safely inside the ThreadedPool.

This piece of code is taen from the stackoverflow
https://stackoverflow.com/questions/13610654/how-to-make-built-in-containers-sets-dicts-lists-thread-safe

Fixes [YOCTO #14775] -- https://bugzilla.yoctoproject.org/show_bug.cgi?id=14775

Signed-off-by: Jose Quaresma <quaresma.jose@...>
---
meta/classes/sstate.bbclass | 24 ++++++++++++++++++++++--
1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 1c0cae4893..a3ba748a1e 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -918,8 +918,28 @@ sstate_unpack_package () {
BB_HASHCHECK_FUNCTION = "sstate_checkhashes"

def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True, **kwargs):
- found = set()
- missed = set()
+ # https://stackoverflow.com/questions/13610654/how-to-make-built-in-containers-sets-dicts-lists-thread-safe
+ import threading
+ class LockedSet(set):
+ """A set where add(), remove(), and 'in' operator are thread-safe"""
+ def __init__(self, *args, **kwargs):
+ self._lock = threading.Lock()
+ super(LockedSet, self).__init__(*args, **kwargs)
+
+ def add(self, elem):
+ with self._lock:
+ super(LockedSet, self).add(elem)
+
+ def remove(self, elem):
+ with self._lock:
+ super(LockedSet, self).remove(elem)
+
+ def __contains__(self, elem):
+ with self._lock:
+ super(LockedSet, self).__contains__(elem)
+
+ found = LockedSet()
+ missed = LockedSet()

def gethash(task):
return sq_data['unihash'][task]
--
2.35.3

Join openembedded-core@lists.openembedded.org to automatically receive all group messages.