<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 15, 2015 at 9:04 AM, Paul Barker <span dir="ltr"><<a href="mailto:paul.barker@commagility.com" target="_blank">paul.barker@commagility.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On Sun, 6 Dec 2015 11:26:33 +0000<br>
Paul Barker <<a href="mailto:paul.barker@commagility.com">paul.barker@commagility.com</a>> wrote:<br>
<br>
> I ran into a race condition building multiple external modules against a 3.10.y<br>
> series kernel using the dylan branch of OpenEmbedded. This is difficult to<br>
> reproduce as it requires very specific timing: the do_make_scripts task for one<br>
> module was linking the modpost script whilst the do_compile task for another<br>
> module was attempting to use the modpost script. This resulted in a permission<br>
> error:<br>
><br>
> ERROR: Function failed: do_compile (see /home/COMMAGILITY/paul.barker/linux-bsp/work-1.0.1/ca-linux-bsp/build/tmp/work/amc_d24a4-oe-linux-gnueabi/ti-hplib-mod/01.01.00.04-r3d/temp/log.do_compile.25434 for further information)<br>
> ERROR: Logfile of failure stored in: /home/COMMAGILITY/paul.barker/linux-bsp/work-1.0.1/ca-linux-bsp/build/tmp/work/amc_d24a4-oe-linux-gnueabi/ti-hplib-mod/01.01.00.04-r3d/temp/log.do_compile.25434<br>
> Log data follows:<br>
> | DEBUG: Executing shell function do_compile<br>
> | make -C /home/COMMAGILITY/paul.barker/linux-bsp/work-1.0.1/ca-linux-bsp/build/tmp/sysroots/amc-d24a4/usr/src/kernel M=$PWD clean<br>
> | make[1]: Entering directory `/home/COMMAGILITY/paul.barker/linux-bsp/work-1.0.1/ca-linux-bsp/build/tmp/sysroots/amc-d24a4/usr/src/kernel'<br>
> | make[1]: Leaving directory `/home/COMMAGILITY/paul.barker/linux-bsp/work-1.0.1/ca-linux-bsp/build/tmp/sysroots/amc-d24a4/usr/src/kernel'<br>
> | make -C /home/COMMAGILITY/paul.barker/linux-bsp/work-1.0.1/ca-linux-bsp/build/tmp/sysroots/amc-d24a4/usr/src/kernel M=$PWD modules<br>
> | make[1]: Entering directory `/home/COMMAGILITY/paul.barker/linux-bsp/work-1.0.1/ca-linux-bsp/build/tmp/sysroots/amc-d24a4/usr/src/kernel'<br>
> |Â Â CC [M]Â /home/COMMAGILITY/paul.barker/linux-bsp/work-1.0.1/ca-linux-bsp/build/tmp/work/amc_d24a4-oe-linux-gnueabi/ti-hplib-mod/01.01.00.04-r3d/git/ti/runtime/hplib/module/hplibmod.o<br>
> |Â Â Building modules, stage 2.<br>
> |Â Â MODPOST 1 modules<br>
> | /bin/sh: scripts/mod/modpost: Permission denied<br>
> | make[2]: *** [__modpost] Error 126<br>
> | make[1]: *** [modules] Error 2<br>
> | make[1]: Leaving directory `/home/COMMAGILITY/paul.barker/linux-bsp/work-1.0.1/ca-linux-bsp/build/tmp/sysroots/amc-d24a4/usr/src/kernel'<br>
> | make: *** [default] Error 2<br>
> | ERROR: Function failed: do_compile (see /home/COMMAGILITY/paul.barker/linux-bsp/work-1.0.1/ca-linux-bsp/build/tmp/work/amc_d24a4-oe-linux-gnueabi/ti-hplib-mod/01.01.00.04-r3d/temp/log.do_compile.25434 for further information)<br>
> ERROR: Task 1284 (/home/COMMAGILITY/paul.barker/linux-bsp/work-1.0.1/ca-linux-bsp/meta-mcsdk/meta-arago-extras/recipes-bsp/ti-hplib/<a href="http://ti-hplib-mod_git.bb" rel="noreferrer" target="_blank">ti-hplib-mod_git.bb</a>, do_compile) failed with exit code '1'<br>
><br>
> Later kernel versions do not rebuild the modpost script every time that 'make<br>
> scripts' is invoked so they should be safe from this particular failure. However<br>
> I'm not convinced that running 'make scripts' whilst also building an<br>
> out-of-tree module is always safe on later kernels and there is always the<br>
> potential for vendor kernels to have different behaviour here.<br>
><br>
> Although this was seen on the dylan branch the behaviour of master and jethro<br>
> looks to be the same here - do_make_scripts is locked so that only one instance<br>
> of it may run at one time but there is nothing to prevent one instance of<br>
> do_make_scripts running at the same time as an instance of do_compile.<br>
><br>
> The patch I'm sending attempts to solve this issue by locking the do_compile<br>
> task with the same lockfile as the do_make_scripts task in module.bbclass so<br>
> that an instance of do_copile can't run at the same time as an instance of<br>
> do_make_scripts. I don't know enough about the task locking to guarantee that<br>
> this is the right solution or to be able to test that it works as expected so<br>
> I'm marking the patch as an RFC.<br>
><br>
> Please let me know if this is the right approach and if there is any easy way to<br>
> test this.<br>
><br>
> Paul Barker (1):<br>
>Â Â module.bbclass: Fix potential do_compile/do_make_scripts race<br>
>Â Â Â condition<br>
><br>
>Â meta/classes/module.bbclass | 4 ++++<br>
>Â 1 file changed, 4 insertions(+)<br>
><br>
<br>
</div></div>ping on this.<br></blockquote><div><br></div><div>Sorry. I was traveling when this landed .. made a mental note .. and then never<br></div><div>looped around.<br></div><div>Â </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I've just got bitten by this again so it's not a one-off. Is anyone able to<br>
give me some feedback on the patch, whether this is the right approach to fix<br>
the problem and whether this is applicable to jethro/master.<br></blockquote><div><br></div><div>The approach makes sense to me, and it was what I was considering for generating<br></div><div>symbols after do_compile_modules. As long as it isn't serializing a huge part of<br></div><div>the build, the impacts are even measurable.<br><br></div><div>So this change looks sane to me.<br><br></div><div>Bruce<br></div><div>Â </div><div><br>Â </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Thanks,<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Paul Barker<br>
CommAgility Ltd<br>
</font></span><div class="HOEnZb"><div class="h5">--<br>
_______________________________________________<br>
Openembedded-core mailing list<br>
<a href="mailto:Openembedded-core@lists.openembedded.org">Openembedded-core@lists.openembedded.org</a><br>
<a href="http://lists.openembedded.org/mailman/listinfo/openembedded-core" rel="noreferrer" target="_blank">http://lists.openembedded.org/mailman/listinfo/openembedded-core</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature">"Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end"</div>
</div></div>