On 1/18/17 11:01 AM, Richard Purdie wrote:

On Wed, 2017-01-18 at 09:24 -0600, Mark Hatle wrote:

I have seen other systems declare these in different ways. Things
like:

RPROVIDES_pn += "user(name)"

RREQUIRES_pn += "user(name)"

Automation can declare provides and dependencies users that are
need. But of course there will always be manual users/groups that
are needed as well.

You're thinking too specifically about my example. What about ca-
certificates which needs ca-certificates-native to run its postinstall
at do_rootfs time?

Keep in mind a -native dependency doesn't really make sense in a R*
namespace. This isn't really a package manager level problem but a
build system one. How do we know which -native dependencies a
postinstall needs?

yes, I was missing the native piece in my thinking.... I see why this is
potentially a big problem for the sstate-cache.

Theoretically we should already be doing some of this via
dependencies today -- but it can be a complex problem.

We have some gross hacks in the system today but nothing really marks
up which postinsts need which native dependencies.

Of course this doesn't handle other things postinsts may be needed
for...

Users are really just one very special case and the problem is much
wider.

The PACKAGE_WRITE_DEPS can either be in addition to DEPENDS if the
recipe needs these tools at build time, or if its only needed for
the
postinst, PACKAGE_WRITE_DEPS can be sufficient and the DEPENDS can
be
removed.

I'm a little worried this is making things more complicated then they
may need to be.

What is the downside of simply specifying those items in the existing
DEPENDS as we have it today? (Performance, they have to be installed
before the package can begin to build?)

DEPENDS means "this recipe needs dependencies X at build time in order
to build so they must be available by do_configure".

We could insist all DEPENDS for all recipes are installed before we run
do_rootfs but that is a significant performance issue and a large set.

The one thing I keep coming back to on DEPENDS (vs a more specific DEPENDS). In
the general case, when I've built recipes I have say 'I depend on XYZ', thinking
then that anything XYZ depends on will be available for me to build.

That would include the native items. Now, that of course does NOT mean they are
required to unpack/'install into the sysroot' those items, just that the
dependencies exist for my current build.

With recipe specific sysroots, we've reached a point where we really do
need to know which dependencies postinstalls have IMO.

What I'm trying to wrap my head around are the cases where you don't depend on
something for build time (or don't want a dependency to be inherited) that won't
cause more confusion over time.

I can certainly see the optimization cases where I know I only need XYZ for the
install, but the rest of the DEPENDS (and their DEPENDS) are not needed.

So if I'm understanding this right.

If we did all DEPENDS inherited from our DEPENDS, then this would be working
properly. But that means then all dependencies would need to be installed each
time, causing a huge performance impact -- especially on native dependencies
which are often assumed or so?

By default all DEPENDS (not native) will still be loaded and inherited, and
we're really only concerned with adding missing 'native' things that a specific
install script may need?

So for performance reasons, we want to not automatically install all of the
dependent referred native packages -- and specify them as needed -- only where
needed.. helping performance by further being able to delay the extract
installation to as late as possible (the package write function -- or equivalent
sstate-cache install functions [pre/post/etc]).

The name itself doesn't make me understand (without reading your
explanation) why it's required. That also makes me suspect it might
be difficult (over time) to explain to people when they need this
behavior and why.

We could call it PACKAGE_POSTINST_DEPS. The issues here are that it
applies to PREINST too and that it really should list native deps, not
packaging deps. PACKAGE_WRITE_DEPS does accurately reflect where the
dependency goes and there are likely uses for this outside of
postinstalls as there are other places which have to spell out "add dep
X for deb, for ipk, for rpm).

I think where I'm getting confused by the name is, that PACKAGE_WRITE_DEPS
(equivalent of do_package_write*) sounds to me like something required in order
to write the package to the disk. So rpm, deb, opkg that kind of thing would go
there.

It's not clear that the package's install script would be affected by this in
any way (nor the sstate-cache versions.)

(I'm struggling to come up with any name better then PACKAGE_WRITE_DEPENDS
myself...)

I am hopeful that the number of places that need to reference this is
actually quite small and mostly in the core classes (like useradd,
systemd and so on). It really is a piece of core infrastructure rather
than something most developers would run into, and if they do need it,
the explains should make it clear how to use it. Code wise, the
additions are alongside the postinst pieces they apply to in the most
part.

Does that make it clearer why we need this?

I certainly understand it more.. I'm just struggling with how to explain when
someone would need this or not, to someone who hasn't hit one of the extraction
problems. I think that is by far my biggest remaining concern, how to make
sure people can understand they need to do this and when...

--Mark

Cheers,

Richard