Date   

Re: combining trusted/security layers (fix typo in Jia's address)

Randy MacLeod
 

Oops.
Fix typo in Jia's address: .co -> .com


On 05/24/2018 05:26 PM, Trevor Woerner wrote:
Hi everyone, and thanks for all the feedback that's been given already.
I think it would be a great idea if we could get the various trusted/security
layers working together on one layer instead of having separate efforts. As
far as I'm aware, there are currently 3 such layers:
...
From what is presented in the spreadsheet, in my opinion, I don't think it'll
be too hard to get everything in one layer. Surprisingly, there isn't a lot of
overlap. Therefore, all the unique bits from each layer can simply be added to
the one chosen layer. The only real overlap is in the tpm stuff, and that
should be easy to update once in the chosen layer.
The easiest way to combine the layers would be to make meta-security another
sub-layer of meta-secure-core. But I think that might be too simplistic.
meta-security includes a hodgepodge of user-space tools and daemons for
doing miscellaneous security things (recipes-security). meta-secure-core tries
to break logical activities into their own layers (i.e. meta-ids for intrusion
detection systems, meta-integrity for integrity measurement architecture
(ima), etc). If it would be possible to categorize all of the recipes in
meta-security's recipes-security directory, then maybe we could start
distributing them into meta-secure-core and/or creating spaces for them?
Thoughts?
Add Jia, who I've been talking with about our discussion on the
YP tech call yesterday. Hopefully he'll get his email situation
fixed and can carry on without me being a relay node.


Email from Jia:

My email client has a filter problem on receiving emails from gmail for
unknown reason (not a proxy issue) so I cannot directly reply him. Could
you do me a favor to copy my reply to there?

-- reply --

I'm pleasure to see this move. And it sounds great to combine all in one
with a unified design model. Meanwhile, it is effective to avoid the
duplication works on maintainability. Additionally, it also gives a
fine-grained degree on the selection of a subset of feature from the big
one.

Categorizing the recipes in meta-security may be the hardest work in the
whole move. I take a quick glance at the list (thanks for Trevor!) and a
big catalog would be penetration test (meta-penetration-test?). We need
more catalogs to cover the remaining tools. Definitely, the naming
scheme for me is a challenge.

Regarding meta-tpm1/2, we could consider to cherry pick one among the 3
layers as the baseline and move the trivial parts in recipe from other 2
layers into the baseline. Other conflicting recipes would follow the
same methodology.

Thanks,
Jia

--
# Randy MacLeod
# Wind River Linux


Re: combining trusted/security layers

Randy MacLeod
 

On 05/24/2018 05:26 PM, Trevor Woerner wrote:
Hi everyone, and thanks for all the feedback that's been given already.
I think it would be a great idea if we could get the various trusted/security
layers working together on one layer instead of having separate efforts. As
far as I'm aware, there are currently 3 such layers:
...
From what is presented in the spreadsheet, in my opinion, I don't think it'll
be too hard to get everything in one layer. Surprisingly, there isn't a lot of
overlap. Therefore, all the unique bits from each layer can simply be added to
the one chosen layer. The only real overlap is in the tpm stuff, and that
should be easy to update once in the chosen layer.
The easiest way to combine the layers would be to make meta-security another
sub-layer of meta-secure-core. But I think that might be too simplistic.
meta-security includes a hodgepodge of user-space tools and daemons for
doing miscellaneous security things (recipes-security). meta-secure-core tries
to break logical activities into their own layers (i.e. meta-ids for intrusion
detection systems, meta-integrity for integrity measurement architecture
(ima), etc). If it would be possible to categorize all of the recipes in
meta-security's recipes-security directory, then maybe we could start
distributing them into meta-secure-core and/or creating spaces for them?
Thoughts?
Add Jia, who I've been talking with about our discussion on the
YP tech call yesterday. Hopefully he'll get his email situation
fixed and can carry on without me being a relay node.


Email from Jia:

My email client has a filter problem on receiving emails from gmail for
unknown reason (not a proxy issue) so I cannot directly reply him. Could
you do me a favor to copy my reply to there?

-- reply --

I'm pleasure to see this move. And it sounds great to combine all in one
with a unified design model. Meanwhile, it is effective to avoid the
duplication works on maintainability. Additionally, it also gives a
fine-grained degree on the selection of a subset of feature from the big
one.

Categorizing the recipes in meta-security may be the hardest work in the
whole move. I take a quick glance at the list (thanks for Trevor!) and a
big catalog would be penetration test (meta-penetration-test?). We need
more catalogs to cover the remaining tools. Definitely, the naming
scheme for me is a challenge.

Regarding meta-tpm1/2, we could consider to cherry pick one among the 3
layers as the baseline and move the trivial parts in recipe from other 2
layers into the baseline. Other conflicting recipes would follow the
same methodology.

Thanks,
Jia

--
# Randy MacLeod
# Wind River Linux


combining trusted/security layers

Trevor Woerner
 

Hi everyone, and thanks for all the feedback that's been given already.

I think it would be a great idea if we could get the various trusted/security
layers working together on one layer instead of having separate efforts. As
far as I'm aware, there are currently 3 such layers:

meta-measured (http://layers.openembedded.org/layerindex/branch/master/layer/meta-measured/)
meta-security (http://layers.openembedded.org/layerindex/branch/master/layer/meta-security/)
meta-secure-core (http://layers.openembedded.org/layerindex/branch/master/layer/meta-secure-core/)

I personally am most familiar with meta-measured, and I'm mostly only
interested in tpm2 on an RPi3B+.

In an effort to try to help gather the data required to jumpstart this
conversation, I've created a simple google doc that lists these three layers,
their recipes, and provides the results of building against 2 MACHINEs:

intel-corei7-64 (from meta-intel)
raspberrypi3 (from meta-raspberrypi)

https://docs.google.com/spreadsheets/d/1AlH0Q0lGC3idwyFLSt7df09sIXkBuv191fVESUA-oQY/edit?usp=sharing

Please have a look. This spreadsheet is very simple, and only looks at
recipes, it does not include any information about various bbappends, nor
kernel configurations, packagegroups, classes, sample images, etc...

meta-measured is a plain, straight-forward layer that contains recipes.

meta-security contains recipes, but also contains 2 sub-layers:
- meta-tpm
- meta-security-compliance

meta-secure-core is a meta-layer, containing no recipes itself, but collecting
together a set of sub-layers:
- meta
- meta-encrypted-storage
- meta-integrity
- meta-efi-secure-boot
- meta-ids
- meta-signing-key
- meta-tpm
- meta-tpm2

From what is presented in the spreadsheet, in my opinion, I don't think it'll
be too hard to get everything in one layer. Surprisingly, there isn't a lot of
overlap. Therefore, all the unique bits from each layer can simply be added to
the one chosen layer. The only real overlap is in the tpm stuff, and that
should be easy to update once in the chosen layer.

The easiest way to combine the layers would be to make meta-security another
sub-layer of meta-secure-core. But I think that might be too simplistic.
meta-security includes a hodgepodge of user-space tools and daemons for
doing miscellaneous security things (recipes-security). meta-secure-core tries
to break logical activities into their own layers (i.e. meta-ids for intrusion
detection systems, meta-integrity for integrity measurement architecture
(ima), etc). If it would be possible to categorize all of the recipes in
meta-security's recipes-security directory, then maybe we could start
distributing them into meta-secure-core and/or creating spaces for them?

Thoughts?

Best regards,
Trevor


Re: Trusted/secure/etc layers

Trevor Woerner
 

On Tue, May 8, 2018 at 5:04 AM, Joshua Lock <joshua.g.lock@...> wrote:


On 01/05/2018 19:57, Trevor Woerner wrote:
I don't think any of them support ESAPI nor FAPI.

tpm2-tss 2.0, currently in RC, comes with ESAPI support.


Excellent, thanks for the update!


OpenEmbedded Developer Meeting in Edinburgh Oct 21, 2018 (before ELCE)

Philip Balister
 

Once again we will have a developer meeting in Edinburgh the Sunday
before ELCE.

Please go to https://www.openembedded.org/wiki/OEDEM_2018 and add
yourself if you are attending and ideas for topics.

Although it is called a developer meeting, we invite members of the
larger community to attend. The core developers are always interested in
hearing how OpenEmbedded is used, and what we can do to make it better
for building the embedded devices of the future.

Thanks,

Philip


Re: Trusted/secure/etc layers

Joshua Lock <joshua.g.lock@...>
 

On 01/05/2018 19:57, Trevor Woerner wrote:
I don't think any of them support ESAPI nor FAPI.
tpm2-tss 2.0, currently in RC, comes with ESAPI support.

Regards,
Joshua


Re: sstate equivalency

Richard Purdie
 

On Fri, 2018-05-04 at 18:38 +0200, Martin Jansa wrote:
I think the important part is that equivalency mechanism should cover
the output of the tasks, but mostly from the POV of the other tasks
which depend on it.

Adding or changing comments inside do_install shell functions is
something which signature handler (or rather bitbake parser) can skip
to prevent the do_install task to be re-executed in first place.
I think comments are a false lead here. In that case there are things
we could do to remove the rebuild, however, its also something we
comparatively rarely do and not the real problem.


But what I understand as "sstate equivalency" issue is that if you
apply CVE fix for e.g. zlib (includes zlib-native, nativesdk-zlib),
then all the zlib tasks of course need to be re-executed, new
packages created and included in new images and so on, but the
equivalency checker will compare the output of zlib (e.g. from
do_install) and discover that zlib is different the checksum changed
from ABC to DEF (in this case because security bug was fixed), but
the ABI is the same as from the previous build, so this is
"equivalent" and DEF will marked as equivalent to ABC on equivalence
server. Then when e.g. libxml2 tasks are being considered for
execution, then libxml2.do_configure which depends on
zlib.do_populate_sysroot will see that zlib.do_populate_sysroot
changed checksum from ABC to DEF, but after consulting with
equivalence server bitbake knows that this is OK and this change of
dependency can be ignored and libxml2.do_configure checksum 456 is
again marked as equivalent to 123 and libxml2 build doesn't need to
be re-executed (and whole hierarchy depending on it also doesn't need
to be).
This is where a plugable "equivalence" mechanism could be important.

Note that your scenario above is actually potentially dangerous to the
system as you could end up with two different sstate objects, one with
a security fix and one without. They are different and should not
really ever be associated with the same hash as you want people to pick
up the new different one with the fix.

In this case you really want the original object to be not equivalent
but all dependencies to be equivalent.

I think to start with I'd prefer to see a general identical tarball
approach taken as the first pass. People would then be free to
experiment with more interesting tests for equivalence and how the
dependencies may "ripple" (or not).

It gets much more complicated for components like gcc-cross. When
some bug is fixed in gcc-cross then the output might have the same
ABI, but for cross we don't care about ABI, we care about rebuilding
the binaries with fixed toolchain, so for gcc-cross we might still
need to rebuild everything, but still if the above is implemented,
then gcc-cross won't really change the checksum so often, because
e.g. ABI safe change in zlib won't cause gcc-cross to change.

I think that the best attempt on implementing this was from BMW Car
IT.

For reference, couple related posts from thread:
[yocto] RFC: Backwards compatibility checking sstate-cache
it is spread across couple months (so it's shown disconnected when
looking through pipermail):
It really depends on how deeply we dive into ABI comparisons. To start
with I think we need to look at solving some of the simpler problems.
The mechanism can scale to such ABI checks but if we try and start
there I think people would run into issues and quickly lose faith in
the whole idea.

It is good to think about whether the simple approach can scale into
these scenarios though.

Cheers,

Richard


Re: sstate equivalency

Martin Jansa
 

I think the important part is that equivalency mechanism should cover the output of the tasks, but mostly from the POV of the other tasks which depend on it.

Adding or changing comments inside do_install shell functions is something which signature handler (or rather bitbake parser) can skip to prevent the do_install task to be re-executed in first place.

But what I understand as "sstate equivalency" issue is that if you apply CVE fix for e.g. zlib (includes zlib-native, nativesdk-zlib), then all the zlib tasks of course need to be re-executed, new packages created and included in new images and so on, but the equivalency checker will compare the output of zlib (e.g. from do_install) and discover that zlib is different the checksum changed from ABC to DEF (in this case because security bug was fixed), but the ABI is the same as from the previous build, so this is "equivalent" and DEF will marked as equivalent to ABC on equivalence server. Then when e.g. libxml2 tasks are being considered for execution, then libxml2.do_configure which depends on zlib.do_populate_sysroot will see that zlib.do_populate_sysroot changed checksum from ABC to DEF, but after consulting with equivalence server bitbake knows that this is OK and this change of dependency can be ignored and libxml2.do_configure checksum 456 is again marked as equivalent to 123 and libxml2 build doesn't need to be re-executed (and whole hierarchy depending on it also doesn't need to be).

It gets much more complicated for components like gcc-cross. When some bug is fixed in gcc-cross then the output might have the same ABI, but for cross we don't care about ABI, we care about rebuilding the binaries with fixed toolchain, so for gcc-cross we might still need to rebuild everything, but still if the above is implemented, then gcc-cross won't really change the checksum so often, because e.g. ABI safe change in zlib won't cause gcc-cross to change.

I think that the best attempt on implementing this was from BMW Car IT.

For reference, couple related posts from thread:
[yocto] RFC: Backwards compatibility checking sstate-cache
it is spread across couple months (so it's shown disconnected when looking through pipermail):


Cheers,

On Fri, May 4, 2018 at 5:53 PM, Richard Purdie <richard.purdie@...> wrote:
On Wed, 2018-05-02 at 10:41 +0800, ChenQi wrote:
> My focus is on user experience.
> sstate is a successful mechanism because it has simple and clear
> rules. 
>  From the user's view, it's easy to understand why something is
> rebuilt.
> Adding such complex equivalency checking will surprise users, and
> then 
> confuse them.
> So regardless of the solution, the goal itself is trying to break
> these 
> simple and clear rules.

I certainly agree that we want to keep things simple. I believe the
equivalency mechanism can be kept very simple though and shouldn't
cause too much added confusion (compared to everything else we do).

> > I think one of the biggest open questions I have is the mechanism
> > for
> > detecting if some new task hash is equivalent to an old one (known)
> > one. I think there are two basic options:
> >
> > 1) Do an actual build and determine equivalence afterward. This
> > seems
> > the easiest option in my mind, but it does require at least one
> > build
> > of the new hash to determine that it is functionally equivalent to
> > an
> > old hash, and then a mechanism to publish this result so that
> > hopefully no one else has to do it also.
> This solution doesn't converge.
> Say we have in database stating that recipe A's do_install would 
> consider the following changes to not affect the final output.
> 1) adding comment "# this is 1"
> 2) adding comment "# this is 2"
> 3) adding comment "# this is 3"
> Then, when user adds a comment "# this is 1", A's do_install is not 
> rerun; when the user adds a comment "# this is 4", and the do_install
> is  rerun. It's not reasonable.
> Basically we have infinite numbers of comments, so this solution does
> not converge.

do_install is not an sstate task but it could be in theory so lets
pretend it is.

The result of do_install is the ${D} directory containing the set of
installed files which sstate would turn into a tarball.

Imagine we have three builds with each of your different comments, I'll
call them builds 1, 2 and 3.

We would create that sstate tarball and take a checksum of it in each
case. If the checksums are the same, we'd deem the build to be
equivalent since the binary output is the same. I appreciate we'd have
to be careful about timestamps etc. but that is a solvable issue.

All the sstate server would need to know is that the sstate checksums
corresponding to builds 1 to 3 are equivalent.

The setup implies that new configurations have to get built at least
once. The hope is that if something early in the build chain is found
to be the same, large parts of previous builds can be reused from
sstate.

Does that help?

Cheers,

Richard
_______________________________________________
Openembedded-architecture mailing list
Openembedded-architecture@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture


Re: sstate equivalency

Richard Purdie
 

On Wed, 2018-05-02 at 10:41 +0800, ChenQi wrote:
My focus is on user experience.
sstate is a successful mechanism because it has simple and clear
rules. 
 From the user's view, it's easy to understand why something is
rebuilt.
Adding such complex equivalency checking will surprise users, and
then 
confuse them.
So regardless of the solution, the goal itself is trying to break
these 
simple and clear rules.
I certainly agree that we want to keep things simple. I believe the
equivalency mechanism can be kept very simple though and shouldn't
cause too much added confusion (compared to everything else we do).

I think one of the biggest open questions I have is the mechanism
for
detecting if some new task hash is equivalent to an old one (known)
one. I think there are two basic options:

1) Do an actual build and determine equivalence afterward. This
seems
the easiest option in my mind, but it does require at least one
build
of the new hash to determine that it is functionally equivalent to
an
old hash, and then a mechanism to publish this result so that
hopefully no one else has to do it also.
This solution doesn't converge.
Say we have in database stating that recipe A's do_install would 
consider the following changes to not affect the final output.
1) adding comment "# this is 1"
2) adding comment "# this is 2"
3) adding comment "# this is 3"
Then, when user adds a comment "# this is 1", A's do_install is not 
rerun; when the user adds a comment "# this is 4", and the do_install
is  rerun. It's not reasonable.
Basically we have infinite numbers of comments, so this solution does
not converge.
do_install is not an sstate task but it could be in theory so lets
pretend it is.

The result of do_install is the ${D} directory containing the set of
installed files which sstate would turn into a tarball.

Imagine we have three builds with each of your different comments, I'll
call them builds 1, 2 and 3.

We would create that sstate tarball and take a checksum of it in each
case. If the checksums are the same, we'd deem the build to be
equivalent since the binary output is the same. I appreciate we'd have
to be careful about timestamps etc. but that is a solvable issue.

All the sstate server would need to know is that the sstate checksums
corresponding to builds 1 to 3 are equivalent.

The setup implies that new configurations have to get built at least
once. The hope is that if something early in the build chain is found
to be the same, large parts of previous builds can be reused from
sstate.

Does that help?

Cheers,

Richard


Re: sstate equivalency

Richard Purdie
 

On Tue, 2018-05-01 at 15:28 -0500, Joshua Watt wrote:
I have some interest it looking into the sstate equivalency as
discussed in the Yocto Project Technical Meeting. I know there have
been a few discussions about how to handle this problem, so please
chime in with any information or background you may have.

To start things off, I figured I would try to briefly summarize the
problem so that we have a clear idea what we are trying to solve.

My understanding is that there are many changes that can be made to a
recipe that don't actually have any affect on what the recipe
generates (comments in shell tasks(?), that sort of thing), but still
figure into the sstate task hashes. When such changes are detected,
it would be convenient to users if the build could detect that these
task hashes are actually equivalent, and allow one sstate cache
object to be used in place of the other (e.g. If you have sstate
object A and your build calculates the hash as B, you can ask
somewhere, "is A equivalent to B?", and if so, use A in lieu of
rebuilding to produce B).
That sums up the problem, yes.

I think one of the biggest open questions I have is the mechanism for
detecting if some new task hash is equivalent to an old one (known)
one. I think there are two basic options:

1) Do an actual build and determine equivalence afterward. This seems
the easiest option in my mind, but it does require at least one build
of the new hash to determine that it is functionally equivalent to an
old hash, and then a mechanism to publish this result so that
hopefully no one else has to do it also.
As I understand it, this is the only practical mechanism we have.

2) Try to calculate if the new hash is going to be equivalent without
doing a build. This seems *much* harder, IMHO... but also eliminates
the need for a "central" database of equivalent hashes (because
everyone can determine it themselves).
There are probably some cases we could optimise, such as stripping out
comments from shell code but I don't think this is a problem we could
solve generally and the corner cases we could solve wouldn't be real
wins. I'd therefore propose focusing on 1) above.


I *think* based on what I have heard, everyone is leaning toward
option #1, which makes a lot of sense. In that regard, I think that
this is very doable. As a simplistic first approach, I think it
should be entirely possible to piggyback on the reproducable build
effort that has been on-going to solve the question of "when are two
tasks identical". Essentially, if the outputs from two tasks are
equivalent, the two input task hashes can be considered to be
equivalent. You could of course perhaps get more intelligent than
that, but I don't have a clear idea of what that would look like.
Exactly, the reproducible work puts us in a position where we can now
realistically look at this. The equivalence test could be as simple as
comparing the checksums of the tarballs in sstate.


After that, the other question I can think of right now is "how are
results published". I'm a little less clear on what ideas people had
there, so please chime in. I think you could go a simple as "symlinks
in the sstate cache", and as complicated as a dedicated hierarchy of
server with authentication and signing.
Symlinks would only buy us so much and it gets complicated very quickly
since A can depend on B which can depend on C. If A and D are
equivalent, we'd write a symlink for D to A, B using D's checksum, C
using B+D and so on. It scales as NxN which isn't good.

My suggested approach having given this some thought is this idea of an
equivalence server.

The build computes what its going to build as normal. It submits those
sstate checksums to a server. That server "canonicalizes" them if it
has entries for them to some standard parent value (the artefact that
is in sstate). The build then proceeds with the canonicalized version
of the checksum for the purposes of fetching from sstate.

The equivalence server therefore just needs to maintain mappings for
sets of sstate checksums back to a canonical value.

Whatever is populating the sstate cache would also upload equivalence
mappings. The mappings could be postprocessed into place, or done as
part of a build.

For a first pass, I'd not try and be any cleverer about it than this.

For a second level improvement, we could consider artefact comparison
at the end of each task completion, then it would short cut the build
if artefacts were found to be equivalent part way through a build.

This second level is fairly complicated and invasive as the current
bitbake runqueue has no mechanism to rework its dependency trees/build
queue on the fly. Adding such a mechanism would be hard as in theory
things currently building may suddenly not be needed and so on.

We should be able to make something really useful with just the first
pass approach though.

Does that make sense?

Cheers,

Richard


Re: Trusted/secure/etc layers

Rich Persaud
 

On May 2, 2018, at 18:44, Tom Rini <trini@...> wrote:

On Wed, May 02, 2018 at 06:26:31AM +0200, Patrick Ohly wrote:
Trevor Woerner <twoerner@...> writes:
Philip Tricca (for Intel) has been leading a lot of TSS work as well, he also
maintains meta-measured https://github.com/flihp/meta-measured for OE recipes.

Note that he is very open to the idea of moving those recipes elsewhere
and deprecating meta-measured. The recipes themselves aren't getting
updated as often as they used to be, too.

Trying to break this down into small tasks, there's a number of cases
for making use of GRUB_BUILDIN in the context of security (Intel TXT, UEFI
secure boot, I bet the AMD equivalent of TXT).  So we have the generic
hook we need (good!).

Intel TXT (DRTM) [1]  and UEFI secure boot (SRTM) each offer specific security benefits.  They can also be combined [2].

To achieve security benefits with DRTM, there's a bit more involved than grub. E.g. what are you measuring?  If you want to measure rootfs, it needs to be read-only.  If you OTA update and change the measurement, device fails to boot new version.  We've solved some of these issues in one configuration of OE with Xen [3].

DRTM depends on correctly implemented system BIOS.  After we started testing DRTM on endpoints in early 2010s, it took a few years and bricked devices before OEMs worked through the BIOS issues.  DRTM BIOS on servers is likely earlier in the maturity cycle.  Two devices, same CPU generation (BSP), can have different OEM BIOS quirks.  Windows 10 SystemGuard DRTM [4] may improve UEFI and OEM BIOS validation.

There's a conference [2] on these topics in May. Philip Tricca and Bruce Ashfield will be presenting, among other OE contributors.

Rich Persaud






Re: Trusted/secure/etc layers

Tom Rini
 

On Thu, May 03, 2018 at 09:28:40AM -0400, Trevor Woerner wrote:
On Wed 2018-05-02 @ 08:27:01 PM, Tom Rini wrote:
On Wed, May 02, 2018 at 11:31:02PM +0000, Otavio Salvador wrote:
A DISTRO_FEATURES or a MACHINE_FEATURES? It seems more related to the
machine.
Maybe I'm misunderstanding the inherent assumptions of this thread, and maybe
I'm overestimating the effort involved, but my guess is that although we could
throw a bunch of recipes into oe-core or meta-oe to support some of this work,
actually getting secure boot to work on any specific platform is going to need
a good chunk of work by the BSP maintainers?

Would it be fair to say that working secure boot is mostly a BSP issue?

Perhaps a better question might be: if we make this a *_FEATURE in oe-core,
does this mean oe-core needs to take on all of the tpm and secure-boot
recipes? Is it possible to standardize a *_FEATURE in oe-core, but make its
use dependent on an external layer?
This is partly what I've been asking about. What can we and should we
do for oe-core (and IMHO, the very much related topic of
meta-openembedded / git.openembedded.org hosted layers) related to
security?

My opinion is that we need:
- Sufficient hooks in anything that's in oe-core and used for/by popular
security features that something can be hooked in. AFAICT we seem to
have this, but we need more people that have poked more technologies
that fall under the umbrella of security to chime in.
- Try and get the 3+ layers for dealing with a TPM to coalesce into one
layer, or have a real good understanding of why it can't be.
- Work with layers that provide various security features so that it is
easy to apply them to an arbitrary BSP that follows best practices, or
an arbitrary distro that again follows best practices.

--
Tom


Re: Trusted/secure/etc layers

Trevor Woerner
 

On Wed 2018-05-02 @ 08:27:01 PM, Tom Rini wrote:
On Wed, May 02, 2018 at 11:31:02PM +0000, Otavio Salvador wrote:
A DISTRO_FEATURES or a MACHINE_FEATURES? It seems more related to the
machine.
Maybe I'm misunderstanding the inherent assumptions of this thread, and maybe
I'm overestimating the effort involved, but my guess is that although we could
throw a bunch of recipes into oe-core or meta-oe to support some of this work,
actually getting secure boot to work on any specific platform is going to need
a good chunk of work by the BSP maintainers?

Would it be fair to say that working secure boot is mostly a BSP issue?

Perhaps a better question might be: if we make this a *_FEATURE in oe-core,
does this mean oe-core needs to take on all of the tpm and secure-boot
recipes? Is it possible to standardize a *_FEATURE in oe-core, but make its
use dependent on an external layer?


Re: Trusted/secure/etc layers

Tom Rini
 

On Wed, May 02, 2018 at 11:31:02PM +0000, Otavio Salvador wrote:
On Wed, May 2, 2018 at 7:44 PM Tom Rini <trini@...> wrote:

On Wed, May 02, 2018 at 06:26:31AM +0200, Patrick Ohly wrote:
Trevor Woerner <twoerner@...> writes:
Philip Tricca (for Intel) has been leading a lot of TSS work as well,
he also
maintains meta-measured https://github.com/flihp/meta-measured for OE
recipes.

Note that he is very open to the idea of moving those recipes elsewhere
and deprecating meta-measured. The recipes themselves aren't getting
updated as often as they used to be, too.
Trying to break this down into small tasks, there's a number of cases
for making use of GRUB_BUILDIN in the context of security (Intel TXT, UEFI
secure boot, I bet the AMD equivalent of TXT). So we have the generic
hook we need (good!).
As an architecture question do we want to add more DISTRO_FEATURES
(intel-txt, efi-secure-boot, amd-something) and have the main grub-efi
recipe look at those hooks and further update GRUB_BUILDIN? Or just as
part of documentation (and perhaps more content to
local.conf.sample.extended) take care of it there? I think I lean
towards more official DISTRO_FEATURES as that will also help us merge a
number of scc/cfg fragments from this area as well.
A DISTRO_FEATURES or a MACHINE_FEATURES? It seems more related to the
machine.
Oh probably. But the thing is a lot of these aren't exactly machine
features but architecture features. What I think we need to avoid
somehow is the need to whack N machine.conf files to enable something
that's fairly commonly possible. For example, efi-secure-boot is
(relatively) easily done in both intel-corei7-64 and qemux86 and
qemux86-64 and intel-core2-32.

--
Tom


Re: Trusted/secure/etc layers

Otavio Salvador
 

On Wed, May 2, 2018 at 7:44 PM Tom Rini <trini@...> wrote:

On Wed, May 02, 2018 at 06:26:31AM +0200, Patrick Ohly wrote:
Trevor Woerner <twoerner@...> writes:
Philip Tricca (for Intel) has been leading a lot of TSS work as well,
he also
maintains meta-measured https://github.com/flihp/meta-measured for OE
recipes.

Note that he is very open to the idea of moving those recipes elsewhere
and deprecating meta-measured. The recipes themselves aren't getting
updated as often as they used to be, too.
Trying to break this down into small tasks, there's a number of cases
for making use of GRUB_BUILDIN in the context of security (Intel TXT, UEFI
secure boot, I bet the AMD equivalent of TXT). So we have the generic
hook we need (good!).
As an architecture question do we want to add more DISTRO_FEATURES
(intel-txt, efi-secure-boot, amd-something) and have the main grub-efi
recipe look at those hooks and further update GRUB_BUILDIN? Or just as
part of documentation (and perhaps more content to
local.conf.sample.extended) take care of it there? I think I lean
towards more official DISTRO_FEATURES as that will also help us merge a
number of scc/cfg fragments from this area as well.
A DISTRO_FEATURES or a MACHINE_FEATURES? It seems more related to the
machine.
--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750


Re: Trusted/secure/etc layers

Tom Rini
 

On Wed, May 02, 2018 at 06:26:31AM +0200, Patrick Ohly wrote:
Trevor Woerner <twoerner@...> writes:
Philip Tricca (for Intel) has been leading a lot of TSS work as well, he also
maintains meta-measured https://github.com/flihp/meta-measured for OE recipes.
Note that he is very open to the idea of moving those recipes elsewhere
and deprecating meta-measured. The recipes themselves aren't getting
updated as often as they used to be, too.
Trying to break this down into small tasks, there's a number of cases
for making use of GRUB_BUILDIN in the context of security (Intel TXT, UEFI
secure boot, I bet the AMD equivalent of TXT). So we have the generic
hook we need (good!).

As an architecture question do we want to add more DISTRO_FEATURES
(intel-txt, efi-secure-boot, amd-something) and have the main grub-efi
recipe look at those hooks and further update GRUB_BUILDIN? Or just as
part of documentation (and perhaps more content to
local.conf.sample.extended) take care of it there? I think I lean
towards more official DISTRO_FEATURES as that will also help us merge a
number of scc/cfg fragments from this area as well.

That just leaves needing to have a real talk about what to do about TPM
layers :)

--
Tom


Re: Trusted/secure/etc layers

Patrick Ohly
 

Trevor Woerner <twoerner@...> writes:
Philip Tricca (for Intel) has been leading a lot of TSS work as well, he also
maintains meta-measured https://github.com/flihp/meta-measured for OE recipes.
Note that he is very open to the idea of moving those recipes elsewhere
and deprecating meta-measured. The recipes themselves aren't getting
updated as often as they used to be, too.

--
Patrick Ohly


Re: sstate equivalency

Chen Qi
 

On 05/02/2018 04:28 AM, Joshua Watt wrote:
All,

I have some interest it looking into the sstate equivalency as
discussed in the Yocto Project Technical Meeting. I know there have
been a few discussions about how to handle this problem, so please
chime in with any information or background you may have.

To start things off, I figured I would try to briefly summarize the
problem so that we have a clear idea what we are trying to solve.

My understanding is that there are many changes that can be made to a
recipe that don't actually have any affect on what the recipe
generates (comments in shell tasks(?), that sort of thing), but still
figure into the sstate task hashes. When such changes are detected, it
would be convenient to users if the build could detect that these task
hashes are actually equivalent, and allow one sstate cache object to
be used in place of the other (e.g. If you have sstate object A and
your build calculates the hash as B, you can ask somewhere, "is A
equivalent to B?", and if so, use A in lieu of rebuilding to produce
B).
Hi Joshua,

My focus is on user experience.
sstate is a successful mechanism because it has simple and clear rules. From the user's view, it's easy to understand why something is rebuilt.
Adding such complex equivalency checking will surprise users, and then confuse them.
So regardless of the solution, the goal itself is trying to break these simple and clear rules.

I think one of the biggest open questions I have is the mechanism for
detecting if some new task hash is equivalent to an old one (known)
one. I think there are two basic options:

1) Do an actual build and determine equivalence afterward. This seems
the easiest option in my mind, but it does require at least one build
of the new hash to determine that it is functionally equivalent to an
old hash, and then a mechanism to publish this result so that
hopefully no one else has to do it also.
This solution doesn't converge.
Say we have in database stating that recipe A's do_install would consider the following changes to not affect the final output.
1) adding comment "# this is 1"
2) adding comment "# this is 2"
3) adding comment "# this is 3"
Then, when user adds a comment "# this is 1", A's do_install is not rerun; when the user adds a comment "# this is 4", and the do_install is rerun. It's not reasonable.
Basically we have infinite numbers of comments, so this solution does not converge.

2) Try to calculate if the new hash is going to be equivalent without
doing a build. This seems *much* harder, IMHO... but also eliminates
the need for a "central" database of equivalent hashes (because
everyone can determine it themselves).
This is also not doable. Because python and shell codes' results cannot be got by merely parsing them.

The above are my opinions on this issue.

Best Regards,
Chen Qi


this is very doable. As a simplistic first approach, I think it should
be entirely possible to piggyback on the reproducable build effort
that has been on-going to solve the question of "when are two tasks
identical". Essentially, if the outputs from two tasks are equivalent,
the two input task hashes can be considered to be equivalent. You
could of course perhaps get more intelligent than that, but I don't
have a clear idea of what that would look like.

After that, the other question I can think of right now is "how are
results published". I'm a little less clear on what ideas people had
there, so please chime in. I think you could go a simple as "symlinks
in the sstate cache", and as complicated as a dedicated hierarchy of
server with authentication and signing.

I'm sure I'm glossing over a lot here....

Thanks,
Joshua Watt
_______________________________________________
Openembedded-architecture mailing list
Openembedded-architecture@...
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture


Re: sstate equivalency

Mark Hatle <mark.hatle@...>
 

On 5/1/18 3:28 PM, Joshua Watt wrote:
All,

I have some interest it looking into the sstate equivalency as
discussed in the Yocto Project Technical Meeting. I know there have
been a few discussions about how to handle this problem, so please
chime in with any information or background you may have.

To start things off, I figured I would try to briefly summarize the
problem so that we have a clear idea what we are trying to solve.

My understanding is that there are many changes that can be made to a
recipe that don't actually have any affect on what the recipe
generates (comments in shell tasks(?), that sort of thing), but still
Note: Occasionally we've used shell comment changes specifically to force a
rebuild. So if we can get smart enough to avoid certain things like this -- we
will still need a simply way to force a rebuild in the few cases where the
developers knows it's needed, but it's not automated.

figure into the sstate task hashes. When such changes are detected, it
would be convenient to users if the build could detect that these task
hashes are actually equivalent, and allow one sstate cache object to
be used in place of the other (e.g. If you have sstate object A and
your build calculates the hash as B, you can ask somewhere, "is A
equivalent to B?", and if so, use A in lieu of rebuilding to produce
B).

I think one of the biggest open questions I have is the mechanism for
detecting if some new task hash is equivalent to an old one (known)
one. I think there are two basic options:

1) Do an actual build and determine equivalence afterward. This seems
the easiest option in my mind, but it does require at least one build
of the new hash to determine that it is functionally equivalent to an
old hash, and then a mechanism to publish this result so that
hopefully no one else has to do it also.
Determining the post build equivalency needs a rule though -- many of the
existing tools we have are likely adequate.

My next question then is, how do I publish equivalencies to other people (inside
and outside of my organization.)

2) Try to calculate if the new hash is going to be equivalent without
doing a build. This seems *much* harder, IMHO... but also eliminates
the need for a "central" database of equivalent hashes (because
everyone can determine it themselves).



I *think* based on what I have heard, everyone is leaning toward
option #1, which makes a lot of sense. In that regard, I think that
this is very doable. As a simplistic first approach, I think it should
be entirely possible to piggyback on the reproducable build effort
that has been on-going to solve the question of "when are two tasks
identical". Essentially, if the outputs from two tasks are equivalent,
the two input task hashes can be considered to be equivalent. You
could of course perhaps get more intelligent than that, but I don't
have a clear idea of what that would look like.

After that, the other question I can think of right now is "how are
results published". I'm a little less clear on what ideas people had
there, so please chime in. I think you could go a simple as "symlinks
in the sstate cache", and as complicated as a dedicated hierarchy of
server with authentication and signing.

I'm sure I'm glossing over a lot here....
Ya, unfortunately I have no clear answer, but a lot of possibilities. A text
representation like the locked down hashes.. or a semi-database (think PR w/ an
sqlite DB)... or even something like a 'DNS' like solution. Where your query is
on the hash, and your answer is on equivalence... I say DNS 'like' since if the
local query doesn't know it could pass it to more 'authoritative' services until
they are exhausted... While complex, it could allow for the YP to publish
equivalencies, as well as OSV, development groups, etc... and have them all work
together. (Of course we run into network access issues, is there any
identifying info in the hashes, etc etc etc..)

--Mark

Thanks,
Joshua Watt
_______________________________________________
Openembedded-architecture mailing list
Openembedded-architecture@...
http://lists.openembedded.org/mailman/listinfo/openembedded-architecture


sstate equivalency

Joshua Watt
 

All,

I have some interest it looking into the sstate equivalency as
discussed in the Yocto Project Technical Meeting. I know there have
been a few discussions about how to handle this problem, so please
chime in with any information or background you may have.

To start things off, I figured I would try to briefly summarize the
problem so that we have a clear idea what we are trying to solve.

My understanding is that there are many changes that can be made to a
recipe that don't actually have any affect on what the recipe
generates (comments in shell tasks(?), that sort of thing), but still
figure into the sstate task hashes. When such changes are detected, it
would be convenient to users if the build could detect that these task
hashes are actually equivalent, and allow one sstate cache object to
be used in place of the other (e.g. If you have sstate object A and
your build calculates the hash as B, you can ask somewhere, "is A
equivalent to B?", and if so, use A in lieu of rebuilding to produce
B).

I think one of the biggest open questions I have is the mechanism for
detecting if some new task hash is equivalent to an old one (known)
one. I think there are two basic options:

1) Do an actual build and determine equivalence afterward. This seems
the easiest option in my mind, but it does require at least one build
of the new hash to determine that it is functionally equivalent to an
old hash, and then a mechanism to publish this result so that
hopefully no one else has to do it also.

2) Try to calculate if the new hash is going to be equivalent without
doing a build. This seems *much* harder, IMHO... but also eliminates
the need for a "central" database of equivalent hashes (because
everyone can determine it themselves).



I *think* based on what I have heard, everyone is leaning toward
option #1, which makes a lot of sense. In that regard, I think that
this is very doable. As a simplistic first approach, I think it should
be entirely possible to piggyback on the reproducable build effort
that has been on-going to solve the question of "when are two tasks
identical". Essentially, if the outputs from two tasks are equivalent,
the two input task hashes can be considered to be equivalent. You
could of course perhaps get more intelligent than that, but I don't
have a clear idea of what that would look like.

After that, the other question I can think of right now is "how are
results published". I'm a little less clear on what ideas people had
there, so please chime in. I think you could go a simple as "symlinks
in the sstate cache", and as complicated as a dedicated hierarchy of
server with authentication and signing.

I'm sure I'm glossing over a lot here....

Thanks,
Joshua Watt

921 - 940 of 1685