Skip to content

rottweiler: measure settings without region or image registries - #1084

Open
ginglis13 wants to merge 1 commit into
bottlerocket-os:developfrom
ginglis13:pcr8-location-agnostic-sources
Open

ginglis13 wants to merge 1 commit into
bottlerocket-os:developfrom
ginglis13:pcr8-location-agnostic-sources

Conversation

@ginglis13

Copy link
Copy Markdown
Contributor

Description of changes:

PCR 8 is extended with the EC2 IMDS user data and then with the OS settings. Two kinds of values in them describe where a host is deployed rather than what it runs, so measuring them gives an otherwise identical fleet a different PCR 8 value in every region or account, and anyone verifying PCR 8 has to carry one expected value per location:

  • settings.aws.region, which early-boot-config sets from the
    instance identity document on every AWS host; and
  • the registry component of container image sources (settings.host-containers.*.source, settings.bootstrap-containers.*.source)

In both extensions of PCR8, exclude settings.aws.region and remove the registry from every host and bootstrap container source before hashing: <registry>/repo[:tag][@digest] is measured as repo[:tag][@digest], where the first path component is a registry when it contains a . or : or is localhost (the rule Docker and containerd use). The repository, tag, and digest are kept, so a digest-pinned source still identifies the image content. The settings extension keeps excluding settings.updates.seed and settings.network.hostname as before. Stored settings are not modified; only the measured bytes are.

For the user-data extension, the user data is parsed the same way early-boot-config reads it (optional gzip,
then TOML) into plain tables and values, the region and the registries are removed, and the canonical JSON of
the whole document is measured under a new ec2-imds-user-data:v2: framing. For the settings extension, rottweiler now fetches apiclient's standard JSON (its canonical output is not valid JSON), removes the region and the registries, and serializes with the same canonical formatter and trailing newline apiclient uses.

Settings and user data can carry secrets, so the parsed trees are held in a wrapper that zeroizes every string and key on drop, removed and replaced values are zeroized as they go, the decoded user-data text is kept in a Zeroizing<String>, and TOML values are moved rather than copied into the JSON tree. As with the previous raw-bytes measurement this is best effort: copies the TOML and JSON parsers make internally, and stale heap blocks left by reallocation, are not covered.

The threat model in docs/ENCRYPTED_STORAGE.md now states that neither the region nor the registry is attested by PCR 8 and recommends pinning custom host and bootstrap container sources by digest.

Testing done:

I copied bottlerocket-admin:v0.21.6 and bottlerocket-bootstrap:v0.3.6 with crane copy from public.ecr.aws
into ECR repos of the same name in my own account, in us-east-1 and us-east-2.

A Per-region URIs written into user data:

[settings.host-containers.admin]
enabled = true
source = "328549459982.dkr.ecr.<region>.amazonaws.com/bottlerocket-admin:v0.21.6"
[settings.bootstrap-containers.bootstrap]
source = "328549459982.dkr.ecr.<region>.amazonaws.com/bottlerocket-bootstrap:v0.3.6"
mode = "once"
essential = false

B Custom host container pcr8test + custom bootstrap container pcr8boot (bootstrap runs a script from
user-data that writes /local/pcr8-bootstrap-ran:

Case Region Sources
1 us-east-1 my account's ECR
2 us-east-1 Bottlerocket's account ECR, same repo:tag
3 us-east-1 1's user data gzip-compressed (--user-data fileb://X.toml.gz)
4 us-east-1 1 with host source digest-pinned …/bottlerocket-admin:v0.21.6@sha256:c7f3e9c8…
5 us-east-2 1's layout in us-east-2

Collected results via sheltie:

  • cat /sys/class/tpm/tpm0/pcr-sha256/8
  • apiclient get settings --exclude settings.updates.seed --exclude settings.network.hostname
  • systemctl is-active host-containers@pcr8test
  • apiclient get settings.bootstrap-containers.pcr8boot.mode

Results

Case PCR 8 (sha256) Check
A us-east-1 CAB68C0558589E8869EB5A9D22E1BBC9D849BDBB1EA37BD5D01C8E6B2A34A15C == A us-east-2
A us-east-2 CAB68C0558589E8869EB5A9D22E1BBC9D849BDBB1EA37BD5D01C8E6B2A34A15C one value across regions with per-region URIs in user data and no settings
B-1 my ECR DB2BDEE52252274BD707B293EB98DC68ADE58BF6CB09510DD7CD1507A7720171 baseline
B-2 Bottlerocket ECR DB2BDEE5… == B-1 , different account, same image identity
B-3 gzip user data DB2BDEE5… == B-1
B-4 us-east-2 DB2BDEE5… == B-1 , different region
B-5 digest-pinned 25672E3FADD1ECCA0E94F3D105F9CAE9E2A6A505B6AA14E2B24296815AC259B3 != B-1 , digest is measured (bottlerocket-admin:v0.21.6@sha256:c7f3…); pinned pull worked

Measured user-data extension for A (identical in both regions): ec2-imds-user-data:v2:{"settings":{"bootstrap-containers":{"bootstrap":{"essential":false,"mode":"once","source":"bottlerocket-bootstrap:v0.3.6"}},"host-containers":{"admin":{"enabled":true,"source":"bottlerocket-admin:v0.21.6"}}}}

On all five B instances: host-containers@pcr8test active; bootstrap-containers@pcr8boot succeeded and /local/pcr8-bootstrap-ran was written; the digest-pinned pull resolved and ran.

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

PCR 8 is extended with the EC2 IMDS user data and then with the OS
settings. Both carry values that describe where a host is deployed
rather than what it runs: `settings.aws.region`, which early-boot-config
sets on every AWS host, and the registry in host and bootstrap container
sources. Measuring them gives an identical fleet a different PCR 8 value
in every region or account.

In both extensions, exclude `settings.aws.region` and measure every
`settings.host-containers.*.source` and
`settings.bootstrap-containers.*.source` without its registry:
`<registry>/repo[:tag][@digest]` becomes `repo[:tag][@digest]`, where
the first path component is a registry if it contains `.` or `:` or is
`localhost` (the Docker/containerd rule). Repository, tag, and digest
are kept, so a digest-pinned source still identifies the image.
Stored settings are not modified.

The settings extension now fetches apiclient's JSON, transforms it,
and canonicalizes with the same formatter and trailing newline.
User data is parsed as early-boot-config reads it into plain TOML
tables, transformed the same way, and measured as canonical JSON
under a new `ec2-imds-user-data:v2:` framing.

Parsed trees are held in a wrapper that zeroizes strings on drop, and
removed or replaced values are zeroized as they go; as before, this is
best effort. docs/ENCRYPTED_STORAGE.md documents the new measurement
and recommends pinning custom container sources by digest, since
neither the region nor the registry is attested.

Signed-off-by: Gavin Inglis <giinglis@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant