Grundschutz++ as code, part 6: What no repository knows

01 — Two levels

At heart, the checks from part 4 answer one question: is what the code says actually applied? A drift check does that very well, because it computes the plan against the actual state and reports every deviation.

Its view ends at the edge of the state, though. A DNS record, a security group or a deployment someone created by hand does not appear in the plan at all. Where the code is silent, the comparison finds nothing, and it is green anyway.

On the second level the question therefore runs the other way. I list what actually exists and ask of every object which code describes it. If there is none, that is a finding.

02 — Three sources

To start with, I picked three places where, in an environment like mine, something is most likely to be made by hand:

• DNS: every record in the managed zones is compared with the records the OpenTofu code declares. A name nobody wrote down is an entry point nobody knows about.
• Cloud resources: instances, public addresses, networks, security groups with their rules and all buckets are held against the state.
• Cluster: the objects of named kinds in the Kubernetes cluster are compared with what GitOps rolls out.

Every source writes its result just like the checks from part 4, as an OSCAL assessment result with one observation per object. So every finding lands in the chain from part 5 as a matter of course, all the way into the plan of action, without my having to build anything new.

03 — The declared side

A comparison needs two lists. The cloud API supplies one, the other has to come from the code. There I do not read the configuration files but the state, because only the state knows which ID an object got when it was created.

For this, the daily drift check writes two small lists as an artefact: the DNS records with zone, name and type, and the resources with type and ID. Nothing more leaves the job, on purpose. The state also holds values that have no business outside it, and for the comparison the identifier is entirely enough.

tofu show -no-color | awk -f declared.awk | sort -u > declared-resources.txt

04 — A rule is a resource of its own

How large the difference between the two levels is shows most clearly with security groups. In my code every rule is a resource of its own. That is clean and easy to read, but it has a consequence that is easy to miss, precisely because the drift check reports green so reliably.

If someone attaches an extra rule by hand to a declared group, the check stays silent. Nothing about the group itself has changed, and the plan does not know the new rule. The very opening that matters is invisible.

The cloud source therefore compares not only groups but every single rule by its ID. A rule without a counterpart in the state becomes a finding, even if the group it hangs on is properly declared.

05 — Owners instead of a list

Not everything that exists is itself in the code. An Auto Scaling group starts instances, pods come from a deployment, and a storage driver creates volumes. A fixed list of all permitted objects would be out of date within an hour.

Instead of a list, the sources ask for the owner. An object counts as owned if it is itself declared, in the state or in a GitOps application, or if a declared object created it, an Auto Scaling group for instance, or in the cluster through an owner reference. Added to that is what the platform brings by itself, such as the default objects of Kubernetes, and whatever is listed in the configuration as a named exception with a reason.

Everything else has no owner and becomes a finding. The typical example of an exception, by the way, is the bucket with the OpenTofu state: it holds the very state that would have to describe it, so it cannot manage itself.

06 — What a source may not read

A source that may see everything quickly becomes a risk itself, and in the cluster that holds all the more. Whoever may list objects there gets them back in full, not just their names.

The job in the cluster therefore reads only named kinds, and only with list. Secrets and ConfigMaps are not among them, because both can carry values a compromised pod should not get to see. For the question of ownership, the metadata of the other kinds is enough anyway.

In the end only a report leaves the cluster, naming the objects without an owner. The job may write it to exactly one place in the locked bucket and nowhere else. On the network it reaches only the API and public HTTPS targets, while the node's metadata service stays blocked for it.

The gap this creates belongs to the picture and should not go unmentioned: a Secret or ConfigMap created by hand is not found by this source.

07 — New sources run on trial first

The results from part 4 are signed and locked against deletion for 400 days. That is intended, but it has a downside I only came to appreciate in operation: a wrong finding can no longer be deleted afterwards, only explained.

And a new source in particular produces mostly findings about itself in its first runs. Sometimes it does not know an owner yet, sometimes it reads a format wrongly. In the first days of the measurements I made exactly such mistakes, and every one of them sits in the bucket today as a locked result.

Since then every new source starts with trial: true. Its result is built and checked against the schema, but neither signed nor published, and it does not yet appear in the assessment plan. Only once I have read the first runs by hand and explained every finding may it speak for a requirement.

08 — A test with a known outcome

A check that never finds anything can mean two things: either everything is in order, or it is blind. From outside the two cannot be told apart unless you give it something to find.

That is what I did. By hand and past the code I created one object each, a TXT record in one of the zones and an empty security group. Both say in their name what they are, and neither does any harm.

The next run reported exactly these two objects and nothing else. The cloud source even found a little more than I had created, namely the two egress rules AWS gives every new group automatically. They are in no code either, and the check saw them too, although I had not thought of them myself.

Then I removed both objects again. The TXT record appeared as a finding in the run at 16:12 UTC on 1 October. In the run at 16:46 UTC it was gone.

The cluster source got its test on 13 October. At 17:32 UTC I created an empty namespace by hand, again with a name that says what it is. The next run at 22:41 UTC reported it as an object without an owner. In the signed result, KONF.2.1.1, KONF.2.5 and KONF.10.1 were therefore not satisfied. On 14 October at 05:22 UTC I deleted the namespace. In the run at 10:43 UTC it was gone, and the three requirements were satisfied again.

The DNS finding and the namespace finding, by the way, are signed and locked like every result. So they stay in the bucket for 400 days, but their names explain them.

09 — Three outcomes for every finding

A finding without an owner is not a list entry to remember for later. It has exactly three possible outcomes:

• It becomes code: the object is wanted and now gets the description it has been missing.
• It gets a reason: the object has to stay as it is and from now on is listed in the configuration as a named exception with a reason.
• It goes away: nobody needs the object, so it is removed.

A fourth outcome along the lines of “we will look at it later” I deliberately do not allow. It would turn the second level into exactly the list that at some point nobody reads any more.

The DNS source ran 30 times from 1 to 7 October. Its first run reported five records that were none: the source read the name of the zone itself in escaped notation and therefore did not find it in the code. An hour later that was fixed. On 3 October three DKIM records for a mail service turned up, created while setting up the service and not in the code. They became code and were no longer among the findings from 6 October.

The cloud and cluster sources came off trial on 7 October. In 27 runs up to 13 October the cloud source reported itself twice: once it saw fewer resources than expected, once the declared list was too old. Neither is an object without an owner. Both are a source that cannot vouch for itself, and both were gone in the next run. The cluster source found only the test namespace from section 8.

10 — Two views are missing on purpose

A proof of concept is only as honest as the list of what it cannot do. Two views that would belong to a complete second level I have therefore left out on purpose, and I want to name both here.

No port scan from outside. A scan would show what is actually reachable from the internet, including where the code intends something else. That takes a vantage point outside my own environment, though, and I do not have one at the moment. What is meant to be reachable I read from the code instead: addresses and security groups from OpenTofu, Services and Ingresses from GitOps. That is weaker than a scan, because it shows the intention and not the result.

No certificate transparency. The public logs record every certificate issued for a domain. In practice they can only be searched through a third party's index, though, and of the two I tried, one was unreachable during the test while the other wants an account for production use. A hostname without code is found directly by the DNS source. What stays invisible is a certificate someone had issued for one of my domains without a record in my zones, for a SaaS service, say, or through mis-issuance. CAA limits who may issue, but it does not report who did.

I accept both gaps for this stage on purpose. They can be added later as further sources without anything in the chain having to change.

A third view is still to come: the flow logs, the actual network against the intended architecture. So far only the fact that the flow logs are running is measured.

11 — What the second level is not

• Not a complete inventory: each source knows only the kinds it lists and only the region it looks at. What lies outside stays unknown, which is why the list of kinds is itself a result to review regularly.
• Not a safeguard: the sources find an object only on the next run and cannot prevent anything. That is the job of permissions and admission rules.
• Not a replacement for the drift check: it stays responsible for the first level. The second shrinks the unknown; it cannot abolish it.

12 — The series at a glance

1. Grundschutz++ against your own documentation: the method, the catalogue as a data model, the comparison with IaC and GitOps.
2. Applicability as data: the statement of applicability as an OSCAL profile.
3. Containers that say which requirements they meet: a signed component definition on every image the pipeline builds.
4. Measurement as evidence: checks write their results as OSCAL, dated, signed, impossible to delete.
5. The ISMS as a build product: security plan, assessment plan, results and plan of action as an OSCAL chain, rebuilt with every measurement.
6. What no repository knows: the second level, the running system against its code. This post.
7. One measurement, many standards: NIS2, DORA, C5 and SOC 2 as mappings onto Grundschutz++.
8. The chain to take away: general checks, the report over a period and a template that wires it all together.

Parts 1 to 6 describe what runs in my ISMS. Each appears only once its step runs in operation and has been measured. Parts 7 and 8 show tools I built after feedback at the InfoSec-Stammtisch and do not use myself. The code is in grundschutz-worksheets. A map of the whole series is OSCAL in practice: eight models, one chain.

Conclusion

A drift check answers the question whether the code is applied, and it answers it well. It does not answer the auditor's question. For that you have to ask the running system itself and demand an owner for every object.

What convinced me in the end was not the first green run but the test with a known outcome. Once a check has been shown to find something, you can take it at its word when it finds nothing. What it still cannot see is in section 10, and that is just as much part of the result.

Continue with part 7: One measurement, many standards.