About secrets

In the last post, I wrote about the possibility of displaying the status of a server based on events.

The short form: If I save all changes to a server, they will inevitably lead to a result that I will always get if I start with an "empty" server and replay the saved events.

A generated result could look like this:

[
    "users" => [
      [
        "username" => 'example',
        "password" => 'password',
        "publicKeys" => [
          "ssh-ed25519 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICkpqD8aGHWS9dt/vaXcWhSrtwIxJR3+1w8IouHtqioF example@localhost",
        ]
      ]
    ]
  ]

This representation as an array allows me to pass the content on to a configuration management system of my choice. This can be Ansible, Puppet, Chef, Saltstack or whatever is currently modern.

The only problem is that I may write things down as an event that should not be available in plain text. Obviously, I should avoid the following:

And yet I am forced to save this value if I not only want to evaluate these secrets at runtime, but also want to retain the option of rewriting my projections based on the saved events at a different time.

Now there are different approaches. The obvious option is to simply encrypt the event_properties column. Many frameworks offer convenient options for this.

This option is certainly better than saving a plain text password.

The result could look like this, for example:

Problems from the unforeseen side

But even this convenient approach has its problems. The obvious problem is that all contents of the event_properties column are encrypted. Even if they are not worth protecting.

If you accept this overhead, however, the crucial point remains the use of the same key for all content. Good. Various frameworks also offer support for rotating keys.

But: Under certain circumstances, a situation may arise in which data worthy of protection must be removed because this is required by the GDPR, for example. If a user requests the deletion of their data, it is an anti-pattern to subsequently modify the stored events.

Similar to accounting, deletion is not intended. Bookkeeping recognizes correcting entries - but this approach does not free users from the fact that things are stored that should not be stored (anymore).

Deleting the key that encrypted the event_properties column is also not an option, as in this case all events could no longer be read.

Crypto shredding

My solution to this problem is Crypto Shredding. Or at least my interpretation of the approach.

Let me tell you what I mean by that.

The moment a new -aggregate-, let's call it a "thing", is created, an associated key is generated and securely stored. In addition to the stored key, there must be an assignment, for example via the id of the "thing".

As long as this key exists, the event can be linked to this key and decrypted if necessary. If the deletion of the key is requested in the program logic, the event cannot be decrypted. Ideally, this is even possible for individual fields of the stored event.

An example of an event that was encrypted with the above key:

As long as the key exists, the encrypted fields can be decrypted. If this is not the case, the value cannot be restored.

In this way, a secret can be deleted without changing the stored events.