Dynamic Encryption with Cribl and Splunk – Part 1

Some challenges are just awesome.

Every so often a client comes along with a request that tests not only the technologies we use but how we use them. The kind of problem that requires creativity and the occasional late night, but results in a solution that is both elegant and deceptively simple. One of our clients had just such an interesting ask.

The requirements:

1. Sensitive data should be encrypted before being indexed by Splunk.

2. All keys in an event should be preserved in their original form.

3. The values of certain key-value pairs should be encrypted as needed.

4. We can’t predict the names of the keys that the developers would like to encrypt.

5. Certain keys in an event should only be decrypted by certain roles in Splunk.

As you can probably tell, requirement number 4 is the “gotcha” moment. Without being able to predict what fields need to be encrypted ahead of time, how do we know what to encrypt? It looks like we’ll need to build something that takes its cues from the data itself.

Fortunately, Cribl has some powerful and extremely flexible data transformation capabilities that, when combined with structured events and some regex magic (courtesy of our resident wizard Ian Tinney) gives us exactly what we need. So, without further ado, here’s how to dynamically encrypt specific values based on an unknown set of keys within event data.

The Event Format

Fortunately in this case, we had quite a bit of influence in how the received events would be formatted. Working with the developers, we settled on a format that placed certain types of data into “crypto groups”. The value of each key inside a crypto group is encrypted using a key assigned to that crypto group, allowing us to encrypt different parts of the event with different keys (why we did this will become evident later).

In the example below,
crypto_group_0
contains the customer’s personal details and will be encrypted using
key_class_0
,
crypto_group_1
contains customer’s credit card details and will be encrypted using
key_class_1
. Any data
outside
of a crypto group is considered non-sensitive and will be skipped by the encryption process.

{

}

Configuration Workflow

A good rule of thumb when configuring Splunk, Cribl LogStream or anything else that moves logs is to work backwards relative to the flow of incoming events. Configuring the elements in this order will result in everything being ready when the first event arrives.

Next Up

In the next post, we’ll cover setting up Splunk to receive encrypted events from Cribl LogStream.