Runtime Security in Kubernetes deployment might be policed based on a pod-by-pod.  A pod is a group of containers that shares a network namespace, which can underlying mechanisms for runtime security is identical.  The potential to do granular runtime security builds specialist container security tooling a compelling prospect, especially for enterprises with a lot at risks, such as healthcare organizations or banks.

One of the features of containers is they lend themselves to micro-service architectures. Developers can break down a complex software application into self-contained fragments of code that can be scaled individually, each delivered as a container image. Breaking down a large system into its components with well-defined interfaces makes it easier to design, code, and test the individual components. This makes them easier to secure.

Container Image Profiles

If a given container image holds the source code for an application microservice with one small function, it is somewhat easier to reason about what that microservice could do. The source code for the application microservice is built into a container image, and it’s viable to create a runtime profile corresponding to that container image, defining what it should be able to do.  An expected behavior profile can be defined for an image because each container instantiated from a given image should behave similarly. Then this will be used to police the inbound and outbound traffic for all the containers based on that image.

Network Traffic Profiles

The containers need to accept and respond to web requests coming from a particular ingress or load balancer, and they should initiate a database connection to the product database. Aside from common platform functions like logging or health checks, there is no reason for this service to handle or initiate any other network traffic.

It would not be onerous to draw up a profile defining the traffic that is permitting for this service and then use it to define rules that are enforced at the networking level. Some security tools can act in a recording mode in which they monitor messages to and from service over some time to automatically build up a profile of what normal traffic flow looks like. This profile can be converted into container firewall rules or network policies. Network traffic isn’t the only behavior that you can observe and profile.

Executable Profiles

For instance, consider the code which is written as a single Go executable called productsearch. You can only find productsearch, if you were to monitor the executable running inside these productsearch containers. Anything else will be considered as an anomaly and might be an attack sign. Even if the service is written in a scripted language like Python or Ruby, you can make inferences about what is or isn’t acceptable.

If the service need not “shell out” to run other commands, then were you ever to observe bash, sh, or zsh executables running in a product search container, you should be notified. This relies on you treating the containers as immutable and assumes that you are not opening shells directly into containers on your production system. From a security perspective, there is very little difference between an attacker opening a reverse shell through an application vulnerability and an administrator opening a shell to perform some kind of “maintenance.”

File Access Profiles

It is possible to use eBPF or other technologies to observe when a system call is used to start an executable, you can observe the system calls that access files. In general, the set of file locations that you would expect a given micro-service to access is also relatively limited. This list is sufficiently long that even an experienced programmer might omit a few of these files if they tried to draw up a profile by hand, but with the help of some tools, it’s straightforward to create the list of files that a container is expected to access. Again, some security tools offer the ability to profile running containers automatically and then alert on or prevent opening files outside the expected profile.

User ID Profiles

Another aspect that can be policed by security tools at runtime is to define the user ID under which processes run within a container. In general, if the container is doing one job, it probably requires to operate under that particular user identity. It would be another red flag if the container using a different identity. The privilege escalation would be an even greater cause for concern if a process were to be unexpectedly running with root privilege.