Linux Foundation Mentorship: A kickstart to my career
Introduction:
Hi everyone!
In this blog, I’m thrilled to share my Linux Foundation mentorship journey with the KubeArmor project, a cutting-edge cloud-native runtime security system. If you don’t know about runtime security yet, dive into the KubeArmor community and see its benefits for yourself. You won’t be disappointed!
Motivation:
KubeArmor is a runtime security tool that enforces policies to ensure a trusted environment, allowing only safe and intended workloads to perform operations. It can block unauthorized binaries, network activities, and file access. KubeArmor leverages eBPF and Linux LSMs, such as AppArmor and BPF-LSM, to detect and block unwanted system calls within the kernel.
Kata Containers provide a runtime that spins up lightweight virtual machines (VMs) instead of traditional containers, offering an environment that feels and behaves like containers but with enhanced isolation. This isolation helps prevent attackers from compromising the host system, even if they manage to gain elevated privileges within a container and attempt to escape.
Objective:
My project focused on identifying and implementing the necessary modifications for KubeArmor to function as a sidecar container within a Kata Container pod (where each pod is a single VM). The goal was to demonstrate that KubeArmor can enforce policies (block/allow system calls) and monitor (track system calls) all activities within the Kata Containers.
Challenges:
At the start of my mentorship, I was quite positive as in my mind I already had a map of how things would look before the coding period began. This thought turned out to be untrue pretty fast as we started discovering other issues while trying out my approach. Here’s a list:-
- KubeArmor requires certain Linux kernel headers or kernel BTF information which are used to build eBPF kernel code. This was missing in the current kernel image provided by Kata containers. We also needed Linux LSM such as AppArmor or BPFLSM enabled in the kernel config.
- How to send container information to KubeArmor running inside the guest VM?
- Once the above is completed, how to transfer KubeArmor logs running inside the guest VM to the host?
Kernel headers/BTF information:
After considering things, we realised that building our kernel was the best option to go with. This helped us enable kernel BTF and BPFLSM. Luckily BPF was already present on the machine.
Sending container information to KubeArmor:
KubeArmor requires container information such as mntNS and pidNS which was traditionally available via mounting container sockets inside KubeArmor FS, but this wasn’t possible as Kata container places a layer of isolation via using VMs. For this first, we thought of using something called vsock, but that turned out quite insecure for us. Luckily another mentorship along mine was about integrating OCI hooks with KubeArmor which also provided container information but in a secure manner. This led to the discovery of kata hooks. Kata hooks ran inside the guest VM and provided all the necessary details. Once this was achieved, the next question was how would we transfer the hook information to the KubeArmor container.
The good thing about containers they are tightly integrated with the host, unlike VMs. As container FS is directly available to the host it then became as easy as writing some clever code to transfer the hook information to the KubeArmor FS which is available at “/var/run/kata-containers/shared/containers/<CID>/rootfs/” from the guest VM. For handling on the KubeArmor side, we wrote some code to implement a file watcher a pass along container information to KubeArmor native functions which would then register the container using the CID, mntNS and pidNS inside eBPF maps.
Sending KubeArmor logs to host:
This is something that is under development, we plan to release it soon!
Mentorship Experience:
My mentors, Barun Acharya, Rudraksh Pareek and Prashant Mishra, provided invaluable guidance throughout my mentorship. Our weekly sync-ups ensured that we were aligned on progress and goals. Especially whenever I felt stuck thinking that the task item felt insurmountable, my mentors were there to help me out. A notable aspect of my project involved collaborating with another LFX mentee working on a technology crucial to our development process within KubeArmor. This collaboration was unique and highly enjoyable. Special thanks to Greg Kurz for helping out with various issues that I faced during my mentorship period!
Here’s a final diagram of how it looks like after all the engineering!
Conclusion:
If you enjoy hands-on learning through real-world projects, I strongly encourage you to apply for the LFX Mentorship. It might seem daunting at first, but with a willingness to learn and some dedicated time, you’ll thrive. Don’t hesitate to take the leap and apply!
The LFX Mentorship has been a cornerstone of my professional development, significantly enhancing my technical skills.
Please share any feedback in the comments below. Thank you for reading!
