Managing Kubernetes secrets with HashiCorp Vault vs. Azure Key Vault

You wouldn’t leave the keys to your house in a place where every passerby can find it. Similarly, secrets that guard your digital or cloud estate can’t be left exposed within code, or stored in files, or passed through chat conversations. These secrets, usually API keys, access tokens, SSH keys, and other sensitive information, must be locked up and watched closely to remain secure. Although Kubernetes offers a native secrets manager, it isn’t ideal due to its limited scope – it only works within Kubernetes clusters and doesn’t offer much in terms of intelligence. A better way to manage Kubernetes secrets is through external secrets management tools like Azure Key Vault and HashiCorp Vault. While they can manage secrets efficiently, the two key vaults follow different approaches. As much as it is important to protect your secrets from hostile actors, you must choose a tool that works seamlessly with your Kubernetes ecosystem.

In this article, we compare Azure Key Vault and HashiCorp Vault, discussing their security posture, advantages, and disadvantages to guide you through the selection process.

Using HashiCorp Vault for managing Kubernetes secrets

HashiCorp Vault

HashiCorp Vault is a secrets management and encryption solution that helps organizations tightly control their sensitive information by securing the storage and access of secrets. It enables audit trails through its identity-based access management. However, most users do not leverage these audit capabilities. Furthermore – The few that do are not monitoring it regularly for anomaly detection and security issues.

Managing Kubernetes in HashiCorp Vault is a simple and intuitive process, given the ease of integration. Usually, secrets stored in Kubernetes clusters are vulnerable due to a lack of encryption.

Let’s understand the advantages and disadvantages of HashiCorp Vault.

Source: HashiCorp

Its key features:

• • In HashiCorp Vault, you can store arbitrary key/value secrets. Before storing them in persistent storage, the tool encrypts all the sensitive data. Therefore, even if hackers get hold of raw storage, they won’t be able to access your keys and passwords.

Having said that, in order to access Hashicorp, you will usually need to have an api token that is able to access the secrets in the vault. Unfortunately, in many instances the DevOps that created the Hashicorp instance do not follow security best practices, such as the least privilege practice. The result is that any one that holds the api key can fetch all Hashicorp vault secrets.  The encryption in this case becomes useless.

In addition, other challenges arise such as securing the token that can access the Hashi vault.  Will the usage of this token be monitored?  Will there be different tokens for different vaults and different environments? will they rotated regularly? Will it be encrypted? how?

Vault empowers you to generate secrets on-demand for platforms like AWS and SQL databases. If you want to access an S3 bucket, Vault will create a temporary AWS key pair on demand, which will expire after a set duration. It is important to keep in mind that rotating secrets of 3rd party cloud services is not supported by HashiCorp vault .

3rd party vendor secrets can be exposed in Slack, Confluenceetc before they were  secured in the vault

• • You can manage access for humans and machines through strong authentication and authorization methods. For humans, you can implement RBAC to assign permissions, access secrets, and manage users. However, if the human laptop has already been breached this will not be helpful . Vault creates temporary credentials to allow machines to access secrets or systems, which will be automatically revoked.

• • With Vault, your security teams can define guidelines for managing Kubernetes secrets in HashiCorp Vault, which developers can use to store encrypted data. It frees your developers from having to determine encryption protocols from scratch. It is important to note that you must be very careful when accessing vaults or creating secrets/revoking them, this step has to be done manually or at least fully monitored E2E.

Step-wise flow of managing Kubernetes secrets

There are multiple ways to setup Vault. Here is a brief overview:

You can deploy HashiCorp Vault in several ways using Helm, a Kubernetes deployment tool:

• • Use Vault directly with Kubernetes as a certificate management tool

• • Install Vault to your managed Kubernetes provider – Amazon Elastic Kubernetes Service (Amazon EKS) or Google Kubernetes Engine (GKE)

You can use Vault Agent to transfer secrets into pods or CSI to inject secrets into Vault. For authentication and audit logs, you are free to use any agent.

HashiCorp Vault is designed to simplify secrets management through user-friendly UI, and it can be accessed through a few simple steps:

• • Install and deploy Vault UI on the development server

• • Connect to Vault UI and use the token to log into the dashboard

• • Set up a secrets engine from the available options

HashiCorp Vault is designed for Kubernetes, which means it can be integrated with AKS, EKS, or any other managed Kubernetes service with ease.

Security

HashiCorp Vault serves secrets into the Kubernetes cluster and manages user validation. It also allows you to dynamically inject secrets through its APIs, enabling you to follow secure practices like efficient certificate management.

Although Vault is robust in terms of security and encryption, there is one challenge that users may face – pluggable architecture. When you add third-party components, you expand the attack surface area of your system. A vulnerability or misconfiguration at any place can expose sensitive data. The most secure way of using Vault is through the HashiCorp Cloud Platform. You can, however, combine it with a highly interoperable platform like Entro that manages secrets like assets.

Using Azure Key Vault for managing Kubernetes secrets

Azure Key Vault

Azure Key Vault is a cloud-based secrets management solution from Microsoft to help its Azure cloud platform customers secure sensitive data and applications. For Azure organizations, Azure Key Vault provides a secure place to store their cryptographic keys and other secrets they use across Azure services. In addition to secrets management, it allows companies to control encryption keys and manage TSL/SSL certificates.

Source: Stack Overflow

With Azure Key Vault, you don’t need the in-house expertise of HSMs (Hardware security modules) to manage keys, certificates, and other secrets. Its core features are discussed below.

• • Key Vault can streamline secrets management for all your cloud services and apps. It helps enhance your security posture by eliminating the need to store credentials in the app’s code.

• • It controls access to secrets through proper authentication and authorization through Azure RBAC or vault access policies. It validates a user’s identity when they request access to the data, and verifies their privileges.

• • The tool allows logging to help you track who is accessing what keys and secrets for diligent monitoring. You can further control access to these logs and even terminate them. You can gain complete visibility into the Kubernetes cluster with a cloud-based security solution like Entro that gives you full context of the secrets.

• • It ensures your secrets are highly available and automates certification deployment.

• • You can segregate your application secrets so that they access only the data they need for particular workloads. The Azure solution enables you to create multiple vaults to create application-based secrets storage for a better developer experience.

• • Being a part of the Azure ecosystem, Key Vault can be integrated with other Azure solutions like Azure Disk Encryption.

Steps to manage Kubernetes secrets with Azure Key Vault

Azure Key Vault easily connects with an Azure Kubernetes Service (AKS) cluster using a CSI (Container Storage Interface) volume. Once stored in the vault, you can only access them within the AKS cluster.

• • Use the command line tool or Azure portal to create a vault and store secrets.

• • Integrate Key Vault with AKS, using CSI Driver, to allow the clusters to retrieve secrets, keys, and certificates from AKV. These secrets can then be appended to pods.

• • Set up authentication permissions allowing roles to enter the Key Vault and access the data.

• • Configure the secret provider class to store all the secrets from the Key Vault.

• • Create a pod manifest using a CSI volume to house the secrets via the latest secret provider class.

Using Azure Key Vault ensures your secrets are managed securely. This is because AKS only accesses secrets when the associated pods are active. The secrets are eliminated upon termination of pods.

In terms of usage, Azure Key Vault is pretty simple. You can access Key Vault from the search functionality in the Azure portal after creating your account. You can manage end-to-end operations once you land on the Key Vault dashboard. Following this, you can

• • Create a Key Vault and assign a name

• • Call the Vault to assign permissions in AKS

• • Create a Secret Provider Class and pods to inject secrets

• • Use Azure CLI to code permissions

Azure Key Vault is designed for the AKS cloud, which begs the question if it can be used outside of the Azure ecosystem. While it is possible to integrate Key Vault with other Kubernetes platforms, it is a hassle and includes dependency on third-party tools. If your organization’s cloud estate is Azure-centric, this is an ideal option.

Security

Azure Key Vault offers a range of benefits to improve your security posture, such as audit logging. Azure Key Vault allows you to create application-specific vaults for greater control over your secrets and who has access to them.

HashiCorp Vault vs. Azure Key Vault: Deciding between the two

We have explored two leading secrets management tools – HashiCorp Vault and Azure Key Vault – and have evaluated both of them minutely. We’ve seen that both tools are a big improvement over Kubernetes’ default secrets manager. Let’s quickly understand the pros and cons of each of them.

HashiCorp Vault:

Pros:

• • It excels at creating secrets

• • It’s great for managing who can access which secrets for how long.

• • It has an open source version that you can use to dip your toes before taking the plunge.

Cons:

• • It increases your dependency on external components.

• • It doesn’t include advanced capabilities to monitor secrets end-to-end in places such as Jira, Slack, and more.

• • It does not provide context about an exposed secret, such as which resources the secret has access to, and the risk level of the exposure.

• • It does not check if your secrets have been leaked across the darkweb or in recent data breach.

Azure Key Vault:

Pros:

• • It will give you an infrastructure that allows you to write your own revocation scripts

• • It is well-integrated with the Azure platform and all its services.

Cons:

• • It caters primarily to Azure customers and falls short if you go beyond the boundaries of Azure.

• • It is not built to monitor every aspect of the lifecycle of a secret. As a result, it leaves gaps in how secrets are monitored.

• • It is not future-proof, and as your organization’s tech stack becomes more complex with non-Azure services in future, Azure Key Vault may not be able to manage the complexity.

• • It does not provide context on any exposed secret, and does not give you steps to take to respond to an exposure.

Regardless of which vault you use, it is necessary to have end-to-end visibility into every secret. That requires a comprehensive security platform like Entro that gives you complete control over your secrets. Learn more about Entro and bolster the security of your vault, whether it’s Hashicorp Vault, or Azure Key Vault.