Azure Key Vault: Everything You Need to Know

Simplifying storage and easy retrieval of sensitive data using Azure key vault.

The Basic Question — What Is Azure Key Vault?

In simple words, azure key vault is a feature provided by Microsoft in Azure where we can store sensitive information such as passwords (which in azure are called secrets), certificates and encryption keys.

So, is this similar to a safe box where people store their important items?
Yes, but it’s a safe box managed by Microsoft for you in their cloud system.

How Can Azure Key Vault Help?

From my personal experience, below are some of the areas where having a key vault has helped my team —

1. Since my data engineering team has different data engineers who work on different projects, I use key vault to store all of our projects secrets (passwords) in one place.
2. As key vault is part of Azure, I was easily able to read the secrets out of the key vault for our various data pipelines without needing to expose them anywhere in our code.
3. In my team, only the senior data engineers and/or architects need to have access to the secrets but not everybody else. Azure’s key vault helps to establish and maintain the access control through IAM (Identity and Access Management).

What Are Its Advantages?

So now we understand what a key vault is and how it can help a data team. If you need to summarize its advantages and talk about them with your team, use the below points:

1. Data security is maximized because key vault provides a centralized location for your sensitive information.
2. Since it has a seamless integration with azure services and applications, key vault can be conveniently called in our data pipelines to retrieve the secrets.
3. We can maintain different versions of the password without the need to re-create a new line item in key vault. As an example, if you work with an API key that gets refreshed every week, key vault provides the feature to maintain multiple versions of the key with the latest one being on top. Azure services can fetch the latest key without any hassle.
4. Key vault is a scalable service. This means it does not face any throttling if multiple services try to access the secrets and the volume of requests increase.

But, before we continue further, it does not only have advantages.

What Are Its Disadvantages?

1. It’s not free. It’s a paid service and the cost is formed on a variety of factors such as number of keys and secrets stored, number of operations performed etc.
2. A bit of a learning curve is involved if your team has not worked with key vault before. However, from my personal experience, it is a fairly easy service to familiarize with.
3. Using key vault introduces dependency on Azure. If your team is already working with Azure cloud platform, key vault makes for an excellent choice as a safe storage. However, if your team is working with different cloud systems, storing secrets in Azure’s key vault might include unnecessary complexity in working with cross cloud systems.

A little elaboration on pricing

💲💲 matters. When a new service is explored, money inadvertently can be the deal breaker which is why I wanted to provide an example for the 1) above.

I will be using Azure Pricing Calculator for this example. Please note that the key vault is offered in two flavors — Standard and Premium — but for this exercise, I will pick standard option. In a majority of scenarios, standard option should work perfectly for your solutions.

As a use case, let’s say my data engineering team has a data pipeline that processes large volumes of data. It reads the data from databases and APIs. It performs the following operations on a monthly basis —

100,000 read operations- fetch secrets from key vault

50,000 write operations — store new secrets in key vault

10,000 advanced key types operations — generating/importing new keys in key vault

The cost for the standard key vault as per Microsoft is shown below —

My cost can be calculated as below —

Read Operations Cost

Cost per 10,000 secret read operations = $0.03
Total cost for 100,000 secret read operations = $0.03 * 10 = $0.30

Write Operations Cost

Cost per 10,000 secret write operations = $0.03
Total cost for 50,000 secret write operations = $0.03 * 5 = $0.15

Key Operations Cost

Cost per 10,000 key operations = $0.15
Total cost for 10,000 key operations = $0.15 * 1 = $0.15

Total Cost — $0.6 / month

Disclaimer— Above example is based on a simplified situation to give you an understanding about how costs get incurred. It may not reflect the actual pricing at the time of your inquiry because key vault’s pricing can vary based on factors such as the region, the specific pricing tier chosen, and any applicable discounts or agreements.

Who should be responsible for Key Vault?

My recommendation is to follow the principle of least privilege. Access to key vault should be granted only to those who need it. Ideally, only a senior person on the team such as an architect or a lead data engineer should have the necessary permissions to view/manage secrets in key vault.

To control access, use access control (IAM) on the key vault page to grant access to a user or an application.

Monitoring a Key Vault

Organizations that use key vault often want to monitor and/or audit its usage. We can enable logging to store logs of the key vault’s activity. This serves as an important feature for the security and compliance teams due to the following reasons —

1. it helps the team to check if any unauthorized attempts were made to access the key vault.
2. during a security breach, logs provide the black-box evidence of the sequence of events, actions that were taken and vulnerabilities in the system.
3. logs can provide insights into how the key vault is being used, any errors that were incurred by a user and if there are areas for improvement in the key vault process.

To enable logging on key vault, go to diagnostic settings > Add diagnostic setting > check audit and allLogs under category > send to log analytics workspace and/or archive to a storage account. We also have the option of streaming the events happening at key vault to a event hub if there’s a use case for those metrics.

Conclusion

I hope I was able to simplify what azure key vault does, its advantages and disadvantages, relevant use cases and more. My idea behind writing the articles is to personalize them by giving examples of how I have used the technologies I write about.

If you are using logic apps in your projects and curious to know if logic apps can read/fetch information from a key vault, the answer is yes. Please feel free to check out my below article —

Azure — Fetch Key Vault Secrets Using Logic Apps