Passwordless integration tests in an Azure DevOps pipeline.

Recently I blogged about passwordless Service-to-Service calls with Azure Managed Identities and how to make them testable from your local devbox (still with no passwords/secrets directly involved). I also blogged on how to make those calls from inside your integration tests, that run on an arbitrary build agent in Azure DevOps. That works equally well, but requires a dedicated Azure Service Principal, a secret for it and a bit of a configuration hassle. So now the remaining piece of the puzzle is how to make those integration tests work with no Service Principals and no passwords/secrets. Is that at least possible?

Well, not on an arbitrary build agent, because obviously the underlying machine will have no identity to inherit an access token from. But you can install an Azure DevOps build agent on your devbox (or on a dedicated VM) and let your tests run on it! Still as part of your Azure DevOps Release pipeline.

Before going into implementation details, I’ll quickly recap the scenario. I assume, that you have a RESTful endpoint hosted in Azure - as an App Service or as a Function App - and protected with EasyAuth and Microsoft Identity Platform (aka requiring and accepting OAuth 2.0 access tokens generated by https://login.microsoftonline.com). Let’s call this endpoint the-callee. Types of tokens normally accepted by the-callee in production might be app-only, or user-specific, or both. But what you would like to achieve is to call the-callee from your integration tests without specifying any secrets/passwords/keys/certificates, yet also without any user interaction.

Now, for making those calls you first need to obtain a token, and the recommended way of doing that (provided that your integration tests are written in C#) is by utilizing AzureServiceTokenProvider from Microsoft.Azure.Services.AppAuthentication library:

var tokenProvider = new AzureServiceTokenProvider();
string accessToken = await tokenProvider.GetAccessTokenAsync("<CalleeResourceId>");

, where CalleeResourceId is the Application ID URI of your the-callee’s AAD App. On a VM inside Azure this code will normally utilize the assigned Managed Identity. While when being executed on your devbox (or on an arbitrary machine) it can try to impersonate you, or any other particular user, who is currently logged in to Azure CLI. For that to work, a bit of extra configuration is needed, which I already described earlier, but I’ll briefly mention those steps here as well:

1. Ensure you’re logged in with a proper account into Azure CLI on your devbox:

az account show

2. Go to the-callee’s AAD App Registration page, choose Expose an API tab and add Azure CLI to the list of Authorized Client Applications (the Azure CLI’s Client ID is 04b07795-8ddb-461a-bbee-02f9e1bf7b46):

3. Find the-callee’s AAD App in AAD Enterprise Applications, go to Users and Groups:

and whitelist yourself (if you’re not whitelisted there yet):

Give it ~10 minutes to propagate and then validate that both access token generation and the actual service calls succeed. You can also check the access token internals (with e.g. https://jwt.io), to make sure that your name is mentioned there.

And now you can install and configure Azure DevOps Agent on your devbox. That process is pretty well explained on MSDN, so I won’t repeat it here, but the important bit is to configure the agent to run in service mode, yet under your user account (or the one, which has ever logged in into Azure CLI on this particular machine). When your custom agent pool (with your local agent in it) is up and running, you can run your Pipeline on it:

and observe (via Task Manager) it being executed on your devbox. And access token generation works exactly in the same way as above (when you run integration tests locally) - by re-using the credentials stored by Azure CLI. Still with no user interaction and no stored passwords/secrets. Voila!

As a conclusion, let me try to answer some of those many questions you might be having so far.

• How is that safe? Well, it is definitely safer than spreading a powerful shared secret across each and every devbox and/or build machine. Tokens are derived from a particular user account - the one which is logged in into Azure CLI - and on another devbox it can and should be a different account. That account’s credentials are also not stored, only Azure CLI’s authentication refresh token (normally lasts for 90 days).

• How is that passwordless, when Azure DevOps Agent config tool asks for my account’s password? Yes, but it is not used to produce tokens, only to run the agent’s process with it. Windows Service Control Manager stores that password securely. And anyway, this doesn’t need to be your personal account - any account with which you can login interactively every 90 days (to refresh Azure CLI’s login) will do.

• Does that mean that an arbitrary user can now obtain access tokens for my service and/or any other service? Absolutely not. At step2 we explicitly allowed Azure CLI to only generate tokens for our service, not for any other one. And at step3 we whitelisted a particular set of users, to be allowed to do so (though, for this last restriction to have full effect, User Assignment Required should be set to Yes for the-callee’s AAD App).

• Can I use a dedicated VM for that, instead of my devbox? Absolutely yes. A dedicated VM with a dedicated weak test user would even be a more preferred way.

• Can those integration tests include user interaction? Yes, but in that case you’ll need to run Azure DevOps Agent in interactive mode, which is less recommended.

• Can I only do the access token generation step locally and run the rest of my pipeline on a regular Microsoft-hosted agent pool? Yes, but you would then need to pass the token between pipeline jobs, which is not a trivial task, as of today.

That was another small victory in our epic battle with passwords, making the dev process a little more secure :)