Managing multiple Azure diagnostic settings using Azure CLI instead of PowerShell

Microsoft Azure has integrated the entire monitor platform in a single location and experience. We explored a little bit of Azure Monitor in a previous article here at TechGenix. Nowadays, we can configure the Azure diagnostic settings on the vast majority of resources using a consistent interface and it can be done from the properties of the given resource, or through the Monitor feature. When using multiple Azure diagnostic settings, we can send the diagnostic data to either Event Hubs, Storage, or Log Analytics.

In this article, you will see that we have a small challenge configuring multiple Azure diagnostic settings using PowerShell, so we are going to understand the issue and use an alternative solution using Azure CLI to solve the current limitation.

Understanding the issue

Usually, before trying automation and using of PowerShell, I like to validate the feature using Azure Portal and after understanding how it works, then I start creating the logic to implement the same settings using PowerShell. Depending on the scenario, we need to add features such as CSV files or add some logic to get the solution implemented.

We can use any resource to start playing with the diagnostic settings feature. In this article, we are going to use Network Security Groups (NSGs), and within the resource properties, we can click on Diagnostics logs and on the blade on the right-side we can add multiple diagnostic settings. The example below has two diagnostic settings configured, where some information goes to Log Analytics and the other one goes to Storage accounts.

azure diagnostic settings

In each diagnostic settings, the cloud administrator is able to select one of the available targets to receive the diagnostic data, which are Storage Accounts, Event Hubs, and/or Log Analytics. In the example below, we have NetworkSecurityGroupEvent logs being sent to an Event Hub called ops.

Note: If we select more than one target within the same diagnostic settings, we are sending the same set of logs to all defined targets. If we want different logs being sent to a specific target, the use of multiple diagnostic settings is the way to go because they provide this flexibility.

azure diagnostic settings

That is great — in theory. We just need to get the PowerShell cmdlet and start configuring the diagnostic settings for any given resource, right? Well, here is where the problem lies. At this stage, the PowerShell cmdlets are using an old API version that does not allow multiple diagnostic settings to be managed/manipulated using PowerShell.

For illustration purposes, let’s say that we want to add diagnostic settings using the cmdlet below. We will realize that we cannot define a name, which means that any PowerShell configuration will always go to a diagnostic setting called service — and it does not allow us to create additional ones.

Set-AzureRmDiagnosticSetting -ResourceId '/subscriptions/xxx/resourceGroups/AP-RG-Diag/providers/Microsoft.Network/networkSecurityGroups/NSGTest6' -Enabled $True -Categories "NetworkSecurityGroupEvent" -StorageAccountId '/subscriptions/xxx/resourceGroups/AP-RG-Diag/providers/Microsoft.Storage/storageAccounts/apstgenterprises'

azure diagnostic settings

I mentioned the API version being the culprit for the current issue. You may be asking, how do I know which API my PowerShell cmdlet is using at any given time? That is easy. Just add -debug and it will tell you what is going on behind the scenes. In the example below, we can see that the API version in use is 2016-09-01.

azure diagnostic settings

If you search the provider in Microsoft documentation, we see that the current API version for that same provider is 2017-05-01-preview. You can check it out here.

This is kind of a dead end when trying to use PowerShell to manage multiple diagnostic settings. However, we have several interfaces to manage Azure. Azure CLI to the rescue! My next attempt is using Azure CLI (cross-platform command line), which runs on macOS, Linux, Windows, and in the Azure Cloud Shell.

My first attempt was to read the diagnostic data from any given resource (in our case, the same NSG that we have been using to keep consistency on the results among interfaces), and when using Azure CLI we noticed that the output of the command below brought all the configured settings, which is a good sign.

az monitor diagnostic-settings list --resource '/subscriptions/xxx/resourceGroups/AP-RG-Diag/providers/Microsoft.Network/networkSecurityGroups/NSGTest6'

Just to be safe, let’s add debug to the same command and check which API version is in use. To my surprise, it is the latest one, so we are good to use this interface to manage multiple diagnostic settings.

azure diagnostic settings

Managing Azure diagnostic settings using Azure CLI

Now that we have settled on the interface for this task, we can explore the commands available that will help us manage Azure diagnostic settings using Azure CLI. One difference between PowerShell and Azure CLI is that the log configuration in PowerShell is done through parameters in the command line. In Azure CLI, however, they are JSON code. A good hint is to configure it first in Azure Portal, then list the resource in Azure CLI. The output will provide the JSON file that you have to use when defining the diagnostic settings. Compare that output with the parameter that you are specifying and they should be the same.

Any configuration will require the ResourceID for the object as a parameter, using Azure CLI we can run az resource list –name <name> and the output will be a JSON format of the given resource.

azure diagnostic settings

Having the ResourceID of any given object makes it easier to retrieve the diagnostic settings. The following command shows how to retrieve diagnostic setting information.

az monitor diagnostic-settings list --resource '/subscriptions/xxx/resourceGroups/AP-RG-Diag/providers/Microsoft.Network/networkSecurityGroups/NSGTest6'

If you need to add a diagnostic setting using Event Hubs, here is the format that we can use to move the logs related to AuditEvent to an Event Hub called ops. The name of the diagnostic settings is DiagEventHub. All items in bold are static and need to be added as part of the command.

az monitor diagnostic-settings create -n DiagEventHub --resource '/subscriptions/xxx/resourceGroups/RGName/providers/Microsoft.KeyVault/vaults/KeyVault000000' --event-hub-rule '/subscriptions/xxx/resourceGroups/RG-EH/providers/Microsoft.EventHub/namespaces/EHName/authorizationrules/RootManageSharedAccessKey' --event-hub ops --logs '[{"category":"AuditEvent","Enabled":true}]'

If we want to add a diagnostic setting to Log Analytics, this following command can be used:

az monitor diagnostic-settings create -n DiagLogAnalytics --resource '/subscriptions/xxx/resourceGroups/RG-Name/providers/Microsoft.KeyVault/vaults/KV0000' --workspace '/subscriptions/xxx/resourcegroups/RG-OMS/providers/microsoft.operationalinsights/workspaces/LogAnalytics000' --logs '[{"category":"AuditEvent","Enabled":true}]'

If you want to configure a Storage Account instead, we can use the following command to a specific Storage Account. In this example, we are configuring an additional diagnostic setting called DiagStorage. One thing that can be configured with Storage Account is the number of days to retain the information (that information can be found in the last part of the command).

az monitor diagnostic-settings create -n DiagStorage --resource '/subscriptions/xxx/resourceGroups/RGName/providers/Microsoft.Network/networkSecurityGroups/NSGTest6' --storage-account '/subscriptions/xxx/resourceGroups/RGName/providers/Microsoft.Storage/storageAccounts/storageaccountname' --logs '[{"category":"NetworkSecurityGroupRuleCounter","Enabled":true,"retentionPolicy":{"days":"6","enabled":true}}]'

Trial and error

Microsoft Azure is dynamic and constantly receiving updates and new features, and sometimes we need to play around with the available interfaces to get the desired solution implemented now instead of waiting for an update.

This article illustrated a simple scenario where the PowerShell cmdlets available were not using the latest API version, but we were able to finish the task to enable multiple Azure diagnostic settings using Azure CLI.

Leave a Comment

Your email address will not be published.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top