Analyzing Exchange Logs with Azure Log Analytics (Part 2)

If you would like to read the other parts in this article series please go to:

• Analyzing Exchange Logs with Azure Log Analytics (Part 1)
• Analyzing Exchange Logs with Azure Log Analytics (Part 3)
• Analyzing Exchange Logs with Azure Log Analytics (Part 4)

Signing up for Log Analytics

We have seen how we can subscribe to Log Analytics using the Operations Management Suite website and a free data plan. However, if we already have an Azure subscription, we can use the Azure portal instead:

1. Navigate to the Azure portal and sign in;
2. Browse the list of services, and then select Log Analytics (OMS):

3. As this is the first time we are using Log Analytics, there is nothing to be displayed:

4. Click Add, complete all the required fields in order to create our first OMS workspace, and press OK:

5. Once the deployment has completed, and assuming all went well, we should have our newly created workspace listed:

6. If we click on the workspace, we access Log Analytics (remember you can have multiple workspaces):

From here we can see details regarding Log Analytics for this particular workspace, such as daily usage, data sources, etc. Clicking the OMS Portal link opens the Operations Management Suite website (like when we created our first workspace in the previous article) from where we can also manage Log Analytics.

And now we are finally ready to start using Log Analytics! 🙂

Connecting Servers to Log Analytics

In order to connect servers in our on-premises infrastructure directly to OMS workspace(s), we need to install a customized version of the Microsoft Monitoring Agent (MMA). This agent will send data to OMS and allow us to view and act on that data. Each agent can report to one or multiple workspaces.

Connecting computers to OMS is very easy. Let us start with a small on-premises lab comprised of one Exchange server, one Active Directory Domain Controller and one Windows 10 workstation.

1. In the Log Analytics (OMS) section, in the Azure portal, select your workspace and then select CLICK HERE to connect to a datasource to get started:

2. In the Quick Start windows, select Computers:

3. Since we want to install the agent in on-premise machines, select Computers;
4. Under Direct Agent, click Download Windows Agent (64-bit). This will start the download for the Log Analytics agent:

5. On the right of Workspace ID, click the copy icon and paste the ID into Notepad. Do the same for the Primary Key;
6. Run Setup to install the agent on the computer(s) you want to manage;
7. On the Welcome page, click Next:

8. On the License Terms page, read the license and then click I Agree:

9. On the Destination Folder page, change or keep the default installation folder and then click Next:

10. On the Agent Setup Options page, we can choose to connect the agent to Azure Log Analytics (OMS), Operations Manager, or we can leave the choices blank if we want to configure the agent later:

11. Click Next. Since we chose to connect the agent to our OMS workspace, we paste the Workspace ID and Workspace Key (Primary Key) we copied into Notepad and then click Next:

12. If we click in Advanced, we can configure proxy settings if the agent needs a proxy to be able to communicate with OMS:

13. On the Microsoft Updates page, select the desired option regarding window updates and click Next:

14. On the Ready to Install page, we review our choices and then click Install:

15. On the Configuration completed successfully page, click Finish:

When complete, the Microsoft Monitoring Agent appears in Control Panel:

We can review our configuration here and verify that the agent is connected to Log Analytics. When connected to OMS, the agent displays a message stating: The Microsoft Monitoring Agent has successfully connected to the Microsoft Operations Management Suite service:

Under services there will also be three new MMA services:

If have installed agents but did not configure them, or if you want those agents to report to multiple workspaces, we can enable an agent or reconfigure it through the Control Panel:

1. Open Control Panel;
2. Open Microsoft Monitoring Agent and then click the Azure Log Analytics (OMS) tab;
3. Click Add to open the Add a Log Analytics Workspace box;
4. Paste the Workspace ID and Workspace Key (Primary Key) we copied into Notepad in a previous procedure for the workspace that we want to add, or add the details of an additional workspace, and then click OK:

The easiest way to install an agent on Azure virtual machines is using the Log Analytics VM Extension. This greatly simplifies the installation process and automatically configures the agent to send data to the Log Analytics workspace that we specify. The agent will also be upgraded automatically.

For Windows virtual machines we enable the Microsoft Monitoring Agent virtual machine extension, and for Linux virtual machines, the Oms Agent For Linux virtual machine extension.

There are three ways to enable the Log Analytics virtual machine extension:

• By using the Azure portal (the method I will show here);
• By using Azure PowerShell;
• By using an Azure Resource Manager template.

To install the Log Analytics agent and connect the virtual machine to a Log Analytics workspace using the Azure portal:

1. Sign into the Azure portal;
2. Select Browse on the left side of the portal, and then go to Log Analytics (OMS) and select it;
3. In your list of Log Analytics workspaces, select the one that you want to use with the Azure VM;
4. Under Log analytics management, select Virtual machines:

5. In the list of Virtual machines, select the virtual machine on which you want to install the agent (make sure the virtual machine is not in a Stopped (deallocated) state). The OMS connection status for the VM will indicate that it is Not connected:

6. In the details of the virtual machine, select Connect. The agent is automatically installed and configured for your Log Analytics workspace. This process will take a few minutes, during which time the OMS Connection status will be Connecting…:

7. After you install and connect the agent, the OMS connection status will be updated to show This workspace:

That’s it! Within a couple minutes our Azure VM is sending data to OMS.

OMS Portal

We have now connected three servers and one workstation to our Log Analytics (OMS) workspace. Although through the Azure portal we can perform a certain level of configuration and even search/analyze the logs collected, this portal is still somewhat limited, so let us move to the OMS portal instead by clicking any of the following two links to it or by navigating to the Operations Management Suite website:

Once in the OMS portal we are presented with a practically empty dashboard since we haven’t yet configured any data to be collected. All the dashboard tells us is that we have 4 machines connected and have completed one third of the configuration so far:

So let us continue configuring Log Analytics. To do that, the easiest way is to click on Get started which presents us with a useful wizard:

We can see that step 2. Connect a data source is green because we have already connected machines to Log Analytics using the Azure Portal in the previous article.

The first step in the wizard helps us to add Solutions. But let us leave the default options selected as we will dive into Solutions in more detail later in this article series when we look at Office 365. Click Add selected solutions:

Below are the installed solutions we have just selected. Later we will be installing additional ones. Actually, we are not using Azure Automation or Azure Site Recovery, so we can easily remove these by clicking the Remove text next to them:

If we click in Connected Sources we can see that we have our four machines connected to Log Analytics, and zero Office 365 subscriptions (this is what we will be adding later). We can also download the agent from here in order to connect additional servers:

Step 3. Add logs of the wizard is where we will configure which logs to collect data from. So let’s have a look at Data Sources in Log Analytics.

Data Sources

Log Analytics will use the agents we installed to collect data from those Connected Sources in our OMS workspace and stores that information in the OMS repository as a set of records. The data that is collected from each is defined by the Data Sources we configure. Each data source creates records of a particular type with each type having its own set of properties.

The following are the data sources that are currently available in Log Analytics:

To configure data sources, we either go to step 3. Add logs of the wizard, or we use the Data menu in Log Analytics Settings. A limitation of Log Analytics at this stage, in my opinion, is that any configuration is delivered to all connected sources in our OMS workspace. We cannot currently exclude any agents from this configuration, meaning that if we decide to collect Warning and Information events from the Windows System event log, we will be collecting these events from all Windows machines… But I am sure (hopeful) this will change soon.

Data source configurations are delivered to agents that are directly connected to OMS within a few minutes. The specified data is collected from the agent and delivered directly to Log Analytics at certain intervals specific to each data source.

Windows Event Logs

These are the most common data sources used for Windows agents since this is the method used by most applications to log information and errors. We can collect events from standard logs such as System and Application in addition to specifying other custom logs created by applications we need to monitor such as Exchange.

Log Analytics will only collect events from the Windows event logs that are specified in the settings. We can add a new log by typing in the name of the log and clicking +. If we want to start collecting data from the Application event log for example, we start typing application in the search box and Log Analytics will auto-complete with all the logs that match our search criteria:

We then select Application and click the + plus icon to add that event log to our collection:

A drawback here is that we cannot right-click to select multiple logs, we need to search and add them one by one…

Once we have selected the log(s) we want to collect data from, we need to choose the severity of the events we want to collect. For each log, only events with the selected severities will be collected.

If we search for Exchange there seems to be only two logs available…

But not to worry! All we have to do is type the name of the log we want and the agent will gather its events! Let’s say we want to capture events from the Managed Availability’s Monitoring log:

All we need to do is type “Microsoft-Exchange-ManagedAvailability/Monitoring” in the search field and click the + plus icon. Log Analytics will now start capturing data from this log! Easy 🙂

Log Analytics will collect each event that matches a selected severity from a monitored event log as the event is created. The agent will record its place in each event log that it collects from. If the agent goes offline for a period of time, then Log Analytics will collect events from where it last left off, even if those events were created while the agent was offline.

Performance Logs

Performance Counters in Windows and Linux provide insight into the performance of hardware components, operating systems and applications. Log Analytics can collect performance counters at frequent intervals as well as aggregating performance data for longer term analysis and reporting.

When we first configure Windows or Linux Performance counters for a new OMS workspace, we are given the option to start collecting data from several common counters. We select the ones we want to monitor and click Add the selected performance counters:

The same way we added Windows event logs, we can collect data from additional counters by typing the name of the counter in the text box in the format object(instance)\counter. We can also return all instances for a particular counter by specifying object\counter instead.

Let’s say we want to gather data from the Exchange RPC Client Access – User Count performance counter. If we type Exchange no counters will be returned, so we have to type “MSExchange RpcClientAccess(*)\User Count” and click the + plus icon to add it to our list.

When a counter is added, it will use the default of 10 seconds for its Sample Interval. We can change this to a higher value of up to 1800 seconds (30 minutes) to reduce the storage requirements of the collected performance data.

The following screenshot shows all the performance counters I am currently monitoring with Log Analytics, including a few ones specific to Exchange:

Log Analytics will collect data from all performance counters at their specified sample interval on all agents that have that counter installed. It is important to note that all collected performance data is aggregated at 30 minute intervals.

A rough estimate for collection of a particular counter at 10 second intervals is about 1 MB per day per instance. We can estimate the storage requirements of a particular counter with the following formula:

1MB x (number of counters) x (number of agents) x (number of instances)

Internet Information Services (IIS)

Log Analytics can collect entries from log files created by IIS (but only in W3C format), so we must configure IIS for logging (which in Exchange it is by default) and select the fields we want logged. IIS does not log all fields by default, so you may want to manually select additional fields beyond the default.

In order to start collecting IIS logs, there is no configuration required other than selecting Collect W3C format IIS log files:

Log Analytics will collect IIS log entries from each connected source approximately every 15 minutes.

The last Data Source we are interested in are Custom Logs. Exchange has countless logs so in the next part we will see if we can import these into Log Analytics.

Wrapping it up…

In this second part of this article series, we signed up to Log Analytics using the Azure portal, saw how to connect our servers to Log Analytics, had a quick tour of the OMS Portal, and went through some different data sources we can collect using Log Analytics. In the next part we will see if we can collect data from other sources such as Exchange Message Tracking Logs, and look at searching and analyzing the data we have been collecting.

If you would like to read the other parts in this article series please go to:

• Analyzing Exchange Logs with Azure Log Analytics (Part 1)
• Analyzing Exchange Logs with Azure Log Analytics (Part 3)
• Analyzing Exchange Logs with Azure Log Analytics (Part 4)