Windows 7 Compatibility Testing (Part 3)

If you would like to read the other parts in this article series please go to:


In my previous article in this series, I showed you how to perform the initial configuration of the Application Compatibility Toolkit. In this article, I want to continue the discussion by showing you how to create a data collector package for the Application Compatibility Toolkit.

Creating a Data Collector Package

As you may recall from Part 1, a data collector package is a package containing an inventory collector component and one or more compatibility evaluators.

To create a data collector package, begin by opening the Application Compatibility Manager. The Application Compatibility Manager is the Application Compatibility Toolkit’s primary management interface. When the Application Compatibility Manager opens, click on the Collect option, shown in Figure A.

Figure A: Click on the Collect option

At this point, Windows will open the New Package dialog box. The first thing that you have to do is to provide the Application Compatibility Manager with a name for the package that you are creating. After doing so, make sure that the Deploying a New Operating System option is selected, as shown in Figure B.

Figure B: Take a moment to verify that the Deploying a New Operating System option is selected

Now, click the Advanced button, and Windows will display the Advanced Settings dialog box, shown in Figure C. As you can see in the figure, this is where you can choose which individual evaluators that  you want to include in the data collector package that you are creating. Normally, if you are trying to evaluate application compatibility as it relates to an operating system upgrade, then you should leave the default options selected, but you can make changes if you choose to do so.

Figure C: Select the compatibility evaluators that you want to include in your data collector package

When you click OK, Windows will return you to the dialog box that is shown in Figure B. You must now set the duration for your compatibility testing. The Application Compatibility Toolkit is able to compile an inventory of the applications that are installed on each computer fairly quickly, but the process of analyzing applications takes some time. After all, the compatibility evaluators are designed to monitor applications in real time while they are running to look for potential compatibility issues.

As such, it is necessary to set the duration of your compatibility testing in a way that ensures that most, if not all, of the user’s applications will have been used during the testing period. The compatibility testing is configured by default to monitor application usage for three days. In my opinion, this isn’t a long enough period of time; especially if you deploy the package just before the weekend or if your organization uses a lot of different applications.

Once you have decided how long you want to monitor applications for, you must choose how frequently you want to upload the resulting data. By default, the package is configured to upload any collected data every eight hours. My advice is to either keep the default setting or to upload the collected data even more frequently. This is because the more time that passes between collection periods, the greater the chances that a workstation hard drive failure could wipe out the data before it can be uploaded.

The last step in preparing the data collector package is to provide the package with an optional label. If you choose to use a label then it will be appended to all recorded data for identification purposes.

When you finish adjusting the various settings, click the Save icon. When you do, Windows will prompt you for a location in which to save the newly created data collector package. When the process completes, you will see your data collector package listed within the Application Compatibility Manager, as shown in Figure D.

Figure D: The Application Compatibility Manager displays your data collector package

Deploying the Data Collector Package

Now that you have created a data collector package, it must be deployed to the computers on your network. There is nothing magical about the deployment process. As you saw in the previous figure, the data collector package is an .MSI file. As such, it can be manually deployed, or you can deploy it using group policy settings or network management software. However, the way in which you deploy the data collector package is not nearly as important as deciding on which computers to deploy the package to.

Technically, you could deploy the data collector package to every workstation that you are considering upgrading to Windows 7. While this is probably a good approach to take in organizations that have less than a hundred workstations, Microsoft generally recommends that you deploy the data collector package to a subset of the computers that you are considering upgrading, rather than to all of them. As such, the million dollar question becomes a matter of which workstations should you choose?

Believe it or not, choosing the PCs on which to deploy the data collector package can be a really tedious process, because there are a lot of different factors that you will have to consider.

One of the first factors to consider is the computer’s role. As IT professionals, we talk a lot about server roles. This same concept can also be applied to workstations. In other words, users who perform similar job functions probably have similarly configured PCs. For example, there is a good chance that most of the computers in the Finance department are configured in a consistent manner. The same might also be true for the computers in the HR department.  If you are able to identify specific workstation roles (even if those roles have never been formally defined) then it’s a good idea to make sure that at least two computers performing each role are included in the testing process.

A lot of administrators assume that when they are selecting the computers that should be tested for Windows 7 compatibility, they should also base the testing on the user’s job function. I believe that this philosophy is correct, but for the wrong reasons.

The generally accepted school of thought is that any time you are going to do any kind of testing or pilot software deployments that involve workstations, you should pick users with the least important job functions. That way, if something goes wrong you don’t impact critical business processes.

While I will be the first to admit that this is usually a good philosophy to follow, it does not really work for compatibility testing. Remember that the data collection package examines applications as they run to see how those applications interact with the operating system. As such, it is important to deploy the data collector packages on workstations belonging to the busiest people. The more heavily the test PCs are used, the more reliable your test results will be. Besides, the data collector package is generally stable and non disruptive, so there is little risk of interrupting the user’s work.


In this article, I showed you how to create a data collector package by using Microsoft’s Application Compatibility Toolkit, and I talked about the process of selecting PCs as candidates for compatibility testing. In Part 4 of this series, I want to spend some more time talking about the workstation selection process, particularly with regard to computers that are running Windows XP.

If you would like to read the other parts in this article series please go to:

About The Author

Leave a Comment

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top