In my last post, I described the process I followed to build, run tests and produce coverage results with .NET 4 and TFS Build 2008. It fell short of showing coverage results in the build summary in Visual Studio (specifically Visual Studio 2010). I only got as far as copying the coverage report to the build drop folder.
This isn’t good enough. Practically, another developer would view the build summary in Visual Studio, see the test results but not see any coverage report. This person would most likely assume that no coverage report has been generated, get frustrated and move on. For the coverage report to provide any value to the development team, it needs to not stay isolated in the build drop folder. It needs to be seen. I saw two places for the coverage results to go to make it’s integration with Visual Studio near seamless: in the build summary and with the test results.
1) The first place the coverage results need to go is on the build summary. Using TFS 2008, .NET 3.5 and Visual Studio 2008, a summary of coverage results are published in the build summary automatically. Since I’m dealing with TFS 2008, .NET 4 and Visual Studio 2010, it doesn’t happen automatically. I’m not sure whether TFS 2008 or Visual Studio 2010 is to blame for this breakdown, but that doesn’t seem important. The solution I’m using is to write an extension to the build process to explicitly write the coverage results to TFS. Visual Studio comes with the libraries needed to communicate to TFS. Here is how to accomplish that:
//using Microsoft.TeamFoundation.Build.Client
//using Microsoft.TeamFoundation.Build.Common
//using Microsoft.TeamFoundation.Client
var tfs = TeamFoundationServerFactory.GetServer(teamFoundationServerUrl);
var buildServer = buildServer = (IBuildServer)tfs.GetService(typeof(IBuildServer));
var build = buildServer.QueryBuildsByUri(new[] { buildUri }, new[] { "*" }, QueryOptions.All)[0];
var configSummary = InformationNodeConverters.GetConfigurationSummaries(build)[0];
var coverageSummary = configSummary.AddCodeCoverageSummary();
coverageSummary.BlocksCovered = blocksCovered;
coverageSummary.BlocksNotCovered = blocksNotCovered;
coverageSummary.LinesCovered = linesCovered;
coverageSummary.LinesNotCovered = linesNotCovered;
coverageSummary.LinesPartiallyCovered = linesPartiallyCovered;
coverageSummary.Name = testRunName;
coverageSummary.RunId = testRunId;
coverageSummary.RunUser = runUser;
//workaround for MS assemblies not setting BuildInformationNode.Build
//otherwise there's a NullReferenceException during Save()
var coverageInfoNode = build.Information.GetNodesByType(InformationTypes.ConfigurationSummary)[0].Children.GetNodesByType(InformationTypes.CodeCoverageSummary)[0];
var buildPropertyInfo = coverageInfoNode.GetType().GetProperty("Build", BindingFlags.Instance | BindingFlags.DeclaredOnly | BindingFlags.NonPublic);
buildPropertyInfo.SetValue(coverageInfoNode, build, null);
coverageSummary.Save();
The example code includes a workaround for a NullReferenceException I kept seeing when trying to save coverage summary to TFS. There might be a cleaner way to do it, or there might be some other way to do it; but this works.
2) The second place the coverage results need to go to make the coverage results more tightly integrated with Visual Studio is attached to the test results. When someone clicks “View Test Results” from the build summary in Visual Studio, the coverage results should open with the test results, but that didn’t happen. Instead there was an error displayed in Visual Studio that it couldn’t find one of the reference assemblies similar to how the .coverage file behaves when you open it remotely. Clicking on “View Test Results” opens a test results file (.trx) in the build drop. The first one I saw was <build_drop_folder>\TestResults\<test_run_id>.trx. In the TestResults directory, there is also a folder with the name of the test run ID. This folder contains all supporting files for the test results including the coverage results. The coverage results are typically found in the <build_drop_folder>\TestResults\<test_run_id>\In\<machine_name> directory. The original coverage file (data.coverage) is there, and I also already had the coverage xml (data.coveragexml) copied there.
I looked at the .trx file in notepad, and the first thought that came to mind was that the old coverage file is being opened because it’s referenced in the <ResultFiles> element. Yet when I changed the coverage file references multiple ways, there was no change in how the coverage was loaded. Thanks to the Process Monitor, I found out my assumptions were incorrect in two ways. One, <build_drop_folder>\TestResults\<test_run_id>.trx isn’t the file Visual Studio 2010 is opening; instead, it’s <build_drop_folder>\TestResults\<test_run_id>\<test_run_id>_vs10.trx. Two, Visual Studio pays no attention to the <ResultFiles> element in the .trx file to load the associated coverage. It just searches the sub-directories for coverage files. It finds the data.coverage file before data.coveragexml; so, it tries to load data.coverage and fails.
My solution to loading the coverage with the test results is ultra simplistic. I delete the old data.coverage file from the build drop folder. After that, data.coveragexml is loaded instead. As a result, the coverage results load with the test results when someone clicks “View Test Results” from the build summary without assembly reference errors.
I haven’t found any more places yet where I can integrate the coverage results with Visual Studio 2010. If anyone knows of any more places, please let me know.
Here are some links that have helped me immensely through the whole process:
http://msdn.microsoft.com/en-us/library/aa337604%28v=VS.90%29.aspx
http://blogs.msdn.com/b/phuene/archive/2009/12/01/programmatic-coverage-analysis-in-visual-studio-2010.aspx
http://blogs.msdn.com/b/buckh/archive/2007/08/14/tfs-2008-a-basic-guide-to-team-build-2008.aspx
http://www.woodwardweb.com/dotnet/tfs_build_api_b.html
