Where are We?

In Part 1 I talked about the steps necessary to get a Transport Extension linked into CCTray application.  The ITransportExtension interface handles plugging our Team Foundation Server [TFS] extension into CCTray, but it does nothing to actually monitor builds running within the TFS infrastructure.  Actual server and project integration are handled through the ICruiseServerManager and ICruiseProjectManager interfaces.  In this post, I'll walk through the process of attaching to the build portions of TFS and exposing the results to CCTray.

The Team Foundation Server Client API

Before I get into the details of the CCTray integration, let's take a step back and look at what we need to do.  We really have two main tasks, first we need to retrieve the list of available build projects for a given Team Foundation Server and Team Project.  Microsoft helpfully provides a number of client API libraries for use when integrating with TFS.  To retrieve the list of build projects from a server, we'll utilize the Microsoft.TeamFoundation.Client and Microsoft.TeamFoundation.Build.Client namespaces.  The TeamFoundationServer and TeamFoundationServerFactory classes provide a method to connect into the TFS instance, and the IBuildServer interface provides methods to retrieve information about the builds defined on the server.

  // Instantiate a TeamFoundationServer through the factory
  TeamFoundationServer tfsServer = TeamFoundationServerFactory.GetServer(settings.ServerName);

  // Retrieve the IBuildServer service
  IBuildServer tfsBuildServer = tfsServer.GetService(typeof(IBuildServer)) as IBuildServer;

  // Retrieve the lits of projects available on this build server
  IBuildDefinitionSpec defSpec = tfsBuildServer.CreateBuildDefinitionSpec(Settings.TeamProject);
  IBuildDefinitionQueryResult query = tfsBuildServer.QueryBuildDefinitions(defSpec);

  IBuildDefinition[] definitions = query.Definitions;

Once we have the list of projects in the form of IBuildDefinition instances, we can query for specific details about a particular build, details about all currently running/queued builds, etc.  For example:

  IQueuedBuildSpec queueSpec = tfsBuildServer.CreateBuildQueueSpec(Settings.TeamProject);
  IQueuedBuildQueryResult result = tfsBuildServer.QueryQueuedBuilds(queueSpec);

  IQueuedBuild[] builds = result.QueuedBuilds;

  // Retrieve the build definition
  IBuildDefinition definition = ...;

  // Query for recent historical builds for this definition
  IBuildDetailSpec querySpec = tfsBuildServer.CreateBuildDetailSpec(Settings.TeamProject, definition.Name);
  querySpec.MaxBuildsPerDefinition = 10;
  querySpec.QueryOrder = BuildQueryOrder.FinishTimeDescending;

  IBuildQueryResult queryResult = tfsBuildServer.QueryBuilds(querySpec);
  IBuildDetail[] buildDetails = queryResult.Builds;

Again, we see the combination of building the query specification and executing the resulting query against the build server.  Similar methods exist to query most aspects of the build server, both for active and historical data.  The IBuildDetail interface shown above is the interface through which much of the project specific interactions will be performed.

Now that we've got a little background on what APIs are used for accessing the TFS build information, let's take a look at how we surface that information to CCTray.

The ICruiseServerManager Interface

The first step in surfacing TFS build information to CCTray is to implement the ICruiseServerManager interface.  This interface handles connecting to the TFS server, retrieving the list of available projects and returning a snapshot of the current 'server state' to the CCTray application.  The interface is shown below:

The TFSServerManager class implements the ICruiseServerManager interface and provides the first integration point with TFS.  Our ITransportExtension implementation that was built in Part 1 returns an instance of this class from the ITransportExtension.GetServerManager method.  In the case of TFS, the 'SessionToken' property is unused, as are the CancelPendingRequest and Logout methods.  Technically, the Login method can be skipped as well, since the TFS integration components will utilize the current windows authentication to connect to the TFS server.  To repeat, if you cannot use your current windows login context to login to the TFS server, this extension will not work as described.  Additional work would need to be done to authenticate with the TFS server.

The two key methods on this interface are GetCruiseServerSnapshot() and GetProjectList(), both of which return information about the currently connected server in slightly different formats.  We'll start with GetProjectList() since the implementation is pretty straightforward.  We're going to borrow the code outlined in the TFS API section above to retrieve the set of projects exposed by the connected TFS server and team project, then translate the resulting array of IBuildDefinition instances into CCTrayProject instances.

  public CCTrayProject[] GetProjectList()
  {
      IBuildServer buildServer = TfsServer.GetService(typeof(IBuildServer)) as IBuildServer;
      if (buildServer != null)
      {
          IBuildDefinitionSpec defSpec = buildServer.CreateBuildDefinitionSpec(Settings.TeamProject);
          IBuildDefinitionQueryResult query = buildServer.QueryBuildDefinitions(defSpec);

          if (query != null)
          {
              CCTrayProject[] projects = new CCTrayProject[query.Definitions.Length];

              for (int i = 0; i < query.Definitions.Length; i++)
              {
                  projects[i] = new CCTrayProject()
                                  {
                                      BuildServer = Configuration,
                                      ExtensionName = Configuration.ExtensionName,
                                      ExtensionSettings = Configuration.ExtensionSettings,
                                      ProjectName = query.Definitions[i].Name,
                                      SecuritySettings = Configuration.SecuritySettings,
                                      SecurityType = Configuration.SecurityType,
                                      ServerUrl = Configuration.Url,
                                      ShowProject = query.Definitions[i].Enabled
                                  };
              }

              return projects;
          }
      }
 
      return null;
  }

Once we are returning the list of projects, we can move onto the more detailed GetCruiseServerSnapshot() method.  This method will return not only the current state of all the builds, but also a snapshot of the build queues that are available on the server.  For this extension, I decided to consider each 'Build Agent' to have a queue of its own.  So a server with multiple build agents will display as having multiple queues.  The GetCruiseServerSnapshot code is a little too involved to include in its entirety, and I've covered how to retrieve the list of recent builds for a build definition in the TFS API section, but one interesting feature that I decided to utilize as the ability to 'hook into' a running build and get 'live' status updates.  The relevant code looks something like this:

  private bool AttachToNextBuild(string buildDefinition)
  {
      IQueuedBuildSpec queueSpec = _manager.TfsBuildServer.CreateBuildQueueSpec(_manager.Settings.TeamProject);
      queueSpec.DefinitionSpec.Name = buildDefinition;
      queueSpec.QueryOptions = QueryOptions.All;
      queueSpec.Status = QueueStatus.All;

      IQueuedBuildQueryResult queryResult = _manager.TfsBuildServer.QueryQueuedBuilds(queueSpec);
      if (queryResult != null && queryResult.QueuedBuilds.Length > 0)
      {
          IBuildDetail buildDetail = queryResult.QueuedBuilds[0].Build;
          if (buildDetail != null)
          {
              buildDetail.StatusChanged += new StatusChangedEventHandler(Build_StatusChanged);
              buildDetail.Connect();
              return true;
          }
      }

      return false;
  }

  void Build_StatusChanged(object sender, StatusChangedEventArgs e)
  {
      IBuildDetail buildDetail = sender as IBuildDetail;
      if (buildDetail != null && e.Changed)
      {
          // something has changed so we need to update the project status
          if (buildDetail.Status != BuildStatus.InProgress)
          {
              // we're in some sort of completion state
              buildDetail.StatusChanged -= new StatusChangedEventHandler(Build_StatusChanged);
              buildDetail.Disconnect();
          }

          lock (_statusCacheLock)
          {
              ProjectStatus oldStatus = new ProjectStatus();
              if (_projectStatusCache.ContainsKey(buildDetail.BuildDefinition.Name))
              {
                  oldStatus = _projectStatusCache[buildDetail.BuildDefinition.Name];
                  _projectStatusCache.Remove(buildDetail.BuildDefinition.Name);
              }

              ProjectStatus newStatus = _manager.GetProjectStatus(buildDetail, oldStatus);
              _projectStatusCache.Add(buildDetail.BuildDefinition.Name, newStatus);
          }

          if (buildDetail.Status != BuildStatus.InProgress)
          {
              AttachToNextBuild(buildDetail.BuildDefinition.Name);
          }
      }
  }

When the 'Connect()' method is called on the IBuildDetail instance, the StatusChanged event is fired when build status changes.  This event sets up internal polling within the TFS API [the default interval is 5 seconds] against that build, when the polling interval elapses, the API determines if the build data has changed, and if so updates the current IBuildDetail instance with the new information, which the extension then uses to create a new ProjectStatus object to expose to CCTray.

Sidebar: It should be noted that there is at least one 'anomaly' with the current codebase.  When a build includes some continuous integration unit tests, the TFS API seems to indicate that the build is no longer 'InProgress' and as such we disconnect from the build and the resulting status in CCTray shows up as whatever the project status was on the last build completion.  Once the build does actually complete, the status then updates to the correct final status.  This has caused us some issues in our red-yellow-green workflow, and is something that I'm actively looking into to see if there are additional status fields that I need to query to avoid this situation.  When I track down an answer, I'll be sure to post about it.

Once we have our ICruiseServerManager implementation in place, we have essentially enabled project tracking by CCTray without doing any additional work.  However, we would not get any of the functionality to trigger builds, view build details, etc that are enable through the various right-click options for a given project.  To enable those features, we need to look at the ICruiseProjectManager interface.

The ICruiseProjectManager Interface

The ICruiseProjectManager interface provides CCTray with project specific integration capabilities, including the ability to force a build, view the build details and cancel an in progress build.  The interface looks like this:

Team Foundation Server 2008 doesn't support some of the features exposed by the CCTray interface, for this extension, I've decided to essentially treat 'Build' and 'Project' equivalently.  So both the start and stop features of projects actually start/stop the current build.  As mentioned, cancellation of pending requests doesn't work, as all requests are currently synchronous.  There are also features to support retrieving build package data and 'file transfer' information.  In our case, these features aren't used and either throw NotImplementedException instances, or simply return NULL. 

The interesting function here is 'ListBuildParameters()' which is the first part of a 'ForceBuild' in the normal process flow.  Normally, we would throw up a dialog of some sort to collect the relevant parameters and return them here so they can be used to start the build process.  Unfortunately, there is a slightly unpleasant side effect of doing this.  It would disable the 'StartProject' functionality for our extension, which wouldn't be much of a problem, if it weren't for the fact that the menu entry would still be displayed [more on this problem in a future article].  So, rather than code up a dialog and query the TFS APIs directly, I've chosen to take a bit of a shortcut. 

A little reflection against the libraries deployed with Team Explorer nets the handy Microsoft.TeamFoundation.Build.Controls namespace, which is largely internal.  There is an extremely handy dialog hidden in there that allows for the queuing of a new build, complete with selection of build agent and other details.  This dialog will be familiar to those of you who use Team Explorer to queue up new builds, and is, ironically, used by the Team Foundation Power Tools' Build Notification application that I mentioned in the first part of this article. :)

Unfortunately, since this dialog is internal, we need to do a little reflection to retrieve an instance of the dialog.  Everything after that works 'out-of-the-box' to queue up a new build, without any interaction from our client.  The sticky part is that we must include a reference to a 'private' assembly of the team explorer and include it in the deployment package for our extension.  Which is why you won't see a download of a setup project to install this extension into a working copy of CCTray 1.5 RC1, and why you must have Team Explorer installed on your the machine you use to build a local copy of the extension for everything to work.  For those looking for many of the TFS client libraries that Team Explorer uses, you can find them in your [Program Files]\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies folder.

It would be great if Microsoft would choose to expose a few more of these controls and helpers for public consumption, and I hope to do a 'Part 3' of this article addressing how we could remove this final dependency on 'private' implementation details at some point in the future.  For those interested, the necessary code to interface with this dialog is below:

  public void ForceBuild(string sessionToken, Dictionary parameters)
  {
      System.Reflection.Assembly dlgAsm = typeof(BuildPolicy).Assembly;
      Type dlgType = dlgAsm.GetType("Microsoft.TeamFoundation.Build.Controls.DialogQueueBuild", false);

      if (dlgType != null)
      {
          Form buildDlg = default(Form);
          try
          {
              string teamProject = _serverManager.Settings.TeamProject;
              IBuildDefinition buildDefinition = _serverManager.TfsBuildServer.GetBuildDefinition(teamProject, _projectName);

              IBuildAgent[] agents = _serverManager.TfsBuildServer.QueryBuildAgents(_serverManager.TfsBuildServer.CreateBuildAgentSpec(teamProject)).Agents;

              object[] args = new object[] { teamProject, new IBuildDefinition[] { buildDefinition }, 0, agents, _serverManager.TfsBuildServer };
              buildDlg = dlgAsm.CreateInstance(dlgType.FullName, true, System.Reflection.BindingFlags.CreateInstance, null, args, null, null) as Form;

              if (buildDlg.ShowDialog() == DialogResult.OK)
              {
                  IBuildDetailSpec querySpec = _serverManager.TfsBuildServer.CreateBuildDetailSpec(
                                                        teamProject,
                                                        _projectName);
                  querySpec.MaxBuildsPerDefinition = 1;
                  querySpec.Status = BuildStatus.InProgress;

                  IBuildQueryResult queryResult = _serverManager.TfsBuildServer.QueryBuilds(querySpec);
                  if (queryResult != null && queryResult.Builds.Length > 0)
                  {
                      _serverManager.UpdateProjectStatus(queryResult.Builds[0]);
                  }
              }
          }
          finally
          {
              if (buildDlg != null)
              {
                  buildDlg.Dispose();
              }
          }
      }
      else
      {
          MessageBox.Show("Unable to locate build dialog");
      }
  }

In Closing

We've covered quite a bit of ground here, and there wasn't really time or space to go into detail about any of the individual pieces, but hopefully you've gotten a flavor of what it takes to both integrate with the TFS server programmatically and create CCTray transport extensions.  I'll have future articles expanding on some of the ideas here as well as some proposed enhancements to the CCTray extension concept and how we might utilize those concepts to provide richer interactions between CCTray and TFS.

Thanks for reading!

Downloads

• CruiseControl.NET 1.5 RC1
• CCTray.TFSExtension 1.0 - CCTrayTFS.zip (960.82 kb)

Background

CCTray is a build monitoring application that is part of the CruiseControl.NET suite of build tools.  Originally, when our team began setting up builds within Team Foundation Server 2008 [TFS], we didn't have a build monitoring tool available to us.  A team member had used CCTray on previous projects and put together a web page that we could use to integrate the version 1.4 CCTray tool to monitor the status of our builds using the standard 'red-yellow-green' of continuous integration.  This worked fairly well, but was less than ideal for us, because it required that everyone point their monitoring clients to a server that didn't really belong to our main TFS installation.  So, I embarked upon the task shoehorning native TFS client access into CCTray to monitor the status of TFS builds directly.  This was fairly successful, and is still how the team monitors builds to this day, but unfortunately locks the team into using a fairly old custom build of the CCTray component to do the monitoring.

As I was investigating what it would take to upgrade the system to the new 1.5 version of CCTray, I stumbled upon this blog post by Craig Sutherland, a member of the CruiseControl.NET maintenance team.  In the blog post he described a method by which he added support for extensions to the base CCTray product.  His solution mirrored, in a more generic fashion, the approach I had taken, but had the added benefit of allowing the TFS integration component pieces I had hacked in to be self-contained in an extension DLL and deployed separately from the mainline CCTray executable.  This should allow simpler deployments and easier upgrades to newer versions of the tool for our team.

Sidebar: It should be mentioned at this point that the excellent Team Foundation Server Power Tools does contain a build monitoring application, but we found that it frequently 'lost track' of the builds and would refuse to update its state.  It also didn't give quite the same 'red-green-yellow' experience that we were looking for.

The Investigation

With the extension model in mind, I went out and grabbed the latest 1.5 RC1 release candidate build for CruiseControl.NET.  I was delighted to find that the transport extension pieces that Craig had outlined in his blog post had been [partially] integrated into the build, and proceeded to see what it would take to adapt my TFS integration code into an extension for the 1.5 CCTray client.

The UI for adding an extension based server transport is fairly straightfoward, first the 'Transport Extension' option is selected and the appropriate extension library is selected.  Once selected, you click the 'Configure Extension' button and an extension specific configuration dialog is presented.  Once configured, clicking OK attempts to contact the server through the extension and download a list of available build projects.

Team Foundation Server stores pretty much everything, including build projects, by 'Team Project'.  In CCTray 1.4, I had the user include both the TFS Server name and team project as part of a 'URL' that CCTray would store for server connection.  This was not very user friendly, as the user needed to ensure correct spelling and make sure the right delimiters were included in the URL.  In this new version, I'd like to leverage the configuration dialog to select both the Team Foundation Server and the Team Project that should be associated with a given transport extension instance.

This was much more straightforward than I anticipated, as the Microsoft.TeamFoundation.Client namespace contains a publicly accessible class called RegisteredServers.  For now, the extension will just pull the current list of TFS servers registered for that user.  Getting Team Projects was similarly easy, as the TeamFoundationServer class allows us to retrieve an ICommonStructureService instance through the GetService() method.  The ICommonStructureService interface has a handy ListAllProjects() method which gives easy access to all the team projects that the current user can access on the selected server.

Armed with this knowledge, it was time to tackle the building of the TFS Transport Extension.

Implementing the Transport Extension

The ITransportExtension interface is fairly straightforward, it provides methods to retrieve an ICruiseServerManager that handles interactions with the server, and some other methods for surfacing configuration and providing detail about the extension.  The interface looks like this:

In the previous version of my code, I had already implemented an ICruiseServerManager implementation, so that code came over pretty much 'out of the box' and I was able to focus on getting the server manager hooked into the appropriate server through the CCTray settings area.  This is where I stumbled upon the first problem.  It turns out that the transport extension implementation that's in the 1.5 RC1 codebase [and the current snapshot as well], does not enable you to actually configure the extension because the configuration button isn't hooked up.  Not only that, but the first thing that happens when clicking 'OK' is that an internal variable in the form is accessed which is never initialized, so an exception is immediately thrown.  Strangely enough, the code that I originally downloaded from the SourceForge site seemed to have a mockup of some instantiation code in a paint event, but recent downloads no longer have that code either.  So, if you want to follow along you'll need to download the patch from the above link and apply it to a local copy of the CCTray 1.5 RC1 source code.

Once the button was hooked up, the next task was to determine how [and when], the various properties that are exposed by the extension are assigned relative to the call to Configure(), which is the target for the custom configuration UI dialog.  It turns out, the flow looks something like this:

1. Instantiate the Transport Extension implementation
2. Call Configure() on the extension
3. Construct a BuildServer instance using ITransportExtension.Configuration.Url and ITransportExtension.Settings property

The BuildServer instance created above is then used to assign the Configuration and Settings properties on future instantiations of the transport extension.  In essence, during Configuration, a 'blank' extension is populated with configuration data, then that configuration data is used to populate future instances of the extension for all build projects tied to a given build server.  The only potentially confusing bit is that during configuration, while the ONLY property in the Configuration class that matters is the Url, you MUST still populate the Configuration property during the call to Configure() or the transport extension cannot be used.

Storing Configuration Details

CCTray's ITransportExtension interface provides very simple string-based configuration settings storage, but each transport extension gets only a single string to store the entirety of its configuration.  In order to facilitate more detailed settings storage, it is necessary for the extension developer to decide how to map the single string of configuration data into some other form for consumption by the extension.  In the case of the TFS extension, I chose to store the configuration details in an in-memory class structure.  Currently the class contains only two properties, the Team Foundation Server hostname and the selected Team Project.  Eventually, I could see this being extended to support setting timeouts, pointing to source control, etc, but for now the server hostname and team project are sufficient.

The next task was to figure out how to provide for the serialization and deserialization of the settings data.  While an external configuration file would have been possible, and maybe desirable in certain scenarios, I wanted to avoid the complexity of another configuration file and decided to leverage the built-in XML Serialization capabilities of the .NET Framework to handle the configuration data.  I marked up the configuration class with the appropriate attributes from System.Xml.Serialization and added some static methods to the class to handle the serialization and deserialization duties.  The resulting class looks like this:

    [XmlRoot(ElementName="TFSServerManagerSettings")]
    public class TFSServerManagerSettings
    {
        #region Configuration Properties
        [XmlElement(ElementName="Server")]
        public string ServerName { get; set; }

        [XmlElement(ElementName = "Project")]
        public string TeamProject { get; set; }
        #endregion

        #region Serialization/Deserialization Support
        public static TFSServerManagerSettings GetSettings(string settingsString)
        {
            if (String.IsNullOrEmpty(settingsString))
            {
                return new TFSServerManagerSettings();
            }
            else
            {
                XmlSerializer ser = new XmlSerializer(typeof(TFSServerManagerSettings));
                using (StringReader rdr = new StringReader(settingsString))
                {
                    return ser.Deserialize(rdr) as TFSServerManagerSettings;
                }
            }
        }

        public override string ToString()
        {
            XmlSerializer ser = new XmlSerializer(typeof(TFSServerManagerSettings));
            StringBuilder sb = new StringBuilder();
            using (StringWriter writer = new StringWriter(sb))
            {
                ser.Serialize(writer, this);
            }

            return sb.ToString();
        }
        #endregion
    }

In this case I've chosen to override the ToString() method, whether or not this is the best solution is left up to the reader, but for this example it seemed to be the most 'natural' choice, given that the goal was to turn the class into a string.  Luckily in this case, this particular class is never thrown into list boxes or the like that would naturally call ToString() and expect some other representation than XML.

Next Steps

Now that we've defined our configuration data and have the UI linked together to provide the configuration bits, we can move into the meat of the extension, which is actually communicating with the Team Foundation Server and monitoring the builds.  We'll cover those pieces in Part 2.