VSTS ClassCleanup Method

A colleague and friend of mine Andre who in my opinion is a guru of Test Driven Development discivered this issue.

If you're used to Nunit and how it fires methods in your test Fixture you may assume VSTS fires the equivalent methods in a similar way. Well think again. I am specifically talking about the method you decorate with the ClassCleanup attribute.

Lets assume you have some code in your ClassSetup and ClassCleanup methods you would expect it to setup and cleanup for all the tests in that class. According to the comments generated by MS when a test class is created automatically, you would assume these methods execute before the first test runs and after the last test in that class runs.

Here is the comment you see in the Additional Test Attributes Region of a auto generated Unit Test.

//Use ClassInitialize to run code before running the first test in the class
//Use ClassCleanup to run code after all tests in a class have run
 

The Comments seem clear enough, or do they?

The Nunit equivalents would fire in the order you would expect from the above comments:

Class1.ClassInitialize
 Class1.TestInitialize
  Class1.Test
 Class1.TestCleaup
Class1.ClassCleanup

Class2.ClassInitialize
 Class2.TestInitialize
  Class2.Test
 Class2.TestCleaup
Class2.ClassCleanup

In VSTS they fire in this order:

Class1.ClassInitialize
 Class1.TestInitialize
  Class1.Test
 Class1.TestCleaup
Class2.ClassInitialize
 Class2.TestInitialize
  Class2.Test
 Class2.TestCleaup
Class1.ClassCleanup
Class2.ClassCleanup

Do you see the difference? The cleanup from class1 is not done until after class2 tests have been executed. All the class cleanups are done at once.

What if I wanted to set something up and tear it down for each class? Do I have to keep track of what order people will be running the tests and make sure I perform the setup only once in the first class to execute a test. What if I only want to run the tests in Class 2.

To me this is a design error, intentional or not.

Expression Web

I just sat through a session on Expression Web. They were showing some of the functionality that was currently being used, along with some of the plans for the next version. There are a couple of points that I'd like to make clear that came out of the session.

  • Expression Web fills the same space as Visual Studio. They are targeted to different audiences, naturally (Expression Web is aiming at designer-type people), but they are both development environments. They even work on the same project structure, although Expression Web is incapable of modifying the code behind files.
  • Expression Web does not currently have any support for Team Foundation Server. My first thought was 'ewwww'. But (and this is a killer but) you can only modify the Web pieces of the application. No code behind can be touched. While I'm not thrilled with not having integration, I'm more willing to relax it for the Web content.
  • They are currently thinking about whether to include TFS support in xWeb (the cutesy name for Expression Web) in the next version. If you want to influence their decision, send them a message through http://connect.microsoft.com.

I'm looking forward to playing with the elements of Expression Web to see what kind of trouble I can get into. :) If you want the same opportunity, you can download a trial version from http://www.microsoft.com/expression.

Computer Canada Feature Article on Employees as Assets

The April 20th edition of Computer Canada had a feature article on (generally) the asset value that employees bring to a company. I mention this because, as it turns out, I'm quoted in the article. I believe that documentation is not the solution to the knowledge leakage problem discussed in the article. Developers don't like documentation, so the trick is to keep code clean and easy to read. Standards are important. Unit tests are important (they document the assumptions made by the developer). Enforcing both standards and appropriate levels of unit testing would, quite naturally, be even more important.

This combination is a significant benefit (albeit a soft one) that a company can accrue by using tools such as Team Foundation Server. Imposing process on software development efforts can be challenging, however. The metaphor that a development team is like a herd of cats is frequently quite accurate. Getting them to buy into the process is very important to it's success. But a well planned implementation that takes into consideration appropriate levels of training along with a process that mirrors the current situation can greatly help a company achieve it's goal with respect to securing application knowledge.

TFS and Baby Steps

I just finished attending ObjectSharp's SMART (Software MAnagement Round Table) session on the impact of quality assurance on the development process and had a couple of observations to share.

First off, the biggest takeaway from the session is the need to move quality to the front of the development process. In this way, quality is a lot like security. There is no way to create a really secure application by adding security features at the end of the development cycle. In the same way, there is no magic powder that can be sprinkled on a completed application to greatly improve its quality. The overall quality of an application is found in its architectural choices, whether it was designed to be testable and the approach taken by the development staff with regards to testing.

However, one of the conversations I had with an attendee regarding Visual Studio Team Systems was just as interesting. He had just installed Team Foundation Server and had created a team project using the default Agile template. The team project he created was going to be for a working application. I found this approach interesting, not the least because (by his own admission) there was no "process" for developing software currently in play.

The interesting aspect (for me) was the supposed lack of software development process in his company. I would strongly suggest that there really was a process in place. Even though you might not think a process is in place with your development team, doesn't mean that a process doesn't exist. In any kind of team development environment, processes will form. It's just a question of how formal the process is. So your 'lack of process' (call it an unprocess) is really just a lack of documentation and formalization. Because of its lack of formality, unprocesses frequently operate below the radar of even the people involved.

If you haven't worked with Team Systems before, let me preface my comments by saying that it absolutely introduces structure to the development process. If you plan on using it 'out of the box' with no customization, you will need to use the process described by one of the canned templates. For those who don't currently have a process, the Agile template is the most likely starting point. But the Agile template includes process. There is a workflow for tasks. There is a workflow for defects. And workflow equals process.

I can appreciate the idea behind starting out with the canned templates. Its presence would indicate that it's a process with proven success. And using the product as it arrives is a first, small 'baby step' into the world of development process. The problem is that the first step you're taking isn't a small one. Instead, you have taken a giant step into the world of Agile development. Complete with all of the process that is codified in the Agile template. And Agile has more process than most people think. The result is that the first 'baby' step can be a killer for Team Systems.

If someone new to a development process truly wants to take baby steps with TFS, they should start by thinking about the unprocess which is currently being used in their organization. Spend a day or even a few hours talking to someone who is familiar with development methodologies, to identify the flows that are currently at play. The default template should be customized to follow this unprocess. Then, over time, the structure originally defined by the Agile template can be added back in, if needed.

Does this require a little more effort on your part? Yes. But the steps involved in modifying the template are both well documented and fairly easy to follow. By taking this approach, you truly can learn Team Systems by taking baby steps. And this is a lot less painful than trying to make the leap from unprocess to Agile.

Visual Studio Pro to get Unit Testing in Orcas

One of the biggest wrongs in VSTS 2005 was that the Team Architect edition didn't have unit testing. Not only has that been rectified for the Orcas time frame, but this feature has been moved down into the Visual Studio Professional SKU for the next release in Orcas. Fantastic! [via VSTS Quality Blog]

My Recent Visit to MS Research

A couple of weeks back (sorry still catching up) I had the great fortune of being invited to visit a MS Research event called TechFest. MS Research hosts this event annually for fellow MS Employees and opens up their doors so they can see what they've been working on, and for the first time in their 15 year history, they allowed those of us without the trademark MS employee blue-badge to attend. The "exhibit hall" was very "science fair" full of academic research geeks and no marketing folks.

MS Product teams have been going through an era of transparency over the past few years and it was nice to see this principle infect MSR. A few quick facts that impressed me about MSR:

  • MSR has been grown from 0 to 750 researchers over the past 15 years. This is the equivalent of creating a new Berkeley CompSci Faculty...each of those 15 years.
  • MSR has researchers in facilities located in Redmond, Silicon Valley, San Francisco, Cambridge, Beijing and Bangalore.
  • MSR runs the largest CompSci Ph.D. internship program. Currently there are 800 Ph.D. interns working worldwide in MSR labs. In the US last summer, MSR had 300 interns in their labs which is impressive considering that the US produces about 1200 CompSci Ph.D.'s annually

There was lots of interesting projects to learn about, some very space-age and cool, others kind of weird and hard to see any use for, and some that you wish were in a shipping product yesterday. Then there are some that caused attendees with certain allergies to get all stuffed up. Check out some of the videos here and here.

Team System Web Access: Free!

Well this is about the best news I've read all day. Microsoft has acquired DevBiz, the makers of teamplain, which provides web based access to work items and some source control operations.

The good part of the news is that now MS owns it and is rolling into Team System (first as a power toy), and as of right now, you can download it and install it for free. So long as the clients have a TFS Client Access License, you're good.  [via Brian Harry's blog]

Toronto Association of Systems and Software Quality

A few weeks ago I attended the Toronto Association of Systems and Software Quality (TASSQ). It's refreshing to attend a user group outside of the MicroSphere. There was about a 100 attendees and I like the round table format over dinner with lots of opportunity for peer discussions.

This particular month the format was that of a moderated panel and despite being a new member, somehow I ended up as one of the panel members during the first part of the evening. There were a lot of questions and discussions around agile development, TDD and how that plays a part in the QA role. It was my pleasure to meet Scott Ambler and sit on the panel with him. He is an interesting guy who spends much of his time in the agile space.

It was obvious to me from watching and participating in the discussions that the TDD movement is perceived as a huge opportunity to assist Quality Assurance professionals. Unfortunately there is not a good understanding of how Test Driven Developers collaborate their work with the QA team. Not actually knowing what TDD or how to implement it is another common problem, but that's another blog post.

TDD is all about adding quality from the inside out, but per se, not intended to assist the QA profession. But there are good opportunities here in the areas build verification, earlier functional testing, regression testing and code coverage. The end goal here should be to cost-effectively improve our software quality. With that in mind, if you are in the Toronto area on March 29, 2007, I'd like to invite you to our Software MAnagement RoundTable (SMART) breakfast to discuss these issues. Click this link to read more about the event and registration details.

TimeEntry

LEAD Technologies Inc. V1.01

TFS Time Entry Utility

Many clients ask me if there is a tool that will help you keep track of time spent on different tasks in a TFS Project. The answer has always been, "Not that I have seen, but it wouldn't be hard to write one".  

Well I put my code where my mouth was and I wrote this little utility, of which I am happily sharing with anyone who would like it. It's a windows application that sits in your system tray when minimized and allows you to select a project, query and workitem, hit start and work away on that work item. When you hit stop it will add the amount of time you spent to the Completed work field and subtract that same amount of time from the Remaining work field of the same work item.

I want to change the UI but have not had a chance yet. I would also like to split the code up and get it out of the window. I would also like to get the old water heater out of my basement and to the dump but there is always something more important to do.

I have added a project to CodePlex so if anyone wants to work on it please let me know and I will add you as a developer.

Meanwhile your feedback is very welcome.

Some enhancement ideas I have:

  1. Set the Project and Query off the right click on the system tray (similar to the Server)
  2. Make the list of work items a long list so you can just click on the item to start timing. (Or perhaps a clock next to it)
  3. Make the fields it updates configurable in case you have customized the WorkItem.
  4. Configiure the Work Item types that should show in the list. For now the Query can do that.
  5. Make it an add in for VS
  6. Add it as a control to the Work item field it self.

Enjoy!