What do you do when your breakpoint won't break?

Go straight to Andy Pennel's Breakpoint Helper. It's not some sw to download - it's a nice interactive q&a that will help you pinpoint the problem. It has tips on 2002, 2003 and 2005. Ahhh. Thanks Andy.

ps. Andy is a dev lead in C# and owns the debugger (as also used by VB.NET, C++, Script and SQL)

TechEd (Day 3): Hands On Lab Manuals downloads available to the public

No need to have a TechEd commnet password. You can download ALL the pdf's for the plethora of topics. Some good stuff to see how the newly announced stuff (Team System, etc.) works.

Update These links are broken, give this a try: http://www.msteched.com/TechEdLabManuals.aspx

TechEd (Day 1): New Sessions on Visual Studio 2005 Team System

A bunch of sessions that were formerly named “TBA” are really about the new Visual Studio 2005 Team System.

  • DEV200 General Session: Managing the Software Lifecycle with Visual Studio 2005   Halls GH
    Tuesday 10:45
  • DEV300 Visual Studio 2005 Enterprise Tools: Software Project Management  Room 20D
    Tuesday 1:30
  • DEVC36 Testing Visual Studio 2005 Applications Using the .NET Framework  Cabana 06
    Thursday 10:15
  • DEV302 Visual Studio 2005 Enterprise Tools: Building Robust and Reliable Software  Room 20D
    Thursday 1:30
  • DEV303 Visual Studio 2005 Enterprise Tools: Enterprise-Class Source Control and Work Item Tracking  Room 20D
    Thursday 3:15
  • DEVC39 Extending Visual Studio 2005 Team System   Cabana 06
    Thursday 5:00
  • DEVPNL4 Enabling the End-to-End Solution Lifecycle: Microsoft Partner Panel  Room 20D
    Friday 12:15
  • DEV356 Visual Studio 2005: Managing the Enterprise Build Process with MSBuild  Room 31ABC
    Friday 1:00

TechEd (Day 1): Balmer's Keynote & Announcing VS 2005 Team System

It's official. I'll post more thoughts and analysis about this as time permits, but, things you should know.

  • Microsoft now has a new Team version of Visual Studio to be delivered “Next Year“ according to Balmer.
  • new source control - more details to follow.
  • Project Management - so dev's will be able to see “Work Items“ in their IDE. There is also supposed to be a sharepoint portal of some kind that dev's & pm's can go to see a dashboard view of a project, milestone's, etc. integrated with MS Project Server.
  • Unit Testing - yes, a very NUnitish thing built right into visual Studio.
  • Code Coverage - yes in the editor you can see what code was executed and what was not.
  • Static Code Analysis - a la fxCop integrated right inside of visual studio.
  • Check in Source control process policy, so a manager type can say “if you check in something, all tests must pass, all static analysis rules must pass, and your code coverage must be 100%“.
  • Also showed was some Load testing stuff that is going to be better than Application Center Test - more on that later.

Of course whitehorse class modeling & SOA designer were showed quickly. Nothing new to announce yet on that front that wasn't covered at PDC....although the guy doing the demo kept saying “Services Oriented APPLICATION” designer. Is this new? Is he changing the acronym from Architecture?

TechEd (Day 0): BOF36: Integrating Unit Testing Tools and Practices Into the Software Development LifeCycle

This BOF went pretty well and a huge thanks to Jim Newkirk for assisting in the delivery. He's a real authority on the practices around NUnit and a good guy to have a round. If you buy his new book on Test Driven Development with Microsoft .NET onsite at TechEd, you can probably catch him at the MS Pavillion to sign a copy.

Some interesting points discussed:

  • Using Unit Tests to drive “example code“ for a framework or class library would be a nice to have.
  • While Code Coverage statistics may satisfy external parties that we've tested what we've developed, percentages are not an accurate measure of code quality.
  • If you write your tests after you do you coding, you already have too much information about your classes that negatively affects how you test.
  • Testing first can really influence (positively) the design of your classes.
  • Developers will work aggressively against source-code check in policies that stipulate a % of code as been covered in unit tests, and that the tests pass, and that they pass static code analysis.
  • It's difficult to test User Interface code, and for a bunch of reason's, not a really good idea or worthwhile investment because the only person who can see your application from the outside in, is through the eyes of a specific user - and you'll never be able to approach that.
  • At the end we also got into some of the difficulties of testing against a database and a bit about Mock objects. That would probably be a good bof on it's own.

Jim might have more comments, but the general feeling I got was that people still need more education about automating unit tests and that not a lot of people are doing it today, let alone Test First. Jim also mentioned that he didn't think it was possible to lecture to somebody and convince them about Test First, but more that it was something that they just really needed to see for themselves. I agree.

TechEd BOF Session on Integrating Unit Testing Tools and Practices into the Software Development Life Cycle.

This session which was originally scheduled for Tuesday night is now moved to Sunday night at 8pm. This should allow people who were interested in the Peter Provost's hosted Continous Integration Testing BOF to attend both of these sessions.
BOF36  Integrating Unit Testing Tools and Practices Into the Software Development LifeCycle
Sunday , May 23 8:00 PM- 9:00 PM, Location TBA
Speaker(s): Barry Gervin
Automated Unit Testing tools and practices are finding their way into the dev life cycle of more and more IT shops each day. Some people simply use NUnit to demonstrate a bug and prove that's fixed. Others use NUnit to demonstrate a % goal for code coverage. Others go as far as to use Test Driven Development to drive the design of their applications. People are using Unit testing harnesses for Design, Functional Testing, Performance and Security Testing. In this discussion we will raise awareness of the current tools and practices that people are using today and how best to address the more holistic needs with future tools and practices.
And if you happen to be going to TechEd Amsterdam, James Newkirk is going to repeat this session there.

NUnit 2.2 beta released


Things I like:

  • Assert.AreEquals support to compare arrays of the same length, type and values.
  • You can now put a Category attribute on your fixtures AND methods....and then use that as a filter when you go to run tests. Thoughts on categories? Functional Tests,  Performance Tests
  • On a similar note, there is an explicit attribute that will cause a fixture or method not to run unless explicitly selected by the user. You can now put check boxes on the tree to select multiple fixtures/methods.
  • They fixed a problem with background threads that when they raise exceptions, they weren't showing up as a problem in NUnit. Seems they've done some refactoring of how things are loaded in the AppDomain. I'm hopeful that this fixes some issues I've seen when own dynamic loading and Fusion get's lost...but only during the NUnit tests, not the production execution.

Looks exciting.

OpenSource Project for Testing Microsoft Software

Over the past few months, when I question how something works in the .NET Framework (or when somebody asks me).....I have been creating NUnit tests to verify the behaviour of some class and/or methods in the .NET Framework. Initially it is just to observe the behaviour or verify some assumptions, but by the time I'm finished, I usually inject various Assertions into my tests to tighten them up. These now serve as a test bed for me moving to a new version (or even old versions) of the .NET Framework. I can answer the question: Are any of my assumptions about how the 1.1 framework works broken in 1.2? 2.0? 9.0? etc.

I'm building up a nice collection and I might publish my work. But it struck me that this could be an open source project. In fact, I think it should be an open source project and I think it should be started by Microsoft....and not necessarily for the .NET Framework alone - but that would be an easy place to start.

Microsoft has faced increasing pressures over security and quality of their software - to the point that they've actually made windows source code available to key customers, governments and MVP's. I think that's a bit risky if you ask me. I think it is also a bit hypocritical to point the finger at Linux for being “more hackable because source code is available“ but at the same time make your own source code available to the chinese government.

But why not publish the source code to unit tests (say NUnit fixtures) in an open source format for the community to contribute to. When one of these security firms finds a hole in some MS software, they could create an NUnit test to expose it and submit it to Microsoft to fix, and then make the code for that NUnit test part of the open source project.

Instead of publishing source code, which is really meaningless to give people any kind of comfort in the code, publishing unit tests is publishing assumptions and expectations about what software is supposed to do and how it is supposed to behave. I would think this would become more important over time especially moving towards WinFx and Longhorn.

Delegation through NUnit Testing and TodoFixtures

Usually I'm the guy who all the other developers are wiating on to create some reusable framework widget or other. I usually have 10, 000 things on my plate so when somebody asks me to do something or reports a bug with some of my code, I need to find a good way to delegate.

But if you are the subject matter expert (SME), it's tough to delegate the task of making the fix or adding the feature. If you flip that on it's head though, when you find yourself in this situation, by definition you are NOT the SME of the “feature request” or “bug“. Those are external to the actual implementation which is probably what you are really the SME for. The SME for the request or bug, is of course, the finder of the bug or the requestor of the feature. So in the spirit of getting the right person for the job (and delegating a bit at the same time), get the requestor to create the NUnit test that demonstrates the bug or explains (with code - better than english can ever hope to) the request or how the requestor expects the feature to be consumed.

Case in point:

Random Developer A: Barry, there is a bug in the foobar you created. Can you fix it? I know you are busy, want me to give it a go? Do you think it's something I could fix quickly?

Barry: That's a tricky bit but I think I know exactly what I need to do to fix it, it will take me 10 minutes - but I have 3 days of 10 minute items in front of you. Why don't you create an NUnit test for me that demonstrates the bug, and  I'll fix it. Then it will only take me 2 minutes.

I also find NUnit tests a great way for people to give me todo items.

Random Developer B: Hey, I need a method added to FoobarHelper that will turn an apple into an orange, unless you pass it a granny smith, in which case it should turn it into a pickle.

Barry: Sounds reasonable. I can do that - but just to make sure I got all of that spec correct, would you mind creating an NUnit test that demonstrates that functionality you require? Thanks.

I do have to admit though that this requires a certain amount of charisma. On more than one occassion this technique has been met with some friction and unusual and jestures and mumbling. :)

Things I don't like about NUnit

Firstly - I love NUnit - and nobody has done more to increase the quality of .NET Applications than the contributors to this project - nobody.

But I know that next generation Unit testing framework authors are listening so I might as well state the things I'd like to see:

  • I'd like to be able to run a single test from the command line. Not just a single fixture, but a specific test.
  • I'd like my tests to be parameterizable. I'd like from the Command Line to run a test and provide the specific values.
  • Wouldn't it be cool to create a batch file of scenarios? What about doing this in XML? Duh - no brainer.
  • A lot of my tests in this case would simply wrap up and call business objects - so why not be able to have a virtual test in the xml file? That is just call a class/method directly with XML.I'm not saying this is the be all / end all - really the only kinds of assertions I could do realistically would be to test for certain exceptions or no exceptions. Return values? Possibly - but really - some times our methods accept and return things other than value types - but this might be a nice thing to have regardless.

With these things in place, one could conceivably:

  • tie a class modelling tool into test scenarios. Hmm, I need a class, with a method and if I pass this data, I should get these results.
  • If an end user reports a bug in my defect tracking system, I should be able to create an NUnit test that exposes this bug. Then my support people can come back from time to time, have the defect tracking system run the test to see if it's been fixed in a given patch/release/build, etc. and update the status on the defect.