TechEd (Day 1): Balmer's Keynote & Announcing VS 2005 Team System

It's official. I'll post more thoughts and analysis about this as time permits, but, things you should know.

  • Microsoft now has a new Team version of Visual Studio to be delivered “Next Year“ according to Balmer.
  • new source control - more details to follow.
  • Project Management - so dev's will be able to see “Work Items“ in their IDE. There is also supposed to be a sharepoint portal of some kind that dev's & pm's can go to see a dashboard view of a project, milestone's, etc. integrated with MS Project Server.
  • Unit Testing - yes, a very NUnitish thing built right into visual Studio.
  • Code Coverage - yes in the editor you can see what code was executed and what was not.
  • Static Code Analysis - a la fxCop integrated right inside of visual studio.
  • Check in Source control process policy, so a manager type can say “if you check in something, all tests must pass, all static analysis rules must pass, and your code coverage must be 100%“.
  • Also showed was some Load testing stuff that is going to be better than Application Center Test - more on that later.

Of course whitehorse class modeling & SOA designer were showed quickly. Nothing new to announce yet on that front that wasn't covered at PDC....although the guy doing the demo kept saying “Services Oriented APPLICATION” designer. Is this new? Is he changing the acronym from Architecture?

TechEd (Day 0): BOF36: Integrating Unit Testing Tools and Practices Into the Software Development LifeCycle

This BOF went pretty well and a huge thanks to Jim Newkirk for assisting in the delivery. He's a real authority on the practices around NUnit and a good guy to have a round. If you buy his new book on Test Driven Development with Microsoft .NET onsite at TechEd, you can probably catch him at the MS Pavillion to sign a copy.

Some interesting points discussed:

  • Using Unit Tests to drive “example code“ for a framework or class library would be a nice to have.
  • While Code Coverage statistics may satisfy external parties that we've tested what we've developed, percentages are not an accurate measure of code quality.
  • If you write your tests after you do you coding, you already have too much information about your classes that negatively affects how you test.
  • Testing first can really influence (positively) the design of your classes.
  • Developers will work aggressively against source-code check in policies that stipulate a % of code as been covered in unit tests, and that the tests pass, and that they pass static code analysis.
  • It's difficult to test User Interface code, and for a bunch of reason's, not a really good idea or worthwhile investment because the only person who can see your application from the outside in, is through the eyes of a specific user - and you'll never be able to approach that.
  • At the end we also got into some of the difficulties of testing against a database and a bit about Mock objects. That would probably be a good bof on it's own.

Jim might have more comments, but the general feeling I got was that people still need more education about automating unit tests and that not a lot of people are doing it today, let alone Test First. Jim also mentioned that he didn't think it was possible to lecture to somebody and convince them about Test First, but more that it was something that they just really needed to see for themselves. I agree.

TechEd (Day -1): ObjectSpaces = Longhorn, 3rd time charm?

It was bumped from The initial 1.0 release, and then as of last PDC slated for Whidbey. Now it looks like we'll have to wait until Longhorn.

It's not all bad news however. ObjectSpaces is being re-orged into the WinFs file system. When you think about there is an awful lot of correlation to those technologies. I'm sure it's not terribly unrelated to the fact that the Microsoft Business Framework(MBF) that was to build on ObjectSpaces was also pushed off to Longhorn/Orcas. MBF is also to rely on an orchestration engine (Biztalk light?) features going into Longhorn so it all makes sense.

Some people will be disappointed - but this is a good rationalization of the way too many data access/storage visions within Microsoft. Both of these technologies have a common thread about objects/applications and data and breaking down the wall. Sure, MS could have released ObjectSpaces first, but do we really need that legacy and all the effort attached to YADAA (yet another data access api.

Microsoft has taken a lot of criticism (including from me) about the seemingly constant churn of all things data. So this is a good sign that MS is not going to do things, just be cause they can, but do them right. Just ask a Java developer what they think of EJB's. It's important to get it right

TechEd (Day -1): Pieces of ADO.NET 2.0 not going to make Whidbey

Highlights in this exchange of newsgroup posts...

  • Paging is cut
  • Server cursors cut (SqlResultSet class)
  • Async Open is cut, but not excute
  • SqlDataTable class cut (Good - that was a bad design pattern to propogate)
  • Command sets are cut, but you can still do batch DataAdapter updates,

More to come, stay tuned.

NUnit 2.2 beta released

Things I like:

  • Assert.AreEquals support to compare arrays of the same length, type and values.
  • You can now put a Category attribute on your fixtures AND methods....and then use that as a filter when you go to run tests. Thoughts on categories? Functional Tests,  Performance Tests
  • On a similar note, there is an explicit attribute that will cause a fixture or method not to run unless explicitly selected by the user. You can now put check boxes on the tree to select multiple fixtures/methods.
  • They fixed a problem with background threads that when they raise exceptions, they weren't showing up as a problem in NUnit. Seems they've done some refactoring of how things are loaded in the AppDomain. I'm hopeful that this fixes some issues I've seen when own dynamic loading and Fusion get's lost...but only during the NUnit tests, not the production execution.

Looks exciting.

OpenSource Project for Testing Microsoft Software

Over the past few months, when I question how something works in the .NET Framework (or when somebody asks me).....I have been creating NUnit tests to verify the behaviour of some class and/or methods in the .NET Framework. Initially it is just to observe the behaviour or verify some assumptions, but by the time I'm finished, I usually inject various Assertions into my tests to tighten them up. These now serve as a test bed for me moving to a new version (or even old versions) of the .NET Framework. I can answer the question: Are any of my assumptions about how the 1.1 framework works broken in 1.2? 2.0? 9.0? etc.

I'm building up a nice collection and I might publish my work. But it struck me that this could be an open source project. In fact, I think it should be an open source project and I think it should be started by Microsoft....and not necessarily for the .NET Framework alone - but that would be an easy place to start.

Microsoft has faced increasing pressures over security and quality of their software - to the point that they've actually made windows source code available to key customers, governments and MVP's. I think that's a bit risky if you ask me. I think it is also a bit hypocritical to point the finger at Linux for being “more hackable because source code is available“ but at the same time make your own source code available to the chinese government.

But why not publish the source code to unit tests (say NUnit fixtures) in an open source format for the community to contribute to. When one of these security firms finds a hole in some MS software, they could create an NUnit test to expose it and submit it to Microsoft to fix, and then make the code for that NUnit test part of the open source project.

Instead of publishing source code, which is really meaningless to give people any kind of comfort in the code, publishing unit tests is publishing assumptions and expectations about what software is supposed to do and how it is supposed to behave. I would think this would become more important over time especially moving towards WinFx and Longhorn.

Delegation through NUnit Testing and TodoFixtures

Usually I'm the guy who all the other developers are wiating on to create some reusable framework widget or other. I usually have 10, 000 things on my plate so when somebody asks me to do something or reports a bug with some of my code, I need to find a good way to delegate.

But if you are the subject matter expert (SME), it's tough to delegate the task of making the fix or adding the feature. If you flip that on it's head though, when you find yourself in this situation, by definition you are NOT the SME of the “feature request” or “bug“. Those are external to the actual implementation which is probably what you are really the SME for. The SME for the request or bug, is of course, the finder of the bug or the requestor of the feature. So in the spirit of getting the right person for the job (and delegating a bit at the same time), get the requestor to create the NUnit test that demonstrates the bug or explains (with code - better than english can ever hope to) the request or how the requestor expects the feature to be consumed.

Case in point:

Random Developer A: Barry, there is a bug in the foobar you created. Can you fix it? I know you are busy, want me to give it a go? Do you think it's something I could fix quickly?

Barry: That's a tricky bit but I think I know exactly what I need to do to fix it, it will take me 10 minutes - but I have 3 days of 10 minute items in front of you. Why don't you create an NUnit test for me that demonstrates the bug, and  I'll fix it. Then it will only take me 2 minutes.

I also find NUnit tests a great way for people to give me todo items.

Random Developer B: Hey, I need a method added to FoobarHelper that will turn an apple into an orange, unless you pass it a granny smith, in which case it should turn it into a pickle.

Barry: Sounds reasonable. I can do that - but just to make sure I got all of that spec correct, would you mind creating an NUnit test that demonstrates that functionality you require? Thanks.

I do have to admit though that this requires a certain amount of charisma. On more than one occassion this technique has been met with some friction and unusual and jestures and mumbling. :)

Things I don't like about NUnit

Firstly - I love NUnit - and nobody has done more to increase the quality of .NET Applications than the contributors to this project - nobody.

But I know that next generation Unit testing framework authors are listening so I might as well state the things I'd like to see:

  • I'd like to be able to run a single test from the command line. Not just a single fixture, but a specific test.
  • I'd like my tests to be parameterizable. I'd like from the Command Line to run a test and provide the specific values.
  • Wouldn't it be cool to create a batch file of scenarios? What about doing this in XML? Duh - no brainer.
  • A lot of my tests in this case would simply wrap up and call business objects - so why not be able to have a virtual test in the xml file? That is just call a class/method directly with XML.I'm not saying this is the be all / end all - really the only kinds of assertions I could do realistically would be to test for certain exceptions or no exceptions. Return values? Possibly - but really - some times our methods accept and return things other than value types - but this might be a nice thing to have regardless.

With these things in place, one could conceivably:

  • tie a class modelling tool into test scenarios. Hmm, I need a class, with a method and if I pass this data, I should get these results.
  • If an end user reports a bug in my defect tracking system, I should be able to create an NUnit test that exposes this bug. Then my support people can come back from time to time, have the defect tracking system run the test to see if it's been fixed in a given patch/release/build, etc. and update the status on the defect.



ASP.NET Whidbey at CTTDNUG Tonight.

I'm presenting an overview on ASP.NET 2.0 tonight at CTTDNUG.

There isn't a great abstract on the site - and in fact, I will physically be unable to do the objectspaces stuff since the new version of VSNET CTP doesn't even have it in it anymore. Don't read into that - objectspaces will still be coming out - at some point. I should be able to give some nice objectspaces PPT's if the crowd is interested - but I'm guessing that Demo's are going to be more enjoyable.

So I am going to do my best ScottGu thrie impersonation and give a good solid demo lap around ASP.NET. IDE Improvements, Master Pages, the new datasource stuff, Site Navigation, Security, Personalization, SqlCaching.

Security for Developers

Why is that you can't plug a fridge into your house until it's been CSA or FCC approved, and that you have to have a licensed electrician install or at least review any modifications to the wiring in your house plugged into the grid - but any yahoo can build a piece of software and install it on their home computer connected to the internet for the world to hack into? Before I make a case that developers should be forced to do some security training or pass some certification....we have to keep in mind that most of the time the software sitting on somebody's home computer that is getting hacked into is Microsoft's. This is largely due to the size of the huge target on their back. What you think Linux is really more secure? What do you think is easier to hack into? It's easier to hack into something when you have the source code.

So, having said all that, there are changes coming for Microsoft developers in the security space:

  • Microsoft is turfing a few of the existing security training offerings. These include the Microsoft Security Clinic (2800) and Security Seminar for Developers (2805).
  • There is a new security course being developed: Developing Secure Applications (2840) and also a MS Press Training Kit both of which I'll be reviewing during their development.
  • Related to the new course and training kit, there is a new security exam for developer which unfortunately because of timing is only an MCAD/MCSD/MCDBA elective (not a required element - sigh). There are 2 versions - 1 for VB and 1 for C#. I guess they figure C++ and J# developers already write secure code. These are going into beta at the end of next month and I'll be auditing the C# version.
    71-330 Implementing Security for Applications with Visual Basic .NET
    71-340 Implementing Security for Applications with Visual C# .NET

I'm a little torn over this direction. Part of me says that security is so important, it needs to be covered in every MS Training course. To a certain extent that is already true, but I think they could go deeper. When I teach a windows, web or services course, I try to go deep on security. Sometimes you can go to far. Some pieces of security are more relevant to the type of application you are building, while other security issues are common regardless of the application architecture. Obviously we don't want to repeat a lot of content in each course - sometimes that is unavoidable. The other issue is that there is a lot to know about security and frankly I don't think every developer can master all of this. So teams need to dedicate a security architecture role on their project. For these folks - then yes I think it makes sense to have specific and deep training and certification for them. I think MS could probably do better than a single exam “elective”. How about an MCSD.NET+Security designation? MCDBA+Security as well - although you could argue that MCDBA's should be forced to have this security. Perhaps that will happen in the wake of Yukon - although I've heard no rumblings of creating Whidbey or Yukon flavours of exams or certifications at this point.

Our industry and profession needs to take a leadership role and be proactive in accepting responsibility and accountability for the important issue of security. We need to move our discipline to a higher level. I'm not convinced it has to be government that steps up to this place. Governments should only do what we can't do for ourself. Microsoft seems to be taking an increasingly proactive role on these security issues. It will be interesting to see how this pays off in 2-3 years from now.