Data Driven Development

So we have Test Driven Development and Model Driven Development or Design by Contract (similar perspective). But in the past, I've been a fan of Data Driven Development. This is a technique I haven't had the pleasure of using recently....because it relies on you building new applications with new databases.

What is this technique you ask? Well for me it is designing the data model first. In the early days of Client/Server, PowerBuilder and ERwin were my tools of choice. New applications. New databases. My design process (and that of many of my associates) was not so much to design a database but to document the data that existed in the organization - and do that in 3rd normal form. ERwin still stands as one of the best modeling tools ever because it actually made the job of coding up a database schema easier and faster than any other alternative. I could also use my model throughout the entire lifecycle since it did an excellent job at full round trip engineering/synchronization.

One of the cool features of PowerBuilder was your ability to annotate your database schema with UI hints. So you could say that a given column in your database should by default be shown as a checkbox, and that checked should be saved as “true“ and unchecked as “false“ - or whatever weird thing your DBA said it had to store. Whenever you designed a screen with that column, bam you'd have it the way you'd expect - as a checkbox. The downside of PowerBuilder's datawindows of course was that the data store/entity/container was quite pretty tied to your database and they made no attempts to hide that fact. But boy, productivity was really high - although I was producing tightly coupled, loosely coupled code :( .NET let's me build better code now, but productivity is still lacking.

At TechEd a couple of weeks ago, I stopped by the DeKlarit booth for a demo of their product by their lead architect Andres Aguiar. I was happy to see a tool that builds upon the Data Driven Development process. Of course, you don't have to start with an empty database, but this tool does an excellent job of making your job easy when starting from scratch. Andres promised to send me an eval so I can play with it some more to see how it works with existing databases but this tool so stay tuned. I could easily see this tool paying for itself in a matter of a couple of weeks.

As for ERwin, I'm still a fan although it really hasn't changed much in the past 10 years. I remember the first copy I had fit on a single floppy. So did the 200 table model I created with it. I was using LBMS System Designer who stored my model in some kind of 10mb black hole and took 10 minutes to generate a schema. When I first installed ERwin, I had it installed and reverse engineered by LBMS model - and forward engineered to from Oracle to SqlServer inside of 10 minutes. I couldn't believe the schema generation took 20 seconds compared to LBMS at 10 minutes.

SOA Challenges: Entity Aggregation

Ramkumar Kothandaraman has a good article just released on MSDN discussing SOA Challenges: Entity Aggregation. Aggregation is a much better name than “composable entities“ since it's definition implies that property sets of an entity grow as more child entities are merged into it. This also implies that you need a mapping layer and conflict resolution to resolve duplicate property names or just rename them for that matter.

This is becoming an important technique for passing xml documents up the stack of web services, each one adding their own value to the entity - or aggregating in a master/slave hierarchy topology. Either way, one of the subtle things about entity aggregation is that you can also think of it as a lightweight form of multiple inheritance for the properties of your domain objects. Is that useful or am I just bent?

Objects and Web Services

I've  been asked this question enough both on-line and in person that I thought a blog might be a better way to clear up a common misconception. Consider the following situation.  Your application has implemented a class as follows:

public class Dog
{
   public string Breed;
   public string Fetch()
   {
      return "stick";
   }
}

As well, the application has a web service method defined like the following:

[WebMethod]
public Dog GetDog()
{
   return new Dog();
}

So let's move to the client side. In a typical .NET application, you would add a web reference pointing to the web service that includes the GetDog method just described.  Under the covers, Visual Studio .NET invokes wsdl.exe to generate the proxy class. Because of the magic of this tool, not only is a proxy for the web service created, but so too is a class that represents the Dog class that is returned from GetDog.  This means that, on the client side, there is no need to deploy the assembly that implements the Dog class on the server-side.  Or does it.

If you open up the .cs file generated by adding the Web Reference (you might have to “show all files” in the Solution Explorer then navigate to below the Reference Map in the Web Reference - the file is called Reference.cs), you will find a class called Dog defined. Does the automatic addition of the Dog class make it easy for developers to work with web services?  Yes. Does it cause some level of confusion?  Definitely.  Which is the reason for this post.

Take a close look at the Dog class in Reference.cs.  What you will see is a class that contains all of the public properties that were exposed by the server-side Dog class.  What you will *not* see is any of the business logic that might have been implemented in the server-side Dog class.  Nor will you see any methods, public or otherwise.  In other words, these two classes are absolutely not the same class, regardless of the similarity in naming. 

What kind of problem does this cause? Say you actually deploy the server-side assembly onto the client and try to pass an object of that type (the server-side Dog class, that is) into the web service method call.  The result is that an apparently inexplicable InvalidCastException error gets thrown.  The reason for the exception is because the server-side Dog class is not the same Dog class defined in the proxy class.  That should make sense now, but it throws a lot of new web service developers for a loop.

There is a solution to the problem.  Once the proxy class has been created, edit the Reference.cs file directly.  Remove the Dog class from that file and change the return type for the GetDog method to match the server-side Dog class.  Now the InvalidCastException will go away.  Keep in mind that, should you decide to refresh the Web Reference, then you will have to make these changes again.  So when I do this kind of manipulation, I would typically use wsdl.exe manually to create the proxy class file and they add it into the project.  This eliminates the potential for wiping out the changes accidentally.

 

RSS Feeds and DataSets

So as I write this at 1:10 in the morning, I have just finished beating my head against a fairly hard wall.  The task was to load an RSS Feed into a DataSet.  Then once in the DataSet, the feed could be bound to a Datagrid, a DataRepeater, etc.  The plan was simple.  The strategy sound. Or so I thought.

"No plan survives contact with the enemy." -Field Marshal Helmuth von Moltke.

While I suspect that many of you are aware of the quote, it´s place in the process of strategic is frequently forgotten.  The quote is intended to remind strategists to set broad objectives and seize unexpected opportunities when they arise. Of course, in this instance, I´m using the more pessimistic “stuff happens“ meaning that is more commonly ascribed to the quote.

So I naively start the process by creating a DataSet object and using the ReadXml method to build it.  Due to the foresight of the .NET Framework designers, I can simply pass the URL to the RSS Feed as a parameter.  Sweet.

But then I run into a snag. Or, to put it another way, Moltke proved his prescience. Within ReadXml, a DuplicateNameException is thrown. The specific message was  “A column named 'comments' already belongs to this DataTable.“ After an examination of the RSS feed, I discovered that the Slash RSS module uses a Comments tag. In the RSS feed, it is properly namespaced, but the namespace is not recognized by ReadXml.  So the Comments tag in RSS clashes with the Comments tag in Slash RSS.

The solution to this problem is not as clean as I would have liked.  In my ideal world, there would be a way to limit the namespaces that are loaded into the DataSet.  Perhaps using the XmlNamespaceManager, for instance.  But for all the Googling that I did, there doesn't appear to be any solution down this alley. So instead I turned to a kludgey (from my perspective) method that involves transforming the RSS feed using XSL.

My next problem immediately followed this solution.  When I processed the transformed RSS feed, another exception was thrown.  This time it was a ArgumentException that read "The same table (p) cannot be the child table in two nested relations.". Back to the RSS feed we went.  What we saw was that if the post was XHTML compliant, then a separate <Body> block was contained within the post.  In this <Body> block was the post complete with the various markup tags (like <p>) intact.  This differs from the normal format of the post, where the angle brackets surrounding the tag are converted the &lt; and &gt;. The result was that ReadXml was choking on the fact that <P> existed in to separate items.

Back to the XSL, where I excluded the <Body> block from the RSS feed.  Now my XSL file looks like the following:

<?xml version="1.0" encoding="utf-8" ?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
     xmlns:wfw="http://wellformedweb.org/CommentAPI/"   
     xmlns
:slash="http://purl.org/rss/1.0/modules/slash/" 
     xmlns
:xhtml="http://www.w3.org/1999/xhtml" >

    <xsl:template match="*|@*|comment()|text()">
       
<xsl:copy>
           
<xsl:apply-templates select="*|@*|comment()|text()"/>
       
</xsl:copy>
    </
xsl:template>

    <xsl:template match="slash:comments">
        <SlashComments>
            <
xsl:apply-templates select="*|@*|comment()|text()"/>
        </
SlashComments>
    </
xsl:template>

    <xsl:template match="xhtml:body"></xsl:template>

</xsl:stylesheet>

Mission accomplished.  I now have a DataSet that contains the necessary information from an RSS Feed.  Next up is to actually bind it to the desired control.  Haven't gotten around to it yet, but I'm hoping that it is much easier.  It certainly shouldn't be much more challenging.

Presentation at TVBUG

For those of you in the greater Toronto area, I will be giving a presentation on Designing Service Oriented Architecture Based Applications at the Toronto Visual Basic Users Group meeting on March 16.  For more information, check out their web site at http://www.tvbug.com.

Jumping the Hurdle: Moving from OO to SOA

For all those who are interested, I have just posted the latest article in my series on service-oriented architecture, called Jumping the Hurdle: Moving from OO to SOA. Read, learn, be amused, be bored.  Feel free to let me know where on that spectrum you fall.

The First Article of the New Year

Just to let everyone know, I have just posted my first article of the new year, A Spoonful of Web Service's Alphabet Soup.  It is the first in a series that intends to delve into the details of creating a commercial-grade services-oriented architecture.