Visual Studio 2008 SP1 Beta & SQL Server 2008

A quick heads up to let you know that VS 2008 Service Pack 1 is now available (links below). It typically takes a couple of months from this point before we'll see a final release.

This Service Pack includes new cool feature:

One interesting point is that MS is going to simultaneously ship SQL Server 2008 which actually has a hard dependency on SP1.

I thought I’d take a moment to highlight some new features that Dev’s would care about in SQL Server 2008.

  • Change Data Capture: Async “triggers” capture the before/after snapshot of row level changes and writes them to Change Tables that you can query in your app. They aren’t real triggers as this asynchronously reads the transaction log.
  • Granular control of encryption, right through to the database level without any application changes required.
  • Resource Governor – very helpful when you allow users to write adhoc queries / reports against your OLTP database. Allows a DBA to assert resource limits & priorities.
  • Plan Freezing – allows you to lock down query plans to promote stable query plans across disparate hardware, server upgrades, etc.
  • New Date, and Time data types, no longer just DateTime types that you have to manually parse out the time or date to just get the real data you want.
  • DataTimeOffset – is a time zone aware datetime.
  • Table Value Parameters to procs – ever want to pass a result set as an arg to a proc?
  • Hierarchy ID is a new system type for storing nodes in a hierarchy….implemented as a CLR User Defined Type.
  • FileStream Data type allows blobish data to be surfaced in the database, but physically stored on the NTFS file system. ….but with complete transactional consistency with the relational data and backup integration.
  • New Geographic data support, store spatial data such as polygons, points and lines, and long/lat data types.
  • Merge SQL statement allows you to insert, or update if a row already exists.
  • New reporting services features such as access to reports from within Word & Excel, better SharePoint integration

Personally, haven't spent any time with SQL Server 2008 but that's a great set of new features that I can hardly wait to start using in real-world applications.

Downloads

· VS 2008 SP1 : http://download.microsoft.com/download/7/3/8/7382EA08-4DD6-4134-9B92-8585A5B07973/VS90sp1-KB945140-ENU.exe

· .NET 3.5 SP1 : http://download.microsoft.com/download/8/f/c/8fc1fe13-55de-4bf5-b43e-375daf01452e/dotNetFx35setup.exe

· Express 2008 with SP1:

o http://download.microsoft.com/download/F/E/7/FE754BA4-140B-413C-933F-8D35FB150F12/vbsetup.exe

o http://download.microsoft.com/download/F/E/7/FE754BA4-140B-413C-933F-8D35FB150F12/vcsetup.exe

o http://download.microsoft.com/download/F/E/7/FE754BA4-140B-413C-933F-8D35FB150F12/vcssetup.exe

o http://download.microsoft.com/download/F/E/7/FE754BA4-140B-413C-933F-8D35FB150F12/vnssetup.exe

· TFS 2008 SP1: http://download.microsoft.com/download/a/e/2/ae2eb0ff-e687-4221-9c3e-9165a942bc1c/TFS90sp1-KB949786.exe

Feedback Forum: http://go.microsoft.com/fwlink/?LinkId=119125

 

The Entity Framework vs. The Data Access Layer (Part 1: The EF as a DAL)

In Part 0: Introduction of this series after asking the question "Does the Entity Framework replace the need for a Data Access Layer?", I waxed lengthy about the qualities of a good data access layer. Since that time I've received a quite a few emails with people interested in this topic. So without further adieu, let's get down to the question at hand.

So let's say you go ahead and create an Entity Definition model (*.edmx) in Visual Studio and have the designer generate for you a derived ObjectContext class and an entity class for each of your tables, derived from EntityObject. This one to one table mapping to entity class is quite similar to LINQ to SQL but the mapping capabilities move well beyond this to support advanced data models. This is at the heart of why the EF exists: Complex Types, Inheritance (Table per Type, Table per Inheritance Hierarchy), Multiple Entity Sets per Type, Single Entity Mapped to Two Tables, Entity Sets mapped to Stored Procedures or mapping to a hand-crafted query, expressed as either SQL or Entity SQL. EF has a good story for a conceptual model over top of our physical databases using Xml Magic in the form of the edmx file - and that's why it exists.

So to use the Entity Framework as your data access layer, define your model and then let the EdmGen.exe tool do it's thing to the edmx file at compile time and we get the csdl, ssdl, and msl files - plus the all important code generated entity classes. So using this pattern of usage for the Entity Framework, our data access layer is complete. It may not be the best option for you, so let's explore the qualities of this solution.

To be clear, the assumption here is that our data access layer in this situation is the full EF Stack: ADO.NET Entity Client, ADO.NET Object Services, LINQ to Entities, including our model (edmx, csdl, ssdl, msl) and the code generated entities and object context. Somewhere under the covers there is also the ADO.NET Provider (SqlClient, OracleClient, etc.)

image

To use the EF as our DAL, we would simply execute code similar to this in our business layer.

var db = new AdventureWorksEntities();
var activeCategories = from category in db.ProductCategory
                 where category.Inactive != true
                 orderby
category.Name
                 select category;

How Do "EF" Entities Fit In?

If you're following along, you're probably asking exactly where is this query code above being placed. For the purposes of our discussion, "business layer" could mean a business object or some sort of controller. The point to be made here is that we need to think of Entities as something entirely different from our Business Objects.

Entity != Business Object

In this model, it is up to the business object to ask the Data Access Layer to project entities, not business objects, but entities.

This is one design pattern for data access, but it is not the only one. A conventional business object that contains its own data, and does not separate that out into an entity can suffer from tight bi-directional coupling between the business and data access layer. Consider a Customer business object with a Load method. Customer.Load() would in turn instantiate a data access component, CustomerDac and call the CustomerDac's Load or Fill method. To encapsulate all the data access code to populate a customer business object, the CustomerDac.Load method would require knowledge of the structure the Customer business object and hence a circular dependency would ensue.

The workaround, if you can call it that, is to put the business layer and the data access layer in the same assembly - but there goes decoupling, unit testing and separation of concerns out the window.

Another approach is to invert the dependency. The business layer would contain data access interfaces only, and the data access layer would implement those interfaces, and hence have a reverse dependency on the business layer. Concrete data access objects are instantiated via a factory, often combined with configuration information used by an Inversion of Control container. Unfortunately, this is not all that easy to do with the EF generated ObjectContext & Entities.

Or, you do as the Entity Framework implies and separate entities from your business objects. If you've used typed DataSets in the past, this will seem familiar you to you. Substitute ObjectContext for SqlConnection and SqlDataAdapter, and the pattern is pretty much the same.

Your UI presentation layer is likely going to bind to your Entity classes as well. This is an important consideration. The generated Entity classes are partial classes and can be extended with your own code. The generated properties (columns) on an entity also have event handlers created for changing and changed events so you can also wire those up to perform some column level validation. Notwithstanding, you may want to limit your entity customizations to simple validation and keep the serious business logic in your business objects. One of these days, I'll do another blog series on handing data validation within the Entity Framework.

How does this solution stack up?

How are database connections managed?

thumbs up Using the Entity Framework natively itself, the ObjectContext takes care of opening & closing connections for you - as needed when queries are executed, and during a call to SaveChanges. You can get access to the native ADO.NET connection if need be to share a connection with other non-EF data access logic. The nice thing however is that, for the most part, connection strings and connection management are abstracted away from the developer.

thumbs down A word of caution however. Because the ObjectContext will create a native connection, you should not wait to let the garbage collector free that connection up, but rather ensure that you dispose of the ObjectContext either explicitly or with a using statement.

Are all SQL Queries centralized in the Data Access Layer?

thumbs down By default the Entity Framework dynamically generates store specific SQL on the fly and therefore, the queries are not statically located in any one central location. Even to understand the possible queries, you'd have to walk through all of your business code that hits the entity framework to understand all of the potential queries.

But why would you care? If you have to ask that question, then you don't care. But if you're a DBA, charged with the job of optimizing queries, making sure that your tables have the appropriate indices, then you want to go to one central place to see all these queries and tune them if necessary. If you care strongly enough about this, and you have the potential of other applications (perhaps written in other platforms), then you likely have already locked down the database so the only access is via Stored Procedures and hence the problem is already solved.

Let's remind ourselves that sprocs are not innately faster than dynamic SQL, however they are easier to tune and you also have the freedom of using T-SQL and temp tables to do some pre-processing of data prior to projecting results - which sometimes can be the fastest way to generate some complex results. More importantly, you can revoke all permissions to the underlying tables and only grant access to the data via Stored Procedures. Locking down a database with stored procedures is almost a necessity if your database is oriented as a service, acting as an integration layer between multiple client applications. If you have multiple applications hitting the same database, and you don't use stored procedures - you likely have bigger problems. 

In the end, this is not an insurmountable problem. If you are already using Stored Procedures, then by all means you can map those in your EDM. This seems like the best approach, but you could also embed SQL Server (or other provider) queries in your SSDL using a DefiningQuery.

Do changes in one part of the system affect others?

It's difficult to answer this question without talking about the possible changes.

thumbs up Schema Changes: The conceptual model and the mapping flexibility, even under complex scenarios is a strength of the entity framework. Compared to other technologies on the market, with the EF, your chances are as good as they're going to get that a change in the database schema will have minimal impact on your entity model, and vice versa.

thumbs up Database Provider Changes: The Entity Framework is database agnostic. It's provider model allows for easily changing from SQL Server, to Oracle, to My Sql, etc. via connection strings. This is very helpful for ISVs whose product must support running on multiple back-end databases.

thumbs down Persistence Ignorance: What if the change you want in one part of the system is to change your ORM technology? Maybe you don't want to persist to a database, but instead call a CRUD web service. In this pure model, you won't be happy. Both your Entities and your DataContext object inherit from base classes in the Entity Framework's System.Data.Objects namespace. By making references to these, littered throughout your business layer, decoupling yourself from the Entity Framework will not be an easy task.

thumbs down Unit Testing: This is only loosely related to the question, but you can't talk about PI without talking about Unit Testing. Because the generated entities do not support the use of Plain Old CLR Objects (POCO), this data access model is not easily mocked for unit testing.

Does the DAL simplify data access?

thumbs up Dramatically. Compared to classic ADO.NET, LINQ queries can be used for typed results & parameters, complete with intelli-sense against your conceptual model, with no worries about SQL injection attacks.

thumbs up As a bonus, what you do get is query composition across your domain model. Usually version 1.0 of a convention non-ORM data access layer provides components for each entity, each supporting crud behaviour. Consider a scenario where you need to show all of the Customers within a territory, and then you need to show the last 10 orders for each Customer. Now I'm not saying you'd do this, but what I've commonly seen is that while somebody might write a CustomerDac.GetCustomersByTerritory() method, and they might write an OrderDac.GetLastTenOrders(), they would almost never write a OrderDac.GetLastTenOrdersForCustomersInTerritory() method. Instead they would simply iterate over the collection of customers found by territory and call the GetLastTenOrders() over and over again. Obviously this is "good" resuse of the data access logic, however it does not perform very well.

Fortunately, through query composition and eager loading, we can cause the Entity Framework (or even LINQ to SQL) to use a nested subquery to bring back the last 10 orders for each customer in a given territory in a single round trip, single query. Wow! In a conventional data access layer you could, and should write a new method to do the same, but by writing yet another query on the order table, you'd be repeating the mapping between the table and your objects each time.

Layers, Schmayers: What about tiers?

thumbs down EDM generated entity classes are not very tier-friendly. The state of an entity, whether it is modified, new, or to be delete, and what columns have changed is managed by the ObjectContext. Once you take an entity and serialize it out of process to another tier, it is no longer tracked for updates. While you can re-attach an entity that was serialized back into the data access tier, because the entity itself does not serialize it's changed state (aka diff gram), you can not easily achieve full round trip updating in a distributed system. There are techniques for dealing with this, but it is going to add some plumbing code between the business logic and the EF...and make you wish you had a real data access layer, or something like Danny Simmons' EntityBag (or a DataSet).

Does the Data Access Layer support optimistic concurrency?

thumbs up Out of the box, yes, handily. Thanks to the ObjectContext tracking state, and the change tracking events injected into our code generated entity properties. However, keep in mind the caveat with distributed systems that you'll have more work to do if your UI is separated from your data access layer by one or more tiers.

How does the Data Access Layer support transactions?

thumbs up Because the Entity Framework builds on top of ADO.NET providers, transaction management doesn't change very much. A single call to ObjectContext.SaveChanges() will open a connection, perform all inserts, updates, and deletes across all entities that have changed, across all relationships and all in the correct order....and as you can imagine in a single transaction. To make transactions more granular than that, call SaveChanges more frequently or have multiple ObjectContext instances for each unit of work in progress. To broaden the scope of a transaction, you can manually enlist using a native ADO.NET provider transaction or by using System.Transactions.

Entity Framework Links for April, 2008

  • During the past month, Danny Simmons let us all officially know that SP1 of VS 2008/.NET Framework 3.5 will be the delivery mechanism for the Entity Framework and the Designer, and that we should see a beta of the entire SP1 very soon as well. No release dates yet.
  • Speaking of the next beta, there have been some improvements in the designer to support iterative development. Noam Ben-Ami talks about that here.
  • There is also a new ASP.NET EntityDataSource control coming in the next beta. Danny demo'd that at DevConnections, and Julie blogged about it here.
  • In April, Microsoft released the .NET 3.5 Enhancements Training Kit. This includes some preliminary labs on ASP.NET MVC, ASP.NET Dynamic Data, ASP.NET AJAX History, ASP.NET Silverlight controls, ADO.NET Data Services and last but certainly not least, the ADO.NET Entity Framework. Stay tuned for updates
  • Julie Lerman has created a spiffy pseudo-debug visualizer for Entity State. It's implemented as an extension method and not a true debug visualizer, but useful just the same.
  • Check out Ruurd Boeke's excellent post on Disconnected N-Tier objects using the Entity Framework. His sample solution is checked in to the EFContrib Project and he demonstrates using POCO classes, in his words "as persistence ignorant as I can get", serializing entities with no EF references on the clients, yet not losing full change tracking on the client - and using the same domain classes on the client and the server (one could argue this last point as being not being a desirable goal - but it does have it's place).

Visual Studio 2008 and Windows Server 2008 for aspiring Architects

imageAre you an architect or an aspiring architect interested in learning what's new this year for the MS platform?

On February 11th, 2008 come to the MS Canada Office in Mississauga and visit yours truly and some other dazzling speakers to learn more about Visual Studio 2008 and Windows Server 2008.

You can choose either the morning or afternoon session as your schedule permits. Only a 1/2 day out of your busy schedule and you'll know everything you need! Ok, well maybe not everything, but I hope that you'll be inspired to take the next steps in learning about technologies such as LINQ, Windows Communication Foundation and Windows Presentation Foundation and how you can make the most of these technologies in your applications.

Register Here.

VS 2008 at The Movies, Feb 7, 2008 Toronto Paramount

Posters_Codefather Our designer is having a field-day with this "at the Movies" theme for our upcoming review of Visual Studio 2008 being held Feb 7th from 8:30am-12:00pm @ the Paramount in Toronto. Grab a copy of this movie poster before it gets "whacked" by the lawyers.

Hope to see you there. Check out all the details after this link.

Visual Studio, SQL Server, and Windows Server 2008 Launch Events in Toronto

clip_image001

On February 27 in Toronto, MS Canada is hosting the official launch of the above mentioned products. The event will be all day long and in addition to a keynote from COO Kevin Turner, there will be some great breakout tracks running in parallel for IT Professionals, Developers, IT Managers, and Architects.

The event will be held at the Direct Energy Centre downtown. Of course ObjectSharp will have a booth there with some great offers for both our Training and Professional Services along with some awesome prize raffles so please stop by.

Also make sure to stop by the expert's area where several MVP's and speakers will be able to answer your individual questions including many of the MVP's from ObjectSharp.

You can also register for this event here along with all of the other cities and their events happening across Canada.

And don't forget, we're also doing a 1/2 day briefing for developers & architects on VS 2008 at the Paramount in Toronto on February 7th. You can view the details here.

TSPUG Presentation: Using Visual Studio 2008 for Developing SharePoint Workflows (and other stuff)

Last night I gave a presentation to the Toronto SharePoint Users Group on using Visual Studio 2008 to build SharePoint Workflows. I also covered a little bit on LINQ to SharePoint and WCF/WF integration at the end. Attached are my slides. Enjoy.

Visual Studio 2008 to ship by end of month!

In case you didn't catch this S. Somasegar announced today during his TechEd Developers Keynote in Barcelona that Visual Studio 2008 will ship by the end of this month (November!). Yeah! Most people were counting on this before the end of the year which mean December or early January so this comes as a nice surprise.

We're talking about some cool technology:

  • Visual Studio 2008 (all editions)
  • Team Foundation Server 2008
  • .NET Framework 3.5
  • Language Integrated Query (LINQ)

Now of course the best feature in Visual Studio 2008 is multi-targeting. This features allows you to continue to develop .NET 2.0 or 3.0 applications without migrating to 3.5. There are lots of great features if Visual Studio 2008 - even if you don't move to .NET 3.5:

And if you are a Team System User

  • SharePoint 2007/WSS 3.0 or MOSS support
  • Simplified Installation
  • Better Offline Support
  • A bunch of other stuff including Power Tool Rollups.

And don't worry - you can install VS 2008 side by side with VS 2005.

Don't let your VS2008 Beta2 VPC turn into a Pumpkin on Halloween at Midnight

Jeff Beehler blogs here and here that the Visual Studio 2008 Beta 2 VPCs are prematurely expiring on November 1st. If you're using your VPC for demos - you'll be limited to 2 hours before you have to shut down. If you're using it for real work - that will be annoying - especially if you are using it for your TFS server Follow Jeff's links to instructions on backing that up and moving it. Look for new VPCs next week - hopefully we'll see a Release Candidate shortly.

Update: Jeff has posted a link to updated VPCs here.

Toronto Architect Forum, this Thursday

This coming Thursday, Microsoft is hosting the annual Toronto Architect Forum at their offices in Mississauga. The target audience is architects that are *not* in the financial service industry. Here's the agenda:

8:00 - 8:30 am Breakfast and Registration
8:30 - 9:00 am Welcome by Mark Relph
9:00 - 9:30 am Architectural Agility as Business Value, Dave Remmer
9:30 - 10:30 am Office Business Applications, Mike Walker
10:30 - 10:45 am Break
10:45 - 12:00 pm Visual Studio 2008 “All Up”, Adam Gallant
12:00 - 1:00 pm Networking lunch
1:00 - 2:15 pm Architectural Implications of LINQ, Barry Gervin
2:15 - 2:30 pm Break
2:30 - 3:00 pm Project Experiences using AJAX, Amalan Ponnampalam
3:00 - 4:15 pm How to be an Effective Architect, Mohammad Akif
4:15 - 4:30 pm Wrap-up and Prize Draw

As you can, I've secured the ever so popular "right after lunch" time slot. I don't know if there are detailed abstracts online for each session, but here is mine:

LINQ: Architectural Implications

Support for Language Integrated Query in the .NET 3.5 Framework promises to simplify and unify querying operations across object collections, relational data, DataSets and XML. The opportunity to simplify or even eliminate the notion  of a data access layer is one many architects are considering. During this session we will quickly introduce the capabilities of LINQ, LINQ to SQL and the upcoming Entity Framework, and then discuss how this may affect the design of our data access logic moving forward.

Registration is still open - here.

Update - the event is for Architects not in the financial services industry (my mistake).