My First CodePlex Project!

A few minutes ago I just finalized my first CodePlex project.  While working on the ever-mysterious Infrastructure 2010 project, I needed to integrate the Live Meeting API into an application we are using.  So I decided to stick it into it’s own assembly for reuse.

I also figured that since it’s a relatively simple project, and because for the life of me I couldn’t find a similar wrapper, I would open source it.  Maybe there is someone out there who can benefit from it.

The code is ugly, but it works.  I suspect I will continue development, and clean it up a little.  With that being said:

  • It needs documentation (obviously).
  • All the StringBuilder stuff should really be converted to XML objects
  • It need's cleaner exception handling
  • It needs API versioning support
  • It needs to implement more API functions

Otherwise it works like a charm.  Check it out!

Visual Studio Output Window Auto-Scrolling

Just a quick one here.

Have you ever been using the output window in Visual Studio, scrolled up, and then lost the auto-scroll functionality?  It’s really annoying when you have a thousand things coming out through the likes of Debug.Write, or even if it’s just a massive build.

To re-enable autoscrolling, while in the output window just hit CTRL+END.

Kinda wished I knew that a year ago…

Six Simple Development Rules (for Writing Secure Code)

I wish I could say that I came up with this list, but alas I did not.  I came across it on the Assessment, Consulting & Engineering Team blog from Microsoft, this morning.  They are a core part of the Microsoft internal IT Security Group, and are around to provide resources for internal and external software developers.  These 6 rules are key to developing secure applications, and they should be followed at all times.

Personally, I try to follow the rules closely, and am working hard at creating an SDL for our department.  Aside from Rule 1, you could consider each step a sort of checklist for when you sign off, or preferably design, the application for production.

--

Rule #1: Implement a Secure Development Lifecycle in your organization.

This includes the following activities:

  • Train your developers, and testers in secure development and secure testing respectively
  • Establish a team of security experts to be the ‘go to’ group when people want advice on security
  • Implement Threat Modeling in your development process. If you do nothing else, do this!
  • Implement Automatic and Manual Code Reviews for your in-house written applications
  • Ensure you have ‘Right to Inspect’ clauses in your contracts with vendors and third parties that are producing software for you
  • Have your testers include basic security testing in their standard testing practices
  • Do deployment reviews and hardening exercises for your systems
  • Have an emergency response process in place and keep it updated

If you want some good information on doing this, email me and check out this link:
http://www.microsoft.com/sdl

Rule #2: Implement a centralized input validation system (CIVS) in your organization.

These CIVS systems are designed to perform common input validation on commonly accepted input values. Let’s face it, as much as we’d all like to believe that we are the only ones doing things like, registering users, or recording data from visitors it’s actually all the same thing.

When you receive data it will very likely be an integer, decimal, phone number, date, URI, email address, post code, or string. The values and formats of the first 7 of those are very predictable. The string’s are a bit harder to deal with but they can all be validated against known good values. Always remember to check for the three F’s; Form, Fit and Function.

  • Form: Is the data the right type of data that you expect? If you are expecting a quantity, is the data an integer? Always cast data to a strong type as soon as possible to help determine this.
  • Fit: Is the data the right length/size? Will the data fit in the buffer you allocated (including any trailing nulls if applicable). If you are expecting and Int32, or a Short, make sure you didn’t get an Int64 value. Did you get a positive integer for a quantity rather than a negative integer?
  • Function: Can the data you received be used for the purpose it was intended? If you receive a date, is the date value in the right range? If you received an integer to be used as an index, is it in the right range? If you received an int as a value for an Enum, does it match a legitimate Enum value?

In a vast majority of the cases, string data being sent to an application will be 0-9, a-z, A-Z. In some cases such as names or currencies you may want to allow –, $, % and ‘. You will almost never need , <> {} or [] unless you have a special use case such as http://www.regexlib.com in which case see Rule #3.

You want to build this as a centralized library so that all of the applications in your organization can use it. This means if you have to fix your phone number validator, everyone gets the fix. By the same token, you have to inspect and scrutinize the crap out of these CIVS to ensure that they are not prone to errors and vulnerabilities because everyone will be relying on it. But, applying heavy scrutiny to a centralized library is far better than having to apply that same scrutiny to every single input value of every single application.  You can be fairly confident that as long as they are using the CIVS, that they are doing the right thing.

Fortunately implementing a CIVS is easy if you start with the Enterprise Library Validation Application Block which is a free download from Microsoft that you can use in all of your applications.

Rule #3: Implement input/output encoding for all externally supplied values.

Due to the prevalence of cross site scripting vulnerabilities, you need to encode any values that came from an outside source that you may display back to the browser. (even embedded browsers in thick client applications). The encoding essentially takes potentially dangerous characters like < or > and converts them into their HTML, HTTP, or URL equivalents.

For example, if you were to HTTP encode <script>alert(‘XSS Bug’)</script> it would look like: &lt;script&gt;alert('XSS Bug')&lt;/script&gt;  A lot of this functionality is build into the .NET system. For example, the code to do the above looks like:

Server.HtmlEncode("<script>alert('XSS Bug')</script>");

However it is important to know that the Server.HTMLEncode only encodes about 4 of the nasty characters you might encounter. It’s better to use a more ‘industrial strength’ library like the Anti Cross Site Scripting library. Another free download from Microsoft. This library does a lot more encoding and will do HTTP and URI encoding based on a white list. The above encoding would look like this in AntiXSS

using Microsoft.Security.Application;
AntiXss.HtmlEncode("<script>alert('XSS Bug')</script>");

You can also run a neat test system that a friend of mine developed to test your application for XSS vulnerabilities in its outputs. It is aptly named XSS Attack Tool.

Rule #4: Abandon Dynamic SQL

There is no reason you should be using dynamic SQL in your applications anymore. If your database does not support parameterized stored procedures in one form or another, get a new database.

Dynamic SQL is when developers try to build a SQL query in code then submit it to the DB to be executed as a string rather than calling a stored procedures and feeding it the values. It usually looks something like this:

(for you VB fans)

dim sql
sql = "Select ArticleTitle, ArticleBody FROM Articles WHERE ArticleID = "
sql = sql & request.querystring("ArticleID")
set results = objConn.execute(sql)

In fact, this article from 2001 is chock full of what NOT to do. Including dynamic SQL in a stored procedure.

Here is an example of a stored procedure that is vulnerable to SQL Injection:

Create Procedure GenericTableSelect @TableName VarChar(100)
AS
Declare @SQL VarChar(1000)
SELECT @SQL = 'SELECT * FROM '
SELECT @SQL = @SQL + @TableName
Exec ( @SQL) GO

See this article for a look at using Parameterized Stored Procedures.

Rule #5: Properly architect your applications for scalability and failover

Applications can be brought down by a simple crash. Or a not so simple one. Architecting your applications so that they can scale easily, vertically or horizontally, and so that they are fault tolerant will give you a lot of breathing room.

Keep in mind that fault tolerant is not just a way to say that they restart when they crash. It means that you have a proper exception handling hierarchy built into the application.  It also means that the application needs to be able to handle situations that result in server failover. This is usually where session management comes in.

The best fault tolerant session management solution is to store session state in SQL Server.  This also helps avoid the server affinity issues some applications have.

You will also want a good load balancer up front. This will help distribute load evenly so that you won’t run into the failover scenario often hopefully.

And by all means do NOT do what they did on the site in the beginning of this article. Set up your routers and switches to properly shunt bad traffic or DOS traffic. Then let your applications handle the input filtering.

Rule #6: Always check the configuration of your production servers

Configuration mistakes are all too popular. When you consider that proper server hardening and standard out of the box deployments are probably a good secure default, there are a lot of people out there changing stuff that shouldn’t be. You may have remembered when Bing went down for about 45 minutes. That was due to configuration issues.

To help address this, we have released the Web Application Configuration Auditor (WACA). This is a free download that you can use on your servers to see if they are configured according to best practice. You can download it at this link.

You should establish a standard SOE for your web servers that is hardened and properly configured. Any variations to that SOE should be scrutinised and go through a very thorough change control process. Test them first before turning them loose on the production environment…please.

So with all that being said, you will be well on your way to stopping the majority of attacks you are likely to encounter on your web applications. Most of the attacks that occur are SQL Injection, XSS, and improper configuration issues. The above rules will knock out most of them. In fact, Input Validation is your best friend. Regardless of inspecting firewalls and things, the applications is the only link in the chain that can make an intelligent and informed decision on if the incoming data is actually legit or not. So put your effort where it will do you the most good.

ASP.NET WebForms are NOT Being Overthrown by MVC

It’s always a fun day when the man himself, ScottGu responds to my email.  Basically it all started last week at Techdays in Toronto (pictures to follow, I promise). 

Quite a few people asked me about MVC, and whether or not it will replace Web Forms.  My response was that it wouldn’t, but I didn’t have any tangible proof.  I discussed new features in .NET 4.0, and how the development is still going strong for future releases.  Some didn’t buy it.

So, earlier today I emailed Scott and asked him for proof.  This was his response:

Hi Steve,

Web Forms is definitely not going away – we are making substantial improvements to it with ASP.NET 4.0 (I’m doing a blog series on some of the improvements now).  ASP.NET MVC provides another option people can use for their UI layer – but it is simply an option, not a replacement.

In terms of the dev team size, the number of people on the ASP.NET team working on WebForms and MVC is actually about equal.  All of the core infrastructure investments (security, caching, config, deployment, etc) also apply equally to both.

Now, MVC is new.  MVC is powerful.  MVC is pretty freakin cool in what it can do.  But it won’t replace WebForms.  Frankly, I like WebForms.  MVC does have it’s place though.  I can see a lot benefits to using it.  It alleviates a lot of boilerplate code in certain development architectures, and that is never a bad thing.

Long Live WebForms!

Consultation to Salary &amp;ndash; Theoretical Head Banging Meets the Real World

A few weeks ago, six or so, I was offered a position as a Software Developer for the Woodbine Entertainment Group.  The position looked appealing so I accepted the job offer.  I am in a probationary period for the next four months and a bit.  Anything I say can be grounds for firing me.  Never liked that part about non-contract jobs.  Ah well.

Woodbine is an interesting company.  I knew very little about it until I got word of the job.  Seems I was the only one in Canada who didn’t know the company.  My grandmother, who moved to California 50 years ago, knew about the company.  Even used to bet there – well, the Woodbine Race Track, before it moved.  It has an interesting history.

It is migrating to be a Microsoft shop, from a more Novell focused infrastructure.  We are working towards standardizing on .NET for our custom applications.

The one thing that caught my eye with Woodbine is that the company is the technology leader for Horse Racing.  Not just in Canada, but throughout the world.  Our services can let you place a bet live, on a track in Australia, and see results immediately.  Can you imagine the infrastructure required for such a feat?  It’s sweet!  The business-people behind this are really keen on letting technology do it’s thing, so we can make money.  Lots of money.  See our Annual Reports on that.  Check back for latest numbers.

Now, some of you may have noticed that our Corporate Portal is written in what looks to be Classic ASP.  For all intensive purposes, it is.  Archive.org shows the portal went live in 2001, and had a major rebuild in 2003.  Since then incremental changes have taken place, most of which have been built using ASP.NET.  We are working on the new portal.  All I can say at the moment is: it’s going to be awesome.  So awesome that a new word will need to created to contain all of its awesomeness.  HorsePlayer Interactive is pretty amazing, but I’d like to think this new site will be just that much more awesomer.  Yes, I said awesomer.

As for the nature of this site, it won’t change.  I’ll still post my thoughts and experiences.  I might need to change stories a little to protect the innocent, but it’s all in good fun.  I may be forced to post details of how horse racing actually works, because I’m still not sure I get all the facets of it.  In time.

More to follow.

What Makes us Want to Program? Part 4

In my previous post, I started talking about using Microsoft technologies over PHP and open source technologies.  There were a couple reasons why I chose to make the move.  First, from a development perspective, everything was object oriented.  PHP was just getting started with OOP at the time, and it wasn’t all that friendly.  Second, development time was generally cut in at least half, because of the built in controls of ASP.NET.  Third, the end result was a more rich application experience for the same reason.  The final reason comes down to the data aspect.

Pulling data from a database in PHP wasn’t easy to do.  The built in support was for MySQL, with very little, if next to nothing for SQL Server.  In a lot of cases that isn’t always a bad thing.  MySQL is free.  You can’t argue with that.  however, MySQL wasn’t what you would call ACID compliant.  Defined, MySQL did not have the characteristics of being Atomic, Consistent, Isolated, and Durable.  Essentially, when data goes missing, there is nothing you can do about it.  SQL Server on the other hand is very ACID compliant.  This is something you want.  Period.

Once .NET 2.0 was released, a whole new paradigm came into play for data in a web application.  It was easy to access!  No more, or at least next to very little boiler plate coding was necessary for data access now.  Talk about a selling point.  Especially when the developer in question is 16 going on 17.

Now that I didn’t need to worry about data access code, I could start working on figuring out SQL.  At the time t-SQL scared the crap out of me.  My brain just couldn’t work around datasets.  The idea of working with multiple pieces of data at once was foreign.  I understood single valued iterations.  A for loop made sense to me.  SELECTs and JOINs confused me.  Mind you, I didn’t start Statistics in math until the following year.  Did SQL help with statistics, or did statistics help me finally figure out SQL?  It’s a chicken and the egg paradox.

So here I am, 17 years old, understanding multiple languages, building dozens of applications, and attending developer conferences all the while managing my education in High School.  Sweet.  I have 3 years until the next release of Visual Studio comes out.  It was here that I figured I should probably start paying more attention in school.  It’s not so much that I wasn’t paying attention, it’s just that I didn’t care enough.  I put in just enough effort to skate through classes with a passing mark.  It was also at this point in time that I made an interesting supposition.

Experts tend to agree that people who are programming geniuses are also good at math and critical thinking or reasoning.  Not one or the other, but both.  Now I’m not saying I’m a programming genius, but I suck at math.  It was just never in the cards.  But, according to all those High School exams and the psychological profiling they gather from them, my Critical Thinking and Reasoning skills are excellent.  Top 10% in Canada according to the exam results.  My math skills sit around top 20-30% depending on the type.

Neurologists place this type of thinking in the left hemisphere of the brain.  The left brain is associated with verbal, logical, and analytical thinking. It excels in naming and categorizing things, symbolic abstraction, speech, reading, writing, arithmetic.  Those who live in the left brain are very linear.  Perfect for a software developer.

The supposition I made had more to do with the Pre-Frontal Cortex of the brain.  It does a lot of work, some of which is planning complex cognitive behaviors.  Behaviors like making a list, calculating numbers, abstracting thoughts, etc.  It plans out the processes our brains use to get things done.  This is true for both sides of the brain.  So, suppose you are left brain-oriented.  You are predisposed to be good at development.  Now, suppose your Pre-Frontal Cortex is very well developed, more so than the average person.  It could be reasoned that part of being a programming genius is having a well developed Pre-Frontal Cortex.

So why does this make us want to program?  Find out in Part 5.

What Makes us Want to Program? Part 3

In my second post I discussed my run in with ASP, and how PHP was far better.  I ended the post talking about an invitation to a Microsoft event.  This was an interesting event.  Greg and I were the only people under 30 there.  When that’s a 15 year difference, things get interesting.  Especially when you need your mother to drive you there…  The talk was a comparison between Microsoft based technologies and Linux based technologies.  The presenter was a 10 year veteran of IBM, working on their Linux platform, who then moved to Microsoft.  For the life of me I can’t remember his name.

His goal was simple.  Disprove myths around Linux costs versus Windows costs.  It was a very compelling argument.  The event was based around the Windows Compare campaign.  It was around this time that Longhorn (Longhorn that turned into Vista, not Server 2008) was in pre-beta soon to go beta, and after discussing it with Greg, we decided to probe the presenter for information about Longhorn.  In a situation like that, the presenter either gets mad, or becomes really enthusiastic about the question.  He certainly didn’t get mad.

Throughout the rest of the talk, the presenter made some jokes at mine and Greg’s expense, which was all in good fun.  Based on that, we decided to go one step further to ask how we can get the latest Longhorn build, at one of the breaks.  the conversation went something like this:

Me: So how do people get copies of the latest build for Longhorn?
Presenter: Currently those enrolled in the MSDN Licensing program can get the builds.
Me: Ok, how does one join such a licensing program?
Presenter: Generally you buy them.
Me: How much?
Presenter: A couple thousand…
Me: Ok let me rephrase the question.  How does a student, such as myself and my friend Greg here, get a the latest build of Longhorn when we don’t have an MSDN subscription, nor the money to buy said subscription?
Presenter: *Laughs* Oh.  Go talk to Alec over there and tell him I said to give you a student subscription.
Me:  Really?  Cool!

Six months later Greg and I some how got MSDN Premium Subscriptions.  We had legal copies of almost every single piece of Microsoft software ever commercially produced.  Visual Studio 2005 was still in beta, so I decided to try it out.  I was less than impressed with Visual Studio 2003, but really liked ASP.NET, so I wanted to see what 2005 had in store.  At the time PHP was still my main language, but after the beta of 2005, I immediately switched to C#.  I had known about C# for a while, and understood the language fairly well.  It was .NET 1.1 that never took for me.  That, and I didn’t have a legal copy of Visual Studio 2003 at the time.

Running a Longhorn beta build, with Visual Studio 2005 beta installed, I started playing with ASP.NET 2.0, and built some pretty interesting sites.  The first was a Wiki type site, designed for medical knowledge (hey, it takes a lot to kill a passion of mine).  It never saw the light of day on the interweb, but it certainly was a cool site.  Following that were a bunch of test sites that I used to experiment with the data controls.

It wasn’t until the release of SQL Server 2005 that I started getting interested in data.  Which I will discuss in the my next post.

Open Source Windows

Some days you just have to shake your head and wonder. As it turns out, I'm a little late to hear about this, but nonetheless, I'm still shaking my head.

It turns out that Windows has gone open source. And (!!) it's not being made by Microsoft anymore. Well, Windows™ is still made by Microsoft. Windows is now made by a group under the gise of ReactOS.
ReactOS® is a free, modern operating system based on the design of Windows® XP/2003. Written completely from scratch, it aims to follow the Windows® architecture designed by Microsoft from the hardware level right through to the application level. This is not a Linux based system, and shares none of the unix architecture.
So essentially, these people are taking the Windows architecture (based on XP/2003), and redesigning it from scratch. Or rather, are re-coding it from scratch, because redesigning would imply making something different. Sounds vaguely familiar to, oh, something called Vista. Except uglier.



Now, that nagging feeling we are all getting right about now should be visualized as a pack of rabid lawyers. Isn't this considered copyright infringement? They outright define the product as a copy.

And what about the end users? Are all programs designed to run on Windows supposed to be able to run on this ReactOS? Why bother with testing? The XP architecture is now almost 8 years old by now. That means anything designed to run on Vista, or soon to be designed to run on Windows 7, wouldn't stand a snowballs chance in hell, running on ReactOS.

I would love to see how a .NET application runs on it.

ADO.NET Entity Framework and SQL Server 2008

Do you remember the SubSonic project? The Entity Framework is kind of like that. You can create an extensible and customizable data model from any type of source. It takes the boiler plate coding away from developing Data Access Layers.

Entity is designed to seperate how data is stored and how data is used. It's called an Object-Relational Mapping framework. You point the framework at the source, tell it what kind of business objects you want, and poof: you have an object model. Entity is also designed to play nicely with LINQ. You can use it as a data source when querying with LINQ. In my previous post, the query used NorthwindModEntities as a data source. It is an Entity object.

Entity Framework
Courtesy of Wikipedia

The Architecture, as defined in the picture:

  • Data source specific providers, which abstracts the ADO.NET interfaces to connect to the database when programming against the conceptual schema.
  • Map provider, a database-specific provider that translates the Entity SQL command tree into a query in the native SQL flavor of the database. It includes the Store specific bridge, which is the component that is responsible for translating the generic command tree into a store-specific command tree.
  • EDM parser and view mapping, which takes the SDL specification of the data model and how it maps onto the underlying relational model and enables programming against the conceptual model. From the relational schema, it creates views of the data corresponding to the conceptual model. It aggregates information from multiple tables in order to aggregate them into an entity, and splits an update to an entity into multiple updates to whichever table contributed to that entity.
  • Query and update pipeline, processes queries, filters and update-requests to convert them into canonical command trees which are then converted into store-specific queries by the map provider.
  • Metadata services, which handle all metadata related to entities, relationships and mappings.
  • Transactions, to integrate with transactional capabilities of the underlying store. If the underlying store does not support transactions, support for it needs to be implemented at this layer.
  • Conceptual layer API, the runtime that exposes the programming model for coding against the conceptual schema. It follows the ADO.NET pattern of using Connection objects to refer to the map provider, using Command objects to send the query, and returning EntityResultSets or EntitySets containing the result.
  • Disconnected components, which locally caches datasets and entity sets for using the ADO.NET Entity Framework in an occasionally connected environment.
    • Embedded database: ADO.NET Entity Framework includes a lightweight embedded database for client-side caching and querying of relational data.
  • Design tools, such as Mapping Designer are also included with ADO.NET Entity Framework which simplifies the job on mapping a conceptual schema to the relational schema and specifying which properties of an entity type correspond to which table in the database.
  • Programming layers, which exposes the EDM as programming constructs which can be consumed by programming languages.
  • Object services, automatically generate code for CLR classes that expose the same properties as an entity, thus enabling instantiation of entities as .NET objects.
  • Web services, which expose entities as web services.
  • High level services, such as reporting services which work on entities rather than relational data.

.NET Visualized: Treemap and Dependency Matrix

In case you haven’t seen this, check out MVP Patrick Smacchia’s visualizations of the .NET Framework, including an NDepend analysis of the number of Types, a Treemap view, and a dependency matrix for the entire framework. The dependency matrix is, IMO, impressively sparse. Also interesting but perhaps not surprising is that PresentationFramework represents the largest [...]