A few minutes ago I just finalized my first CodePlex project. While working
on the ever-mysterious Infrastructure 2010 project, I needed to integrate the Live
Meeting API into an application we are using. So I decided to stick it into
it’s own assembly for reuse.
I also figured that since it’s a relatively simple project, and because for the life
of me I couldn’t find a similar wrapper, I would open source it. Maybe there
is someone out there who can benefit from it.
The code is ugly, but it works. I suspect I will continue development, and clean
it up a little. With that being said:
-
It needs documentation (obviously).
-
All the StringBuilder stuff should really be converted to XML objects
-
It need's cleaner exception handling
-
It needs API versioning support
-
It needs to implement more API functions
Otherwise it works like a charm. Check
it out!
Just a quick one here.
Have you ever been using the output window in Visual Studio, scrolled up, and then
lost the auto-scroll functionality? It’s really annoying when you have a thousand
things coming out through the likes of Debug.Write, or even if it’s just a massive
build.
To re-enable autoscrolling, while in the output window just hit CTRL+END.
Kinda wished I knew that a year ago…
I wish I could say that I came up with this list, but alas I did not. I came
across it on the Assessment,
Consulting & Engineering Team blog from Microsoft, this morning. They
are a core part of the Microsoft internal IT Security Group, and are around to provide
resources for internal and external software developers. These 6 rules are key
to developing secure applications, and they should be followed at all times.
Personally, I try to follow the rules closely, and am working hard at creating an
SDL for our department. Aside from Rule 1, you could consider each step a sort
of checklist for when you sign off, or preferably design, the application for production.
--
Rule #1: Implement a Secure Development Lifecycle in your organization.
This includes the following activities:
-
Train your developers, and testers in secure development and secure testing respectively
-
Establish a team of security experts to be the ‘go to’ group when people want advice
on security
-
Implement Threat Modeling in your development process. If you do nothing else, do
this!
-
Implement Automatic and Manual Code Reviews for your in-house written applications
-
Ensure you have ‘Right to Inspect’ clauses in your contracts with vendors and third
parties that are producing software for you
-
Have your testers include basic security testing in their standard testing practices
-
Do deployment reviews and hardening exercises for your systems
-
Have an emergency response process in place and keep it updated
If you want some good information on doing this, email me and check out this link:
http://www.microsoft.com/sdl
Rule #2: Implement a centralized input validation system (CIVS) in your organization.
These CIVS systems are designed to perform common input validation on commonly accepted
input values. Let’s face it, as much as we’d all like to believe that we are the only
ones doing things like, registering users, or recording data from visitors it’s actually
all the same thing.
When you receive data it will very likely be an integer, decimal, phone number, date,
URI, email address, post code, or string. The values and formats of the first 7 of
those are very predictable. The string’s are a bit harder to deal with but they can
all be validated against known good values. Always remember to check for the three
F’s; Form, Fit and Function.
-
Form: Is the data the right type of data that you expect? If you are expecting a quantity,
is the data an integer? Always cast data to a strong type as soon as possible to help
determine this.
-
Fit: Is the data the right length/size? Will the data fit in the buffer you allocated
(including any trailing nulls if applicable). If you are expecting and Int32, or a
Short, make sure you didn’t get an Int64 value. Did you get a positive integer for
a quantity rather than a negative integer?
-
Function: Can the data you received be used for the purpose it was intended? If you
receive a date, is the date value in the right range? If you received an integer to
be used as an index, is it in the right range? If you received an int as a value for
an Enum, does it match a legitimate Enum value?
In a vast majority of the cases, string data being sent to an application will be
0-9, a-z, A-Z. In some cases such as names or currencies you may want to allow –,
$, % and ‘. You will almost never need , <> {} or [] unless you have a special
use case such as http://www.regexlib.com in
which case see Rule #3.
You want to build this as a centralized library so that all of the applications in
your organization can use it. This means if you have to fix your phone number validator,
everyone gets the fix. By the same token, you have to inspect and scrutinize the crap
out of these CIVS to ensure that they are not prone to errors and vulnerabilities
because everyone will be relying on it. But, applying heavy scrutiny to a centralized
library is far better than having to apply that same scrutiny to every single input
value of every single application. You can be fairly confident that as long
as they are using the CIVS, that they are doing the right thing.
Fortunately implementing a CIVS is easy if you start with the Enterprise
Library Validation Application Block which is a free download from Microsoft that
you can use in all of your applications.
Rule #3: Implement input/output encoding for all externally supplied values.
Due to the prevalence of cross site scripting vulnerabilities, you need to encode
any values that came from an outside source that you may display back to the browser.
(even embedded browsers in thick client applications). The encoding essentially takes
potentially dangerous characters like < or > and converts them into their HTML,
HTTP, or URL equivalents.
For example, if you were to HTTP encode <script>alert(‘XSS Bug’)</script>
it would look like: <script>alert('XSS Bug')</script>
A lot of this functionality is build into the .NET system. For example, the code to
do the above looks like:
Server.HtmlEncode("<script>alert('XSS Bug')</script>");
However it is important to know that the Server.HTMLEncode only encodes about 4 of
the nasty characters you might encounter. It’s better to use a more ‘industrial strength’
library like the Anti
Cross Site Scripting library. Another free download from Microsoft.
This library does a lot more encoding and will do HTTP and URI encoding based on a
white list. The above encoding would look like this in AntiXSS
using Microsoft.Security.Application;
AntiXss.HtmlEncode("<script>alert('XSS Bug')</script>");
You can also run a neat test system that a friend of mine developed to test your application
for XSS vulnerabilities in its outputs. It is aptly named XSS
Attack Tool.
Rule #4: Abandon Dynamic SQL
There is no reason you should be using dynamic SQL in your applications anymore. If
your database does not support parameterized stored procedures in one form or another,
get a new database.
Dynamic SQL is when developers try to build a SQL query in code then submit it to
the DB to be executed as a string rather than calling a stored procedures and feeding
it the values. It usually looks something like this:
(for you VB fans)
dim sql
sql = "Select ArticleTitle, ArticleBody FROM Articles WHERE ArticleID = "
sql = sql & request.querystring("ArticleID")
set results = objConn.execute(sql)
In fact, this
article from 2001 is chock full of what NOT to do. Including dynamic SQL in a
stored procedure.
Here is an example of a stored procedure that is vulnerable to SQL Injection:
Create Procedure GenericTableSelect @TableName VarChar(100)
AS
Declare @SQL VarChar(1000)
SELECT @SQL = 'SELECT * FROM '
SELECT @SQL = @SQL + @TableName
Exec ( @SQL) GO
See this article for a look at using
Parameterized Stored Procedures.
Rule #5: Properly architect your applications for scalability and failover
Applications can be brought down by a simple crash. Or a not so simple one. Architecting
your applications so that they can scale easily, vertically or horizontally, and so
that they are fault tolerant will give you a lot of breathing room.
Keep in mind that fault tolerant is not just a way to say that they restart when they
crash. It means that you have a proper exception handling hierarchy built into the
application. It also means that the application needs to be able to handle situations
that result in server failover. This is usually where session management comes in.
The best fault tolerant session management solution is to store session state in SQL
Server. This also helps avoid the server affinity issues some applications have.
You will also want a good load balancer up front. This will help distribute load evenly
so that you won’t run into the failover scenario often hopefully.
And by all means do NOT do what they did on the site in the beginning of this article.
Set up your routers and switches to properly shunt bad traffic or DOS traffic. Then
let your applications handle the input filtering.
Rule #6: Always check the configuration of your production servers
Configuration mistakes are all too popular. When you consider that proper server hardening
and standard out of the box deployments are probably a good secure default, there
are a lot of people out there changing stuff that shouldn’t be. You may have remembered
when Bing went down for about 45 minutes. That was due to configuration issues.
To help address this, we have released the Web Application Configuration Auditor (WACA).
This is a free download that you can use on your servers to see if they are configured
according to best practice. You can download it at this link.
You should establish a standard SOE for your web servers that is hardened and properly
configured. Any variations to that SOE should be scrutinised and go through a very
thorough change control process. Test them first before turning them loose on the
production environment…please.
So with all that being said, you will be well on your way to stopping the majority
of attacks you are likely to encounter on your web applications. Most of the attacks
that occur are SQL Injection, XSS, and improper configuration issues. The above rules
will knock out most of them. In fact, Input Validation is your best friend. Regardless
of inspecting firewalls and things, the applications is the only link in the chain
that can make an intelligent and informed decision on if the incoming data is actually
legit or not. So put your effort where it will do you the most good.
It’s always a fun day when the man himself, ScottGu responds
to my email. Basically it all started last week at Techdays in
Toronto (pictures to follow, I promise).
Quite a few people asked me about MVC, and whether or not it will replace Web Forms.
My response was that it wouldn’t, but I didn’t have any tangible proof. I discussed
new features in .NET 4.0, and how the development is still going strong for future
releases. Some didn’t buy it.
So, earlier today I emailed Scott and asked him for proof. This was his response:
Hi Steve,
Web Forms is definitely not going away – we are making substantial improvements to
it with ASP.NET 4.0 (I’m doing a blog series on some of the improvements now).
ASP.NET MVC provides another option people can use for their UI layer – but it is
simply an option, not a replacement.
In terms of the dev team size, the number of people on the ASP.NET team working on
WebForms and MVC is actually about equal. All of the core infrastructure investments
(security, caching, config, deployment, etc) also apply equally to both.
Now, MVC is new. MVC is powerful. MVC is pretty freakin cool in what it
can do. But it won’t replace WebForms. Frankly, I like WebForms.
MVC does have it’s place though. I can see a lot benefits to using it.
It alleviates a lot of boilerplate code in certain development architectures, and
that is never a bad thing.
Long Live WebForms!
A few weeks ago, six or so, I was offered a position as a Software Developer for the Woodbine
Entertainment Group. The position looked appealing so I accepted the job
offer. I am in a probationary period for the next four months and a bit.
Anything I say can be grounds for firing me. Never liked that part about non-contract
jobs. Ah well.
Woodbine is an interesting company. I knew very little about it until I got
word of the job. Seems I was the only one in Canada who didn’t know the company.
My grandmother, who moved to California 50 years ago, knew about the company.
Even used to bet there – well, the Woodbine Race Track, before it moved. It
has an interesting history.
It is migrating to be a Microsoft shop, from a more Novell focused infrastructure.
We are working towards standardizing on .NET for our custom applications.
The one thing that caught my eye with Woodbine is that the company is the technology
leader for Horse Racing. Not just in Canada, but throughout the world.
Our services can let you place a
bet live, on a track in Australia, and see results immediately. Can you
imagine the infrastructure required for such a feat? It’s sweet! The business-people
behind this are really keen on letting technology do it’s thing, so we can make money.
Lots of money. See our Annual
Reports on that. Check back for latest numbers.
Now, some of you may have noticed that our Corporate Portal is written in what looks
to be Classic ASP. For all intensive purposes, it is. Archive.org shows
the portal went live in 2001, and had a major rebuild in 2003. Since then incremental
changes have taken place, most of which have been built using ASP.NET. We are
working on the new portal. All I can say at the moment is: it’s going to be
awesome. So awesome that a new word will need to created to contain all of its
awesomeness. HorsePlayer Interactive is
pretty amazing, but I’d like to think this new site will be just that much more awesomer.
Yes, I said awesomer.
As for the nature of this site, it won’t change. I’ll still post my thoughts
and experiences. I might need to change stories a little to protect the innocent,
but it’s all in good fun. I may be forced to post details of how horse racing
actually works, because I’m still not sure I get all the facets of it. In time.
More to follow.
In my previous post, I started
talking about using Microsoft technologies over PHP and open source technologies.
There were a couple reasons why I chose to make the move. First, from a development
perspective, everything was object oriented. PHP was just getting started with
OOP at the time, and it wasn’t all that friendly. Second, development time was
generally cut in at least half, because of the built in controls of ASP.NET.
Third, the end result was a more rich application experience for the same reason.
The final reason comes down to the data aspect.
Pulling data from a database in PHP wasn’t easy to do. The built in support
was for MySQL, with very little, if next to nothing for SQL Server. In a lot
of cases that isn’t always a bad thing. MySQL is free. You can’t argue
with that. however, MySQL wasn’t what you would call ACID compliant. Defined,
MySQL did not have the characteristics of being Atomic, Consistent, Isolated,
and Durable. Essentially, when data goes missing, there is nothing you can do
about it. SQL Server on the other hand is very ACID compliant. This is
something you want. Period.
Once .NET 2.0 was released, a whole new paradigm came into play for data in a web
application. It was easy to access! No more, or at least next to very
little boiler plate coding was necessary for data access now. Talk about a selling
point. Especially when the developer in question is 16 going on 17.
Now that I didn’t need to worry about data access code, I could start working on figuring
out SQL. At the time t-SQL scared the crap out of me. My brain just couldn’t
work around datasets. The idea of working with multiple pieces of data at once
was foreign. I understood single valued iterations. A for loop made sense
to me. SELECTs and JOINs confused me. Mind you, I didn’t start Statistics
in math until the following year. Did SQL help with statistics, or did statistics
help me finally figure out SQL? It’s a chicken and the egg paradox.
So here I am, 17 years old, understanding multiple languages, building dozens of applications,
and attending developer conferences all the while managing my education in High School.
Sweet. I have 3 years until the next release of Visual Studio comes out.
It was here that I figured I should probably start paying more attention in school.
It’s not so much that I wasn’t paying attention, it’s just that I didn’t care enough.
I put in just enough effort to skate through classes with a passing mark. It
was also at this point in time that I made an interesting supposition.
Experts tend to agree that people who are programming geniuses are also good at math
and critical thinking or reasoning. Not one or the other, but both. Now
I’m not saying I’m a programming genius, but I suck at math. It was just never
in the cards. But, according to all those High School exams and the psychological
profiling they gather from them, my Critical Thinking and Reasoning skills are excellent.
Top 10% in Canada according to the exam results. My math skills sit around top
20-30% depending on the type.
Neurologists place this type of thinking in the left hemisphere of the brain.
The left brain is associated with verbal, logical, and analytical thinking. It excels
in naming and categorizing things, symbolic abstraction, speech, reading, writing,
arithmetic. Those who live in the left brain are very linear. Perfect
for a software developer.
The supposition I made had more to do with the Pre-Frontal Cortex of the brain.
It does a lot of work, some of which is planning complex cognitive behaviors.
Behaviors like making a list, calculating numbers, abstracting thoughts, etc.
It plans out the processes our brains use to get things done. This is true for
both sides of the brain. So, suppose you are left brain-oriented. You
are predisposed to be good at development. Now, suppose your Pre-Frontal Cortex
is very well developed, more so than the average person. It could be reasoned
that part of being a programming genius is having a well developed Pre-Frontal Cortex.
So why does this make us want to program? Find out in Part
5.
In my second post I discussed
my run in with ASP, and how PHP was far better. I ended the post talking about
an invitation to a Microsoft event. This was an interesting event. Greg
and I were the only people under 30 there. When that’s a 15 year difference,
things get interesting. Especially when you need your mother to drive you there…
The talk was a comparison between Microsoft based technologies and Linux based technologies.
The presenter was a 10 year veteran of IBM, working on their Linux platform, who then
moved to Microsoft. For the life of me I can’t remember his name.
His goal was simple. Disprove myths around Linux costs versus Windows costs.
It was a very compelling argument. The event was based around the Windows
Compare campaign. It was around this time that Longhorn (Longhorn that turned
into Vista, not Server 2008) was in pre-beta soon to go beta, and after discussing
it with Greg, we decided to probe the presenter for information about Longhorn.
In a situation like that, the presenter either gets mad, or becomes really enthusiastic
about the question. He certainly didn’t get mad.
Throughout the rest of the talk, the presenter made some jokes at mine and Greg’s
expense, which was all in good fun. Based on that, we decided to go one step
further to ask how we can get the latest Longhorn build, at one of the breaks.
the conversation went something like this:
Me: So how do people get copies of the latest build for Longhorn?
Presenter: Currently those enrolled in the MSDN Licensing program can get
the builds.
Me: Ok, how does one join such a licensing program?
Presenter: Generally you buy them.
Me: How much?
Presenter: A couple thousand…
Me: Ok let me rephrase the question. How does a student, such as myself
and my friend Greg here, get a the latest build of Longhorn when we don’t have an
MSDN subscription, nor the money to buy said subscription?
Presenter: *Laughs* Oh. Go talk to Alec over there and tell him I said
to give you a student subscription.
Me: Really? Cool!
Six months later Greg and I some how got MSDN Premium Subscriptions. We had
legal copies of almost every single piece of Microsoft software ever commercially
produced. Visual Studio 2005 was still in beta, so I decided to try it out.
I was less than impressed with Visual Studio 2003, but really liked ASP.NET, so I
wanted to see what 2005 had in store. At the time PHP was still my main language,
but after the beta of 2005, I immediately switched to C#. I had known about
C# for a while, and understood the language fairly well. It was .NET 1.1 that
never took for me. That, and I didn’t have a legal copy of Visual Studio 2003
at the time.
Running a Longhorn beta build, with Visual Studio 2005 beta installed, I started playing
with ASP.NET 2.0, and built some pretty interesting sites. The first was a Wiki
type site, designed for medical knowledge (hey, it takes a lot to kill a passion of
mine). It never saw the light of day on the interweb, but it certainly was a
cool site. Following that were a bunch of test sites that I used to experiment
with the data controls.
It wasn’t until the release of SQL Server 2005 that I started getting interested in
data. Which I will discuss in the my next
post.
Some days you just have to shake your head and wonder. As it turns out, I'm a
little late to hear about this, but nonetheless, I'm still shaking my head.
It turns out that Windows has gone open source. And (!!) it's not being made by Microsoft
anymore. Well, Windows™ is still made by Microsoft. Windows is now made by a group
under the gise of ReactOS.
ReactOS® is a free, modern operating system
based on the design of Windows® XP/2003. Written completely from scratch, it aims
to follow the Windows® architecture designed by Microsoft from the hardware level
right through to the application level. This is not a Linux based system, and shares
none of the unix architecture.
So essentially, these people are taking
the Windows architecture (based on XP/2003), and redesigning it from scratch. Or rather,
are re-coding it from scratch, because redesigning would imply making something different.
Sounds vaguely familiar to, oh, something called Vista. Except uglier.
Now, that nagging feeling we are all getting right about now should be visualized
as a pack of rabid lawyers. Isn't this considered copyright infringement? They outright
define the product as a copy.
And what about the end users? Are all programs designed to run on Windows supposed
to be able to run on this ReactOS? Why bother with testing? The XP architecture is
now almost 8 years old by now. That means anything designed to run on Vista, or soon
to be designed to run on Windows 7, wouldn't stand a snowballs chance in hell, running
on ReactOS.
I would love to see how a .NET application runs on it.
Do you remember the
SubSonic
project? The Entity Framework is kind of like that. You can create an extensible
and customizable data model from any type of source. It takes the boiler plate coding
away from developing Data Access Layers.
Entity is designed to seperate how data is stored and how data is used. It's called
an Object-Relational Mapping framework. You point the framework at the source, tell
it what kind of business objects you want, and poof: you have an object model. Entity
is also designed to play nicely with LINQ. You can use it as a data source when querying
with LINQ. In my
previous post,
the query used NorthwindModEntities as a data source. It is an Entity object.
|
Courtesy of Wikipedia |
The Architecture, as defined in the picture:
-
Data source specific providers, which abstracts the ADO.NET interfaces to connect
to the database when programming against the conceptual schema.
-
Map provider, a database-specific provider that translates the Entity SQL command
tree into a query in the native SQL flavor of the database. It includes the Store
specific bridge, which is the component that is responsible for translating the
generic command tree into a store-specific command tree.
-
EDM parser and view mapping, which takes the SDL specification of the data
model and how it maps onto the underlying relational model and enables programming
against the conceptual model. From the relational schema, it creates views of the
data corresponding to the conceptual model. It aggregates information from multiple
tables in order to aggregate them into an entity, and splits an update to an entity
into multiple updates to whichever table contributed to that entity.
-
Query and update pipeline, processes queries, filters and update-requests to
convert them into canonical command trees which are then converted into store-specific
queries by the map provider.
-
Metadata services, which handle all metadata related to entities, relationships
and mappings.
-
Transactions, to integrate with transactional capabilities of the underlying
store. If the underlying store does not support transactions, support for it needs
to be implemented at this layer.
-
Conceptual layer API, the runtime that exposes the programming model for coding
against the conceptual schema. It follows the ADO.NET pattern of using Connection
objects to refer to the map provider, using Command objects to send the query, and
returning EntityResultSets or EntitySets containing the result.
-
Disconnected components, which locally caches datasets and entity sets for
using the ADO.NET Entity Framework in an occasionally connected environment.
-
Embedded database: ADO.NET Entity Framework includes a lightweight embedded
database for client-side caching and querying of relational data.
-
Design tools, such as Mapping Designer are also included with ADO.NET Entity
Framework which simplifies the job on mapping a conceptual schema to the relational
schema and specifying which properties of an entity type correspond to which table
in the database.
-
Programming layers, which exposes the EDM as programming constructs which can
be consumed by programming languages.
-
Object services, automatically generate code for CLR classes that expose the
same properties as an entity, thus enabling instantiation of entities as .NET objects.
-
Web services, which expose entities as web services.
-
High level services, such as reporting services which work on entities rather
than relational data.
In case you haven’t seen this, check out MVP Patrick Smacchia’s visualizations of the .NET Framework, including an NDepend analysis of the number of Types, a Treemap view, and a dependency matrix for the entire framework.
The dependency matrix is, IMO, impressively sparse. Also interesting but perhaps not surprising is that PresentationFramework represents the largest [...]