Functional/System testing with Visual Studio/Test Manager

If your reading this blog you likely understand what functional testing and you may use the term system testing.

Wikipedia defines these terms as:

System testing of software or hardware is testing conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements. System testing falls within the scope of black box testing, and as such, should require no knowledge of the inner design of the code or logic”

Functional testing is a type of black box testing that bases its test cases on the specifications of the software component under test. Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered.”

If you have read Agile Software Engineering with Visual Studio (by Sam Guckenheimer & Neno Loje) you will have heard about “reducing waste”. Identified as tasks that reduce waste are functional and system testing. These two tasks can be done during code development through unit tests reducing the cost of bug fixes, bug analysis, creating a suite of automated tests and automated regressions tests and reducing the number of people involved in testing and the bug. In some teams developers and testers have been testing the same thing one through unit tests and then again later by a testers. This is duplication of work effort that is a expensive waste.

In Visual Studio there are unit testing tools and third party add in tools that developers can use to create very robust unit tests. You maybe thinking “the developers do not test the same “stuff” that testers do”. Your right, I agree. However since the team is encouraged to make changes to reduce waste why don’t we testers help them. In Visual Studio we can create test cases that are associated to the requirement/user story work item that describes what needs tested. We can pair up with developers to help them write robust unit tests to cover all the testing including boundary, error and data testing.

There are people that believe the future of the “software testers” is about to make a big change. Testers will need to be able to write and execute unit tests themselves therefore requiring the basics of coding and the ability to add assertions (validation) to the unit tests. (Check out MSDN’s Verifying Code by Using Unit Tests topics. ) I believe this will be a reality in the future but I also believe it will evolve.  If you want to start now pair up with your developers to help them create unit tests that execute both the “happy path” and boundaries of an individual method, class or component. Help them to create system integration tests. Getting expose to how unit tests are designed and coded will help you move into the future. In addition having knowledge of what has been unit tested reduces test duplication later. TFS and Visual Studio help us with all this through work item traceability.

Example of work item traceability:

image

Example of a Test Case and Associated Automation:

imageimage

Visual Studio has the tools that will help us move into the future with confidence and the security we’ll need. Humans in general are not adapt to change but change we must. I am one of those people that embraces change and excels in change but then I have had Visual Studio in my pocket!

- Kent Beck

The role of professional testing will inevitably change from “adult supervision” to something more closely resembling an amplifier for the communication between those who generally have a feeling for what the system must do and those who will make it do.

Kent Beck author of Test-Driven Development (Addison Wesley 2002), 86.

Visual Studio – my companion, my mentor, my stability, my aid, my reporter, my success

Testa Smile 

(stay tuned, next blog  I will show you how easy it is to create a unit test!)

Microsoft Update – Multi-line steps in test case

An update is available that includes the following:

Support for multi-line test steps in Microsoft Test Manager

Changes to the 2010 client to allow it to work with a TFS 11 Server

KB2522890 – Team Explorer Crashes when opening build from TFS 2008

KB2552300 - Gated Check-ins fail with the “Preserve local Changes” option

KB2561827 – DiffMerge closes with unhandled exception when comparing two files.

Here is the update: http://go.microsoft.com/fwlink/?LinkID=212065&clcid=0x409 

Testa Smile

Microsoft Lab Manager improvements in TFS11

Microsoft is making improvements to Lab Manager in TFS11. Brain Harry explains in this blog the primary addition of “Standard Environments” that enables the use of VM Ware with Lab Manager. Brian’s blog goes through how it all works and more.

Brain Harry’s blog.

Testa Smile

MTM - Test Case State and Reason fields

If you are using Microsoft Test Manager you will notice that the work item Test Case does not contain the field Reason. This statement really isn’t true, there is a reason and it is being set for you but not displayed.

The field Reason indicates why a work item is in a particular state. By default the reason field is being set.  States Active and Ready only have one reason which get sets by default for you. The state Closed has three options however since it is not displayed by default reason is be set to “Obsolete”. The other reasons available for the closed state are “Deferred” and “Duplicate”.

There maybe other reasons that you are closing a Test Case or setting the state to ready. One example is Test Case reviews. For this scenario you may want to add a reason that indicates the Test Case is in the state of ready “For Review” then a second “Review Done.

We can customize the available reasons but that is for another blog. Below are instructions on how to add the field Reason to display in your work item Test Case.

How do I get the field Reason to display:

1. In the main toolbar of Visual Studio select Tools > Work Item Types > Open WIT from Server

image

2. Connect to Team Project Collection window opens where you select the collection then click the Connect button.

3. Select Work Item Type window opens where you select a Team Project and the work item Test Case. Click Ok

image 

4. The Work Item Type editor opens as displayed below and you can scroll through to see all the fields available. Reason’s reference name is System.Reason and it is a string data type.

image

5. Click on the Layout tab where we will add the field Reason to the Test Case.

6. You are looking at a preview of the form layout. Find Group > Column > Group-State > Column. Right click on the second Column and select New Control.

7. The window pane to the right displays. Find Field Name in the adjoining cell click the dropdown arrow and select System.Reason from the listing. Find Label in it’s adjoining cell enter the label you want to display on the Test Case, likely Reason.

Under Column you will see Control – Reason added to the bottom of the listing you can move it up under Control-&State if you want. Where it is in this view is where it will display in the work item.

image

8. Save your changes and refresh TFS depending on how often TFS is being updated the field Reason will display as below in your Test Case work item.

image

Testa Smile

SharePoint 2010 Performance Testing Tips and Tricks

On Wednesday, January 25th I will be doing a session on Performance Testing SharePoint 2010.  This free session is online with a 20 minute presentation and demo and then Q & A starting at 12:25pm. Join us if you are available and can attend. Contact Denise Faustino for information on how to log into the session.

Denise's contact info.

Email dfaustino@objectsharp.com

Toll Free: 1-877-So-Sharp

Local: 416-649-3690

Testa Smile

Microsoft Test Manager–action recording bulk edit

A MSDN Blogger, Gautam Goenka is giving away the code to enable you to do bulk action recording edits. Click here

Thanks Gautam

Testa Smile

Test Scribe for Test Manager–how to change template

Shai Raiten has posted a blog on how to change the Test Scribe template. Test Scribe is a tool for turning your Test Plan in to a document. The default template can be changed to include or exclude information that meets your needs.

Check out Shai’s blog for instructions – How to change the Test Scribe template.

Testa Smile

 

Microsoft Blogs – Canadian Solution Developer

If you are reading this blog you should check out bloggers, Susan Ibach and Jonathan Rozenblit. Microsoft developer evangelist who are the resident bloggers at Canadian Solution Developer. Susan and Jonathan are posting some very informative information on ALM, TFS, Visual Studio, testing, events coming, events happening now and in the past.

The most recent blog is about LinkedIn with some great tips on setting up a professional profile. What not to do and what is important to do. Click the link below to see what they have to say.

Microsoft Canadian Solution Developer blog

Testa Smile

Microsoft TFS, Test Manager & Visual Studio–Why use it?

At Tech Days in Montreal a gentleman and I were chatting about Test Manager. He made a statement that I thought was well worth sharing.

It goes something like this:

“With Test Manager we no longer have to try and figure out when a tester leaves the company what all the excel spreadsheets on their desktop are all about. We know where the test cases are and what they test. It is great.”

I for one have been an advocate of reducing the paperwork tester’s have been required to generate. I know there were times we spent more time documenting what we were going to do then actually doing it. Including testing!

The comment was well stated and I agree Microsoft has created a great tool.

Testa Smile

Microsoft Test Manager– Test Case export to excel tool

Free tool available for exporting your test cases to excel in a nicely formatted fashion.

Click to download here

You  connect to Team Foundation Server by clicking the Connect TFS button.

Pick the Team Project you want to work with as shown below.

image

The test plans and test suites associated to the team project you selected display.

Pick a test plan and test suite then specify where you want to save the excel file to on your system and a name for the excel file.

Click Export.

image

Here is an example of test cases export to excel. The Actual Results, Pass/Fail and Comments are not populated from Test Manager. These would be fields your tester’s would enter as they are testing! Or you could remove them and add your own columns using this for test case reviews or any other type of reporting required.

image

Testa Smile