Mobile Testing plugin to Visual Studio

A new Visual Studio plug in SeeTest for mobile testing. You can use Visual Studio (all versions) to test real physical phones including iOS, Android, Blackberry, Windows Mobile and Symbian. This includes Windows 8 phones. Just connect your device using a standard USB cable to your computer and record tests. SeeTest auto generates a script that you can copy & paste into Visual Studio. Then you can edit, run & view results in Visual Studio.

Check out Charles Sterling blog all about Visual Studio & SeeTest plugin. There is a video on Charles blog that you can watch.

Testa

Agile, Scrum webcasts to join

Agile.org has some create webcasts on subjects concerning Agile, Scrum. You can either sign up to them or go to the archive listing. Couple of interest are:

Agile Practices in a Traditional Organization

Adopting Test-First Development

Release Duration and Enterprise Agility

On May 15th is Agile and Quality: It is not an Oxymoron but a Necessity

Click here to check out the public webcast series.

Testa

Code Coverage for Manual Testing is coming in vNext …

Microsoft is storyboarding the process of code coverage for manual testing. They are also looking for you to complete a quick survey on the subject. The Visual Studio ALM team are asking for our help so let’s give it to them. Click below to do the survey.

Code Coverage for Manual Testing Survey

This is how tools get made that we will use, thanks for helping.

Testa

Microsoft VS TFS 11 Beta Power Tools are available

TFS11 beta power tools are now available for download.

In the power tool are:

• Best Practices Analyzer

• Team Explorer Enhancements
• Team Foundation Power Tool Command Line (tfpt.exe)
• Test Attachment Cleaner
• Process Template Editor
• Windows Shell Extensions

Read more about what all these power tools give you and download TFS11 Beta Power Tool.

Testa

Microsoft MSDN Forum Gadget

If you are interested in following any of the MSDN Forum’s there is a gadget you can download that makes access to your favourite threads quick and simple.

Download here.

Testa

ALM User Group in Toronto meets May 24th

It is Methodology May at the TALMUG

Methodology. In the world of software development there are not many words that raise contention quite as quickly as this. But why is that? What are the differences between Agile, Iterative, and Rigorous software development methodologies? There has been buzz about Scrum, XP, Lean, Waterfall, Kanban, and RUP for years; how do they fit into this discussion? But most importantly, why should you care? What does the test team think of all this?

In Methodology May the TALMUG brings you a panel of ALM professionals to discuss and debate these very questions and maybe help you see what methodology could work best at your company.

Pizza and Pop will be available at 5:30pm - Come out and join in on the discussions.

Being held at 40 University, Suite 1301, Toronto meeting starts at 6pm.

Follow on twitter @TOALMUG

Click here to Sign Up 

Testa

Microsoft seeing work item history made easier

Dave Lloyd has updated GetHistory a power tool for displaying work item history. Now you can export the work item history to excel. If your company has a audit process that has to be followed this tool may help you report on changes to requirements, test cases, bugs in just a few steps.

Click Get History to get the newest version. Keep your eye out on this tool you never know what features Dave may add!

Testa

ObjectSharp At the Movies … May 1st, 2012 9am to Noon

Here is what you will see and hear from ObjectSharp's Microsoft MVPs at this event:

Metro (Win 8 + Mobile)

The Metro interface has the potential to change how a lot of people use computers. You've already seen a little of it in the Windows Phone. More if you have downloaded the Windows 8 consumer preview. In this session, we will demonstrate how the power exposed by Metro (through live tiles, push notifications, immersive user experiences, etc.) can be harnessed to provide a usable and intuitive application.

VS 11/.NET 4.5

We're still a few months away from its release, but the beta version of Visual Studio 11 and .NET 4.5 is available and with a Go-Live license too. With Visual Studio, reduction and simplification are the keywords. For .NET, assistance with parallelism this session is the big topic. Between these and other topics, the tools that will make your development life easier are covered.

TFS 2012/Test Manager 2012

The word in the TFS/Test manager world for 2012 is "Agile". Lightweight requirements gathering through PowerPoint Storyboarding. Additional reporting, including a task board, in TFS and exploratory/agile testing through Test Manager. Lots of new features to delve into and they will be in this session.

Check out all about it here.

Testa

MTM Test Case iteration identifier –reduce waste

MTM needs a feature that allows testers to describe what an individual test case iteration is testing.

Right now I add a parameter to the last steps expected results "@Notes" and for each iteration of a test case I add to the Notes what I am testing in each iteration. Example would be testing a address site. My test case will go through all the address fields so any testing I would need can be done with parameters and iterations even boundary. But I need a easy way to identify what is being tested by an iteration.

Vote for this feature “Identifier that describes what each test case iteration is testing”

Testa

Create Reports for TFS2010 Test Results

Check out instructions on how to create reports for your TFS2010 Test Results. There is also an example that you can follow.

Testa

Unit Testing 101–check out this webcast on unit testing

Typemock is presenting a webcast “Introduction to Unit Testing” if your a tester this will be a good intro into unit testing. Pass this onto your developers even if they unit test.

Software testing isn’t just a task for QA. In order to prevent bugs and release quality code to market, you also need developer testing, including unit testing. Discover why you should start unit testing, and how you can get started with automated tests quickly.

Click below to register

Introduction to Unit Testing

Testa

Microsoft Visual Studio 2010 Test Vs. QTP

Karthik K.K has posted on LinkedIn (click to see) a chart that compares the features of Visual Studio & Test Manager to Quick Test Professional (QTP).

I like the last item:

VSTS can also be used by Business Analyst, Project Managers, and Stakeholders. It can assist teams being Agile or Scrum or Waterfall thru a process template. The process template can be customized to meet your company need. VSTS reports on all aspects of a project and can tell you at any time where in the project you are at, the quality of the project to date, the status of your requirements/user stories. You can have a “Requirement to Test Matrix” in seconds at anytime.

If this alone has got your attention and you want to know more contact me directly.

Testa

Windows 8 Camps coming…

What is a Windows 8 Camp?

A Windows 8 Camp is a free, two-day event. During the Camp, you’ll discover HANDS-ON how to build your first Metro Style App for Windows 8. You’ll learn from experts in a low-key, interactive way and apply what you’ve learned with support from Microsoft Canada.

How do I register? Click below for the city nearest you.

o Toronto – May 22^nd to June 1^st, Microsoft Downtown Toronto Office

o Montreal – June 4^th to June 8^th, Microsoft Downtown Montreal Office

o Vancouver – June 11^th to June 22^nd, Microsoft Downtown Vancouver Office

Where can I find Windows 8 resources?

· A Cornucopia of Windows 8 Goodness

· Windows 8 Consumer Preview

· Windows 8 Consumer Preview Product Guide for Developers

· Windows Dev Center

· Visual Studio 11 Beta

Testa

Agile vs. Traditional Approach to Unit Testing

Harry Baran from Thoughtworks posted a blog that I think is very much worth sharing. Harry outlines in his posting some of the difference between unit testing in Agile to the Traditional approach.

Unit testing in any process needs to be getting the attention it deserves, if done right it can be an early quality indicator for the solution under development. Test teams need to get involved in unit testing to some capacity no matter what process/methodology being followed. Pair up your developers and testers, offer training to your testers in unit testing and coding, hire testers with coding skills, let your testers see and understand what is being unit tested.

To quote Harry Baran: “Unit tests are foundations of an agile project. They enable fast feedback, continuous testing, continuous integration and refactoring.”

As children we were taught to share, for some reason one of the most important aspects of developing an application has not been shared. Time to change that, isn’t it? 

Unit Testing: Agile vs. Traditional Approach

Great write up Hari, thanks,

Testa

Toronto VS ALM User Group kick off meeting April 12th

The first Toronto VS ALM User Group kick off meeting is being held on April 12th @6:30pm sign up at TALMUG to get the details and register.

The User Group is for all roles within the Application Life Management team. Topics presented will vary from generic ALM practices to ALM with Visual Studio. If your involved in product management, a stakeholder, a business analysis or product owner, a developer or a tester this User Group is for you.

The goal of the first meeting will be to:

• Define the group’s mission
• Select an appropriate name
• Document the roles of the executive
• Recruit volunteers
• Funding
• Discuss how to reach out for sponsors
• Discuss meeting locations
• Discuss ideas for the first few meeting topics

Come out on April 12th and help us to kick-off this new user group code named TALMUG.

Testa

Looking for tester who can create Unit Tests – What?

In the blog Functional/System testing with Visual Studio/Test Manager we talked about unit testing. Testers in the near future are going to need unit testing skills. Now that is a very scary statement given 70% of testing is done manually. Now is the time to start learning by taking programming courses, unit testing courses, pairing with developers to learn and searching the web.

In the Visual Studio magazine Jeff Levinson article Take Unit Testing to the Next Level talks about how to add unit tests to requirements to show test coverage. Jeff supplies code you can download and instructions on creating simple unit tests then how to link them to requirements. If you have Visual Studio give it a try.

Thanks Jeff

 Testa  

Agile book pre order deal – Software in 30 Days

New Agile book by Ken Schwaber and Jeff Sutherland can be pre-ordered at Amazon for a great discounted price.

About the book from Amazon:

A radical approach to getting IT projects done faster and cheaper than anyone thinks possible

Software in 30 Days summarizes the Agile and Scrum software development method, which allows creation of game-changing software, in just 30 days. Projects that use it are three times more successful than those that don't. Software in 30 Days is for the business manager, the entrepreneur, the product development manager, or IT manager who wants to develop software better and faster than they now believe possible. Learn how this unorthodox process works, how to get started, and how to succeed. Control risk, manage projects, and have your people succeed with simple but profound shifts in the thinking.

The authors explain powerful concepts such as the art of the possible, bottom-up intelligence, and why it's good to fail early—all with no risk greater than thirty days.

I highly recommend reading this book.

Testa

Requirements Traceability in Visual Studio

Team Foundation Server (TFS) and Visual Studio(VS) excels when it comes to requirements traceability. Depending on what process you are using in TFS your stakeholders needs are documented in a work item called one of the following: requirements, user stories, use cases or backlog items. For this blog I am using the term requirement.

What is requirements traceability?

Continuous knowledge of the life of a requirement from conception to creation to design to development to verification to implementation and change. The ability to trace a requirements state and/or status at any time during a project.

First step in making requirements traceable.

In Visual Studio other work items are used to identify work that needs to be done to a requirement before it can be deemed done. There are different work item types depending on the process template you are using, however all contain the following: Task, Test Case, Shared Step, Bug. Additional work items can be added if not in your process template like Review, Issue, Change Request to name just a few. All these work items can be linked to the requirement they are helping to fulfill. Visual Studio 2010 introduced the concept of hierarchy by which a work item is linked to another work item using a linked type. The work item task is a child of a requirement, two requirements can be related, a test case tests a requirement. Work items like tasks may be linked to show predecessor or successor of another task. To learn more about choosing link types to effectively track your project click here MSDN Library.

Linked type listing:

Example of a requirement with linked work items:

What can be traced?

In Visual Studio during diagram modeling requirements and other work item types can be attached to objects in the model. During modeling you can either create or link requirements. This allows you to trace work items like requirements from a model diagram.

Example of a Use Case model with requirements linked:

Within the requirement work item I can see all other linked work items and information about them like state, assigned to. Knowing the state of work items linked to a requirement tells me the state and status of the actual requirement itself.

Development when checking in code to VS source control can be forced to associate what requirement or other work item their code relates too. Code check-ins create a Change Set that contains information about the check in and gets linked to the work item(s) associated during a check-in. When fixing bugs this is a nice feature.

In Visual Studio I can create queries to show any type of traceability report. Examples: Requirements to Tests Matrix, Requirements to Tasks, Requirements to Issues, Test Cases with Bugs to name just a few. Queries can also be exported or opened in excel, sent to someone thru email or opened in Microsoft Project. Queries exported to excel allows for adding additional excel reporting features like a Pivot Charts.

Requirements to Test Matrix – is there test cases for each requirement

Visual Studio comes with many reports that trace Requirements showing linked tasks and remaining work left, test case results state and many more scenario’s. Remember depending on what process template you are working with will determine the reports you see.

Example:

Checkout MSDN library for lots more information on Requirements Traceability in TFS & VS.

Testa

Functional/System testing with Visual Studio/Test Manager

If your reading this blog you likely understand what functional testing and you may use the term system testing.

Wikipedia defines these terms as:

“System testing of software or hardware is testing conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements. System testing falls within the scope of black box testing, and as such, should require no knowledge of the inner design of the code or logic”

“Functional testing is a type of black box testing that bases its test cases on the specifications of the software component under test. Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered.”

If you have read Agile Software Engineering with Visual Studio (by Sam Guckenheimer & Neno Loje) you will have heard about “reducing waste”. Identified as tasks that reduce waste are functional and system testing. These two tasks can be done during code development through unit tests reducing the cost of bug fixes, bug analysis, creating a suite of automated tests and automated regressions tests and reducing the number of people involved in testing and the bug. In some teams developers and testers have been testing the same thing one through unit tests and then again later by a testers. This is duplication of work effort that is a expensive waste.

In Visual Studio there are unit testing tools and third party add in tools that developers can use to create very robust unit tests. You maybe thinking “the developers do not test the same “stuff” that testers do”. Your right, I agree. However since the team is encouraged to make changes to reduce waste why don’t we testers help them. In Visual Studio we can create test cases that are associated to the requirement/user story work item that describes what needs tested. We can pair up with developers to help them write robust unit tests to cover all the testing including boundary, error and data testing.

There are people that believe the future of the “software testers” is about to make a big change. Testers will need to be able to write and execute unit tests themselves therefore requiring the basics of coding and the ability to add assertions (validation) to the unit tests. (Check out MSDN’s Verifying Code by Using Unit Tests topics. ) I believe this will be a reality in the future but I also believe it will evolve.  If you want to start now pair up with your developers to help them create unit tests that execute both the “happy path” and boundaries of an individual method, class or component. Help them to create system integration tests. Getting expose to how unit tests are designed and coded will help you move into the future. In addition having knowledge of what has been unit tested reduces test duplication later. TFS and Visual Studio help us with all this through work item traceability.

Example of work item traceability:

Example of a Test Case and Associated Automation:

Visual Studio has the tools that will help us move into the future with confidence and the security we’ll need. Humans in general are not adapt to change but change we must. I am one of those people that embraces change and excels in change but then I have had Visual Studio in my pocket!

- Kent Beck

The role of professional testing will inevitably change from “adult supervision” to something more closely resembling an amplifier for the communication between those who generally have a feeling for what the system must do and those who will make it do.

Kent Beck author of Test-Driven Development (Addison Wesley 2002), 86.

Visual Studio – my companion, my mentor, my stability, my aid, my reporter, my success

Testa  

(stay tuned, next blog  I will show you how easy it is to create a unit test!)

Microsoft Update – Multi-line steps in test case

An update is available that includes the following:

Support for multi-line test steps in Microsoft Test Manager

Changes to the 2010 client to allow it to work with a TFS 11 Server

KB2522890 – Team Explorer Crashes when opening build from TFS 2008

KB2552300 - Gated Check-ins fail with the “Preserve local Changes” option

KB2561827 – DiffMerge closes with unhandled exception when comparing two files.

Here is the update: http://go.microsoft.com/fwlink/?LinkID=212065&clcid=0x409 

Testa

Microsoft Lab Manager improvements in TFS11

Microsoft is making improvements to Lab Manager in TFS11. Brain Harry explains in this blog the primary addition of “Standard Environments” that enables the use of VM Ware with Lab Manager. Brian’s blog goes through how it all works and more.

Brain Harry’s blog.

Testa

MTM - Test Case State and Reason fields

If you are using Microsoft Test Manager you will notice that the work item Test Case does not contain the field Reason. This statement really isn’t true, there is a reason and it is being set for you but not displayed.

The field Reason indicates why a work item is in a particular state. By default the reason field is being set.  States Active and Ready only have one reason which get sets by default for you. The state Closed has three options however since it is not displayed by default reason is be set to “Obsolete”. The other reasons available for the closed state are “Deferred” and “Duplicate”.

There maybe other reasons that you are closing a Test Case or setting the state to ready. One example is Test Case reviews. For this scenario you may want to add a reason that indicates the Test Case is in the state of ready “For Review” then a second “Review Done.

We can customize the available reasons but that is for another blog. Below are instructions on how to add the field Reason to display in your work item Test Case.

How do I get the field Reason to display:

1. In the main toolbar of Visual Studio select Tools > Work Item Types > Open WIT from Server

2. Connect to Team Project Collection window opens where you select the collection then click the Connect button.

3. Select Work Item Type window opens where you select a Team Project and the work item Test Case. Click Ok

4. The Work Item Type editor opens as displayed below and you can scroll through to see all the fields available. Reason’s reference name is System.Reason and it is a string data type.

5. Click on the Layout tab where we will add the field Reason to the Test Case.

6. You are looking at a preview of the form layout. Find Group > Column > Group-State > Column. Right click on the second Column and select New Control.

7. The window pane to the right displays. Find Field Name in the adjoining cell click the dropdown arrow and select System.Reason from the listing. Find Label in it’s adjoining cell enter the label you want to display on the Test Case, likely Reason.

Under Column you will see Control – Reason added to the bottom of the listing you can move it up under Control-&State if you want. Where it is in this view is where it will display in the work item.

8. Save your changes and refresh TFS depending on how often TFS is being updated the field Reason will display as below in your Test Case work item.

Testa

SharePoint 2010 Performance Testing Tips and Tricks

On Wednesday, January 25th I will be doing a session on Performance Testing SharePoint 2010.  This free session is online with a 20 minute presentation and demo and then Q & A starting at 12:25pm. Join us if you are available and can attend. Contact Denise Faustino for information on how to log into the session.

Denise's contact info.

Email dfaustino@objectsharp.com

Toll Free: 1-877-So-Sharp

Local: 416-649-3690

Testa

Microsoft Test Manager–action recording bulk edit

A MSDN Blogger, Gautam Goenka is giving away the code to enable you to do bulk action recording edits. Click here

Thanks Gautam

Testa

Test Scribe for Test Manager–how to change template

Shai Raiten has posted a blog on how to change the Test Scribe template. Test Scribe is a tool for turning your Test Plan in to a document. The default template can be changed to include or exclude information that meets your needs.

Check out Shai’s blog for instructions – How to change the Test Scribe template.

Testa

Microsoft Blogs – Canadian Solution Developer

If you are reading this blog you should check out bloggers, Susan Ibach and Jonathan Rozenblit. Microsoft developer evangelist who are the resident bloggers at Canadian Solution Developer. Susan and Jonathan are posting some very informative information on ALM, TFS, Visual Studio, testing, events coming, events happening now and in the past.

The most recent blog is about LinkedIn with some great tips on setting up a professional profile. What not to do and what is important to do. Click the link below to see what they have to say.

Microsoft Canadian Solution Developer blog

Testa

Microsoft TFS, Test Manager & Visual Studio–Why use it?

At Tech Days in Montreal a gentleman and I were chatting about Test Manager. He made a statement that I thought was well worth sharing.

It goes something like this:

“With Test Manager we no longer have to try and figure out when a tester leaves the company what all the excel spreadsheets on their desktop are all about. We know where the test cases are and what they test. It is great.”

I for one have been an advocate of reducing the paperwork tester’s have been required to generate. I know there were times we spent more time documenting what we were going to do then actually doing it. Including testing!

The comment was well stated and I agree Microsoft has created a great tool.

Testa

Microsoft Test Manager– Test Case export to excel tool

Free tool available for exporting your test cases to excel in a nicely formatted fashion.

Click to download here

You  connect to Team Foundation Server by clicking the Connect TFS button.

Pick the Team Project you want to work with as shown below.

The test plans and test suites associated to the team project you selected display.

Pick a test plan and test suite then specify where you want to save the excel file to on your system and a name for the excel file.

Click Export.

Here is an example of test cases export to excel. The Actual Results, Pass/Fail and Comments are not populated from Test Manager. These would be fields your tester’s would enter as they are testing! Or you could remove them and add your own columns using this for test case reviews or any other type of reporting required.

Testa

Microsoft Impact award given to ObjectSharp

It was a great evening for ObjectSharp Consulting last night. At the Microsoft Impact awards ObjectSharp partners Dave Lloyd, Barry Gervin and Mike Green accepted the

 Partner of the Year award for Application Lifecycle Management.

Congratulations to Objectsharp and the Sharpe's.

Testa

Test Case customization by Dave Lloyd

The test case work item has a tab that lists the requirements/user stories that the test case tests. However, bugs related to the test case are not part of the listing. Dave has come up with a customization that changes the test case tab to include bugs created against the test case. With this customization you will see all work items that the test case tests.

Read Dave’s blog for more details and how to do the customization.

Testa

Microsoft Coded UI– test extension for 3rd party controls

Shubhra Maji has posted two blogs on Coded UI test extension for 3rd party controls. Below are links to both blogs that are very informative and a must to read.

The basics

Windows Forms Controls – How to?

Testa

Microsoft TFS and QTP integration

If you do not have a profile on LinkedIn it is time to join. There are amazing groups that are sharing information, posting questions & getting answers for a lot of different topics.

On the Visual Studio ALM User Group Anna shared a blog she posted  “How to integration TFS and QTP”

Other LinkedIn groups related to TFS are:

• Visual Studi0 2010 Testing
• Microsoft Visual Studio ALM + Team Foundation Server (Team System)
• Microsoft TFS/VST Customization Experts
• Microsoft Testing Visual Studio 2010
• Microsoft Coded UI

There are also Agile groups:

• Agile Testing
• Agile Toronto

This in no way is a compete list.

Testa

Exploratory Testing Blog by Anutthara

Anutthara is the Program Manager in the Visual Studio ALM Test Tools group at Microsoft. She is doing a series blog on Exploratory Testing. Anu's team does exploratory testing on all their projects and is well versed on the topic. Follow Anu's Exploratory Testing series by clicking here.

Testa

Ladies learning code ….

It is amazing what a twit can do. Back in June 2011 Heather Payne twitted about a group of ladies in California that get together to learn how to code. The response was unbelievable. The first meeting to just talk about the idea rallied 85 people. The first session in Sept was sold out. The Ladies are learning how to code from scratch, no experience necessary by volunteers will to spend time training.

Did you know there is a 12% to 88% mix of women to men in the IT industry?

Ladies Learning Code website

Testa

Microsoft Test Manager– test plan confirgurations

If your looking for a way to create test configurations that can be used in all your team projects check out Dave Lloyds blog. Dave explains how to edit the process template to add or change test configurations.

Testa  :-)

Agile with Microsoft Visual Studio

Sam Guckenheimer is the Product Owner of Microsoft Visual Studio product line. He has 30 years' experience as architect, developer, tester, product manager, project manager and general manager in the software industry worldwide.

Sam’s new book “Agile Software Engineering with Visual Studio: From Concept to Continuous Feedback” is a must for all to read. Available in paperback and kindle from Amazon.

Sam recently did a webcast for TesTrek 2011 in Toronto which is also well worth watching. Sam talks about Getting Agile with Testing.

“Testing is central to the success or failure of any organization adopting the Agile Consensus. A modern testing approach can be a great enabler of value flow, transparency and the reduction of waste. An outmoded testing approach can be a huge impediment and source of unending conflict. As testers, it's important that we be on the right side of the consensus and be the enablers. The alternative is untenable. In this session, we will explore fundamental principles of agile project delivery and discuss the impact on the QA lifecycle. “

If your in Toronto the week of November 7th sign up for TesTrek 2011 Conference where Sam is the keynote speaker. Dave Lloyd and I will be doing a hands on workshop on Exploratory Testing. There is lots happening this year at TesTrek check it out.

Testa

Microsoft Test Manager - test case work item customizations

Check out this MSDN blog about a company that is using Test Manager for their automated testing. They have customized the Test Case work item to capture information about what test suite the test belongs too, added test categories like Smoke Test, and test execution statistics. They have added a command line program that updates the TFS TCM periodically updating the information in these new fields.

At the end of the blog is TfsTestcaseUpdaterSample that you can download.

Share you idea’s by adding comments on other ways to customize the Test Case work item to help gather information that can be queried.

I will post any idea’s so stay tuned. 

Testa

Microsoft vNext preview has been released as of yesterday

Microsoft vNext was released yesterday at BUILD and is available on MSDN if you have a subscription. The released name is Visual Studio 11. There is lots of new stuff for the developer’s but of interest to me and likely anyone following my blog is the Agile features shown in the diagram below. The Exploratory Testing tool is very cool as is the Feedback tool. What am I saying it is all cool.

Check out more information on each of these items here.

Visual Studio 11

Stay tuned I will do my own review of these tools for you in the days to come.

Testa

The 10 Minute Test Plan by Jim Whittaker

Jim Whittaker recently wrote an article called The 10 Minute Test Plan. It is a must to read. If you’ve been in testing where you had to write a Test Plan you will be grinning as you read and nodding you head.

To get your interest here is the first 2 paragraphs of Jim’s article:

Anything in software development that takes ten minutes or less to perform is either trivial or is not worth doing in the first place. If you take this rule of thumb at face value, where do you place test planning? Certainly it takes more than 10 minutes. In my capacity as Test Director at Google I presided over teams that wrote a large number of test plans and every time I asked how long one would take I was told “tomorrow” or “the end of the week” and a few times, early in the day, I was promised one “by the end of the day.” So I’ll establish the task of test planning to be of the hours-to-days duration.

As to whether it is worth doing, well, that is another story entirely. Every time I look at any of the dozens of test plans my teams have written, I see dead test plans. Plans written, reviewed, referred to a few times and then cast aside as the project moves in directions not documented in the plan. This begs the question: if a plan isn’t worth bothering to update, is it worth creating in the first place?'

The Ten Minute Test Plan

The nice thing about using Microsoft Test Manager is my test plan is built in and I can update it as things change. I can use Test Scribe to output it to a word document is anyone does want to really read it.

Testa

Microsoft Test Manager–control recognition, plus supported configurations & platforms.

Some controls are not recognized without added help. You need to make sure that the controls being added to the solution by development are first compatible and second have be referenced so that they are recognizable by MTM.

Silverlight Control recognition:

How to setup up your Silverlight Application for Testing instructions.

Set a Unique Automation Property for Silverlight Controls for Testing

Testing WPF application check this out.

Supported configurations and platforms for Action Recordings and Coded UI Tests check this out.

Testa

Microsoft Test Manager–enable event log

If you are having issues with Test Manager make sure you have downloaded the hot fixes and feature pack in the order below:

Hot Fix for MTM :

1. 1. KB2387011
2. 2.  KB2403277
3. 3. Feature Pack 2 (MSDN download)

4. Expression Encoder with SP1 (video recording)

If you are still experiencing issues enable tracing to isolate any issues:

Check out this site for enabling tracing.

Testa

Microsoft Test Manager (MTM)– binding parameters

2. Those of you who like to use your mouse be aware when binding parameters to input fields. After getting your parameter value you cannot go to the input field and right click then select the Paste option.
4. Reason: the action recording recognizes and records this action as a mouse click action.

You have to use CTRL+V or you can just type the value into the input field.

After entering the value (either way above) you have to remove focus from that field. I find tabbing works 95% of the time or clicking in a white space.

If your parameter does not get marked as bound try again. Remember, parameters only work with input fields. They will not work on controls like radio buttons at this time.

Testa  

Microsoft Test Manager(MTM) – multiple instances on desktop

At this time you cannot have more then one instance of Test Manager open on a desk top with out doing the following:

Made another copy of the MTM.exe and MTM.exe.config  files found on your C:\ drive under Microsoft Visual Studio 10.0\Common7\IDE

Then you have two instances of Test Manager for opening, this comes in handy when you need to look at different area’s of MTM at the same time.

Testa

Microsoft Test Manager and Infragistic Controls

If your developers are using Infragistic controls and your using Test Manager you will not be able to create action recording. Jeff Levinson advised me that:

“Playback won’t work with the infragistics controls (unless it is a web app because MTM actually works against the DOM in a web app and not against the controls directly). The Infragistics controls won’t be recognized if you try to add them to a UIMap (again – unless this is a web app) because they don’t support the interfaces to allow MTM to interrogate the control.”

You can however play around with the Coded UI tests. A client I recently worked with was able to create CUI test recordings and play them back where Infragistic controls existed. Note this was not using action recordings to create.

If your development team is using Infragistic controls please contact Infragistics requesting they add the interfaces to allow MTM to interrogate their controls. Telerik at this time is the only company that has added support for their controls and MTM.

Testa

Microsoft Test Manager (MTM) – Test Case steps

I am seeing people talking about having a test case with up to 50 and even 100 steps. I might be mad but isn’t that a lot of steps in one test case? When I talk to test teams my recommendation is no more then 10 steps per test case.

Make usage of Shared Steps that can be used to navigate through an application. Make usage of other test cases that will get your test case to the right spot in the application. Use the MTM ordered test feature to organize test cases to run in a specific sequence when needed.

I like to tell testers about the KISS principle of design.

Keep it simple and straightforward.

Keep it simple and short.

The KISS principle states that simplicity should be a key goal in design, and that unnecessary complexity should be avoided. This applies to code design as well.

Test Cases you’ve kissed will be easy to understand, maintain and report on.

Testa 

Microsoft Test Manage MTM– test run pausing and resuming

Questions about the ability to pause a test case during execution then restart it has come up in discussions a few times. Thought I would share that it can be done and how.

In the Test Runner you click the Pause button…

In the Test Runner next click the Return to the Testing Center icon…

Once back in the Testing Center in the window header will be a new icon that when clicked returns you to the Test Runner ….

Clicking the Return to Test Runner icon opens Test Runner with a Resume button that when clicked starts your test execution where you left off.

Note:

Closing Test Manager  will close the paused test run and set it to Failed.

During a Test Run pause, if you start another Test Case Run the message below is posted. 

Save and/or Don’t Save ends the paused test run setting it to Failed. (original test run stops)

Cancel stops the newly select test case run and returns you to the Test Center with the icon displays to resume the original test.

If there are things about Test Manager you would like to see changed add a backlog item or vote on the one’s already added. Microsoft is looking for our input on what changes people want and by voting on them how important it is to you.

A backlog item already exists for allow you to pause one test run and start another then return to the paused run. If this is important to you click below find this backlog item and vote.

Visual Studio UserVoice

Testa

Microsoft - Search Work Items for TFS 2010

Check out the add-in tool that allows you to do searches for work items and it can be downloaded free. If your still using VS2008 or VS2005 there is a version of this tool available for them too.

Search Work Items for TFS2010 

Examples of work item searches with this tool:

Blog on how to use click here.

Enjoy Testa

The Testing Planet– The Evil Tester Question Time

Just released the July edition of The Testing Planet.

Available for your Kindle

Testa … enjoy reading about testing by testers.

TesTrek 2011 in Toronto – Nov. 7th to 10th

Sign up for an Object Sharp workshop on Exploratory Testing at TesTrek 2011 in Toronto, Ontario on November 11th at 1:30pm. Myself and Dave Lloyd will be presenting plus giving you hands on experience doing exploratory testing. See how to overcome the challenges of tracking the steps you took, reporting bugs and retesting exploratory bugs.

Experience and Overcome the Challenges of Exploratory Testing

In 1983, the term Exploratory Testing was introduced. Prior to that, we called it ad-hoc testing. Exploratory testing is said to be a mindset, a way of thinking, freestyle testing that liberates the tester to explore. Testers, through experience, know their applications inside out; they are the users in reality. Freeing testers to explore and use the application has been proven to identify more bugs then traditional scripted testing. However, there are obstacles and challenges related to this freestyle testing. Join this workshop to explore those obstacles hands on and learn how we can overcome them. Explore an application with known bugs and see if you can find them.

• Experience exploratory testing using the Microsoft Test Manager Tool.
• Do your own exploratory testing on a Virtual Machine in the cloud.
• Discover the difficulties of exploratory testing, how to track the steps, report the bug, and perform retesting.
• Participate in idea generation for overcoming the issues of exploratory testing.

Sign up for the first TesTrek hands-on workshop.

Testa

Agile–not just for software development

Very interesting article on turning the school system in the US to being Agile. The article talks about turning the 12 principles of Agile Software into “Schoolware”. Interesting concept Canada should look at this too.

Agile School Education

From the article:

The Twelve Principles of Agile Schools

We follow these principles:

1. Our highest priority is to satisfy the needs of children and their families through early and continuous delivery of meaningful learning.

2. Welcome changing requirements, even late in a learning cycle. Harness change for the benefit of children and their families.

3. Deliver meaningful learning frequently, from a couple of days to a couple of weeks, with a preference to the shorter timescale.

4. School and family team members work together daily to create learning opportunities for all participants.

5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

6. The most efficient and effective method of conveying information to and within a team is face-to-face conversation.

7. Meaningful learning is the primary measure of progress.

8. Our processes promote sustainability. Educators, students, and families should be able to maintain a constant pace indefinitely.

9. Continuous attention to technical excellence and good design enhances adaptability.

10. Simplicity-the art of maximizing the amount of work not done-is essential.

11. The best ideas and initiatives emerge from self-organizing teams.

12. At regular intervals, teams reflect on how to become more effective, then tune and adjust their behaviour accordingly.

Where else can we be Agile?

Testa

Agile– retrospectives

Retrospectives after a sprint can be fun, challenging and at times trying. It is not a finger pointing session but a chance to reflect on what was good or bad about the last sprint. All team members should participate if they aren’t this would be the first issue to address. One way to get participation is by having one member start by sharing a good experience.The group then takes turn discussing this experience. Often other good and bad experiences come from that discussion. During the discussion a ball can be used to toss to the person that is talking. The ball becomes the signal that a member is talking and should not be interrupted. When the member is done they toss the ball to another member. This keeps the session moving, gives everyone a chance to talk and it can be fun. All good and bad items should be put on a white board. Each bad experience should have a plan to resolve. This is not to say that every bad experience can be resolved in the next Sprint, it may take a few Sprints. One bad experience teams encounter is getting the business involved and/or getting the business to address User Story questions in a timely manner. If you read Lisa Crispin’s article “Experimenting” she gives a great example of how her team addressed a bad experience using faces to rate User Stories. A happy face given to User Stories that were documented well and the team was able to assign to a Sprint. A sad face was given to User Stores that were badly documented and/or waiting on the business for information. The User Story face rating was shared with the business and it was known going into a Sprint the business information needed and that if it was not obtained the Story would not be done. Read the article it is very interesting.

I think we should add to the Product Back Log work item a field that can be used for face rating. It can be done! 

Testa, ALM MVP

Agile, Manage & Test–Stickyminds launches TechWell for the test community.

Sticky minds.com has launched a new site called TechWell for the Agile, Manage and Test communities. You can interact with others through blogs, forums and groups. Access videos, podcasts, articles, and more all pertaining to testing.

With todays changes in the Software Industry happening quickly testers are facing challenges in the skills they need. The future is Agile with a Scrum methodology and framework. Testers need to be ready and it is going to  happen quickly, in fact it has started already.

I recommend checking out TechWell and getting the scoop from your peers in the community. Click on TechWell to check out.

Testa

Basics of Securing Applications – Steve Syfuhs talks about security

Objectsharp’s Steve Syfuhs the Canadian Developer Security MVP is being interviewed by Jonathan Rozenblit a Microsoft evangelist on the basics of securing applications.

Click here to follow.

Part 1: Development Security Basics (This Post)

Part 2: Vulnerability Deep Dive (Coming Thursday, June 2, 2011)

Part 3: Secure Design and Analysis in Visual Studio 2010 (Coming Thursday, June 9, 2011)

Part 4: Architectural Considerations for Developing Secure Applications (Coming Thursday, June 16, 2011)

Part 5: Incident Response Management with Team Foundation Server (Coming Thursday, June 23, 2011)

Testa

Microsoft ALM – the next release code named vNext

At TechEd 2011 in Atlanta Microsoft showcased some of the new features coming in the next release code named  vNext. Check it out by clicking the different links below:

VNext –  click the link to the vNext demo.

New Features are:

• Storyboarding– A plug-in for PowerPoint that connects the creation and review of story boards with the rest of the team.
• Backlog & Sprint Planning– New web based product backlog, sprint planning and task board.
• Client Feedback –New tooling support to invite and receive feedback from stakeholders during development.
• Team Navigator– More time ‘in the zone’, through improved experiences for day-to-day tasks.
• Continuous Testing – A new unit test runner continuously running unit tests in the background
• Agile Quality Assurance – Increased code quality with code review support, enhanced unit testing frameworks and new exploratory testing support.
• Aligning Development with Operations (Intellitrace in Production) – Increased connections and insight between the operations and development teams lowering the time it takes to fix a bug in production.
• SCOM & TFS Integration – Software Centre Operations Management now integrated with TFS.

Channel 9 - The Future of Microsoft Visual Studio Application Lifecycle Management Cameron Skinner talk at TechEd 2011 

Whitepaper: PDF whitepaper which reinforces the value propositions for what we’re delivering in vNext.

Testa

Microsoft Test Manager & TFS–copy test cases from one Team Foundation Server to another

Shai has done it again. In my last blog I explained a tool that lets you copy test cases & shared steps from one Team Project to another. We had a need to copy test cases from one team project to another where the project resided on different Team Foundation servers.

Shai in no time revised the tool to enable copy of test cases between projects residing in different TF servers.

If you have a need for this tool email me.

Great tool – thanks again Shai

Testa

Microsoft Test Manager & TFS – copy test cases from one team project to another team project.

In Test Manager you can copy existing Test Suites into Test Plans but only when they are both in the same Team Project. You can also create a copy of a test case adding it to a different team project but only one at a time in Visual Studio or MTM. So what happens if I have test cases that are in other Team Projects that I want to add to a new Team Project?

Shai Raiten who is a VS ALM MVP & Microsoft Regional Specialist has created a tool for this function.

The tool is a wizard that walks you through the:

If your source test case has:

If you include the coping of links from your source to your target and the link is another work item from your source team project it does not get copied into the target team project.

If you would like to use this tool email me.

Shai great tool and thanks for providing me a copy.

Testa

The Testing Planet – “Read all about it!”

The Testing Planet is available on your Kindle.

Think of me as the kid standing on the corner back in the 50’s with a newspaper held high yelling, “Read all about it”. I have downloaded my copy to my Kindle it cost 4.99US which these days is 4.99 Canadian. You can still download the PDF format or have home delivery should you not have a electronic reader yet. I’ll be reading from my Kindle on the train, on my deck (with a cold beer), anywhere I want.

Congratulations to The Software Testing Club, Rob Lambert and Rosie Sherry.

I am so excited, I absolutely love my Kindle for so many reasons. 

Did you know if the book your reading has text to speech you can plug your kindle into your car (need a USB plug) and listen while you drive. My brother-in-law and sister-in-law read a book on a recent road trip  with their kindle. Next road trip we’ll be reading.

The March 2011 Issue links: www.thetestingplanet.com or Amazon

Testa

Migration to Microsoft TFS 2010 from Quality Center with Scrat

Quality Center has been the popular kid on the block over the years. Since Microsoft Test Manager came out in April 2010 it has and is giving QC a run for it’s money. Many companies are wanting to convert over to Microsoft TFS 2010 from Quality Center but are not sure how and do not want to lose existing artefacts.

SELA a Microsoft Gold Partner have created Scrat that will let you migrate full HP Quality Center projects into TFS2010 in days. Requirements, Bugs, Test Cases, Attachments and Links between items plus their interrelationship links can all be migrated to TFS 2010. Scrat is completely configurable to migrate what you want and need.

I personally have met a number of test teams that are converting to TFS2010 here are some comments being made about TFS2010 and Test Manger:

• the collaboration between teams is outstanding
• there is one place that stores all the project artefacts and anyone on the team can see them, how organized and commutative is that.
• love the alerts, people know about bugs as soon as I save them
• bug information automatically added would take me hours to put together
• snap shot capability during test execution, no more folders full of screen prints
• the test steps are added to the bug for me what a time saver
• Test Impact – (normally it is the facial expression, such emotion, then demo it for people!)
• now that I don’t have to manually note my steps during Exploratory testing, I can do a lot more testing and concentrate 100% on what I am doing

Note: the above are not direct quotes they are however truly what I am hearing testers saying.

Check out Scrat and if you need help give us a call.

Testa

IE9 support in Test Manager and Coded UI testing

Overall IE9 is supported by Test Manager and Coded UI testing. You should check here for details on what exactly is supported and what isn’t.

You will need to download SP1 feature pack.

Testa

Microsoft Test Manager– What is my test state at the Test Plan level?

Check out the Visual Studio Team Test blog on using Excel reports with MTM to identify test state at a test plan level. This is a step by step on how to create the report in excel using information on your test state from Test Manager.

Testa

Microsoft Unlimited Load Testing Announcement

Microsoft announces unlimited Load Testing for Visual Studio 2010 Ultimate with MSDN Subscribers.

Read up about this huge deal and download today the VS2010 Load Test Feature Pack then start your load testing.

If you need help call Object Sharp we can help you with your Load Testing.

1-877-SO-SHARP

Testa

.

Testing Tools Your Teams Want–Microsoft Visual Studio & MTM

Microsoft has launched a new web site Testing Tools Your Teams Want

There are concept video’s covering testing topics.

White Papers on Performance and Stress Testing, Effective & Efficient Testing, Integrated Lab Management

Research and Evidence speaks for itself.

If you want to find out about Microsoft Testing tools this is the place.

Testa

Microsoft Test Manager (MTM) – action recordings

MTM 2010 has opened up the world of manual testing with data-driven and fast forward capabilities. Plus, they have taken extra steps to ensure your successful here is an example of one.

Your are opening an application from a short cut on your desktop. Test Manager will capture and store the path to where you have placed the application executable and the path based on the %USERPROFILE%. In my example it looks like this:

 Launch '%USERPROFILE%\Documents\Visual Studio 2010\Projects\Calculator\Calculator\bin\Debug\Calculator.exe'

In this example if I remove the executable from my desktop and pin it to my start menu the test will execute successfully. If I then remove it from my start menu and pin it to my task bar the test again executes successfully. In addition the test actions is opening the executable not following the human interaction of double-clicking. This makes your test even less subset able to failure. Note that these types of failures are not application issues and therefore we’d really like to not have to deal with them.

Thanks to Microsoft there is less chance of this slowing our testing down due to setup issues.

Testa

The Testing Planet–all about testing

The Software Testing Club puts out a magazine call “The Testing Planet”. It is a must to read you can either buy a subscription to the magazine or download the PDF version for free. The magazine is created by “testers” with articles about testing by testers. The new issue just came out – check it out.

You can follow the club on Linked IN, Facebook and Twitter.

Enjoy … Testa

Microsoft Test Manager, Coded UI (CUIT), Web Performance & Load, Bugs, using TFS2010 – 3 day course in Toronto 10% off

If you have never been to Toronto here is your chance plus while here you can learn about Microsoft Test Manager, Coded UI testing, Web & Load testing, the new Bug, Fast Forwarded Manual Testing, and all you need to know about TFS2010 the communicator. Here is your chance. Check out our 3 day course offering being held March 9th to 11th and get 10% off.

Contact: trainging@newhorizon.ca 

Tell them Testa send you …

Microsoft Test Manager–Preview a step during runtime

Have you every been in a situation where you have had to execute a test that you did not create or you were not involved in the project at the being. You could use a walk through or in the least more information on what this test is about, how each step will interact with the application, it might be a bug your retesting. It could even be an exploratory test that someone had not created a true test case with steps. Test Manager during run time has a handy feature that lets you preview a step or steps.

See how and examples:

Select a step – any step within the test click on the Play dropdown arrow and select Preview

Here is an example of selecting a shared step for preview.

Here is an example of a step with multiple lines (see blog: Microsoft Test Manager–test case multi steps tool) and a parameter.

Last an example of a exploratory test case – you got to like this one.

Microsoft Test Manager–Test Attachment Cleaner tool

One of my favourite Test Manager feature is the data and diagnostics that collects information during my test runs to pass on in any bugs I log or get attached to my test run. The action recording proves the test case steps were followed, they allow me to fast forward the test next time I run the test or iterations. It has eliminated the bug state “Cannot Reproduce” The time all this data collected saves me in dealing with just bugs is unlimited. Developers love it too, they can diagnose bugs in no time.

I wonder if this helps in reducing the affect of the bug fix breaking other code. Now that would be interesting to investigate. Next project I’ll have to investigate all Test Impact and track how many were due to code changes involved in a bug fix.

Now I am in year five of using Test Manager and Visual Studio data diagnostics and my test runs and bugs are adding up in quantity therefore my database is getting full of all these test attachments. Do I really need to store the test attachments from a project that is done and 9 months old. Maybe not. What can I do?

The Test Attachment Cleaner a command line power tool that accepts parameters to remove test attachments form the database. Here are some examples:

• Identify list of attachments taking up database space of more than 1 GB per attachment
• View/Delete IntelliTrace log files over a size of 500 MB
• View/Delete all Video log files with Test run creation date older than specific date with no active bugs
• View/Delete all TRX & COV log files for test runs that happened between 30 and 90 days in age & do not perform the Linked Bugs lookup query
• View/Delete all custom/user attachments over size of 5 MB with no active or resolved bugs on test runs between 2 dates

Find out more and download Test Attachment Cleaner

If you need to keep the test attachment data it can be stored remotely which is setup in the Test Plan properties, test settings.

Testa

Microsoft Test Manager–test case multi steps tool

Testers have been asking for multi-step capability in test cases. Shai Raiten has created a tool that enables test case step multi lines. The Test Steps Editor enables editing of actions and expected results to add multi-lines. The tool works with both Shared Steps and Test Cases.

Here is an example of the tool:

You need to know your work item id, enter then click Load, the test case steps displays.

Put your cursor at the end of the line above where you want to add a line.

Alt+Enter adds a line below the selected line.

Click to download the Test Steps Editor.

Thanks to Shai,

Testa

Microsoft Test Categories to the rescue

Do you recall the Test List editor (the file in the Test Project .vsmdi) that was constantly causing the team headaches,? In VS2010 Test Categories has come to the rescue. Test Categories allow us to grouping our tests together which then can be run and reported by category.

Test Categories are set up through the Test Viewer in Visual Studio 2010.

1. Open Test View window (Test –> Windows –> Test View)

2. Select a Test and open the test properties (right-click on test and select Properties)

3. Find Test Categories then click the dropdown arrow to the right

4. Test Category window opens where you can:

• Add new Category
• See Available Categories
• Assign Categories

Microsoft recommends using Test Categories instead of the Test List functionality, however if your using the Test Check-in Policy you will still require the test list. You can use both the test list, for your check-in policy and then test categories for all other tasks.

If created a  Build Verification Test list using the Test Editor list no problem, create a Test Category named BVT and instead add this to your Build definition.

Build Definition:

Under the Basic section, in the automated tests section open the Test Assembly then click and select a  Category Filter. The test category filter consists of one or more test category names separated by the logical operators '&', '|', '!', '&!'. For example, TestCategory1&TestCategory2 will run all tests with a test category of TestCategory1 and TestCategory2 . Or you can just select all tests in one category by entering TestCategory3. (The logical operators '&' and '|' cannot be used together to create a test category filter.)

1. Share your usages of Test Categories by adding comments to this blog, I will publish them.

2. Test Category for Smoke or Regression tests, a portion of an application …

Testa

Test Manager, The Story of a

I am a member of the The Software Testing Club which is a global and professional community of software testers. The community shares information through online question and answer sessions and access to test related bloggers, updates on testing events worldwide, The Testing Planet online articles, and much more. If you  are not a member I highly suggest you join. Follow the link above.

A member of the club, Rob Lambert wrote a story “The Story of a Test Manager” that I spent an hour while drinking my morning coffee reading. The story is fictional however, I bet you can relate to the main points through your own experiences as a software tester. As I read the story I wanted to tell the characters about TFS2010 but then I want to tell everyone even my hair dresser. My dogs love hearing the words TFS they think it’s a cookie!  Microsoft TFS nerd and proud of it.

Thanks to Rob Lambert for a great story.

Testa

Microsoft Test Manager–configurations

I’ve been asked about the managing of test configurations. In the sample below I have two test configurations set up as I add test cases they are automatically assigned both test configurations.

You can have multiple test configurations set to be the default. Which means all test cases are assigned these default configurations.

When I click on a Test Case then the icon Configurations I can override the default and remove one or both configurations from the selected test case.

You can select all your test cases in a test suite (using the std Microsoft selection hotkey) and then click Configurations to update a bunch at once.

I just added a new configuration named My Added Configuration. When I select the test case and click Configurations the Select Test Configurations opens. I then have to click All configurations and my view looks like display below:

If you do not click All configurations you will not see the newly added configuration(s). It’s a trick I know why not just show them all. Hey, were tester’s we should be enjoying the challenge!

Hope this help ..

Testa

Why Software Fails - your not using TFS2010 & Test Manager?

A colleague emailed asking if anyone had seen this article. It was written in 2005 and is very enlightening to read even today. I thought I’d post it for people to read. Click here Why Software Fails

As outlined in the article here are the most common factors:

• Unrealistic or unarticulated project goals

• Inaccurate estimates of needed resources

• Badly defined system requirements

• Poor reporting of the project's status

• Unmanaged risks

• Poor communication among customers, developers, and users

• Use of immature technology

• Inability to handle the project's complexity

• Sloppy development practices

• Poor project management

• Stakeholder politics

• Commercial pressures

Has this changed, in my opinion not enough yet. As a strong advocate of Agile and a user of Microsoft TFS2010 and Test Manger - the communications is available we just need to get better at it and share the responsibilities as a team.

Testa

Microsoft Test Manager - Feature Pack 2 is available

Feature Pack 2 is now available and includes the following: Click to Download 

• Codes UI Test [CUIT] Editor

• new CUIT Editor, opens by selecting the UIMap.uitest in Solution Explorer

• click here to see Editor

• LightSwitch applications [Microsoft]

• tested and works well with LightSwitch applications

• Microsoft VS 2010 Test Package for Silverlight 4

• test Silverlight apps and other desktop applications

• enables testing with Coded UI tests and record, playback in Microsoft Test Runner

• data and diagnostics can be collected during runtime however, Intellitrace logs are not as of yet

• tested to work with Silverlight 4.0 apps hosted in IE (adding Silverlight apps in future)

• Note: read the “about” for restrictions

• Microsoft VS2010 Test Package for Mozilla FireFox

• enables playback and UI actions for Firefox 3.5 and above
• enables testing with Coded UI tests (CUIT), automated Web Performance and Manual fast-forward tools
• you can create a set of tests once and execute on both IE and Firefox from Microsoft Test Runner

You need to download KB2403277 which can be done before installing or during the installation process of Service Pack 2.

Click to Download 

Testa

Microsoft Test Manager– Test Plan shortcuts

Quick access to Test Plans using the “Copy URL for plan” option in the Testing Center window.

Steps:

1. 1. Open Test Manager
2. 2. Open the TFS Team Project that your Test Plan resides in
3. 3. Highlight the Test Plan by selecting that you want to create a shortcut for
4. 4. Click on the Copy URL for plan icon in the toolbar menu (see below)

(Note: when you click the “Copy URL for plan” icon the URL address to the plan is stored on your clipboard)

5. Next, go to your desktop and right click, select New then select Shortcut

6. The Create Shortcut window opens, paste the URL from the clipboard, using either Ctrl-V or right-click Paste, now click the next button.

7. Create Shortcut window opens where you enter a name for your shortcut. Make sure you are entering the actual Test Plan name or using a naming convention that allows you to easily identify the shortcut. Click Finish. (Notice a name by default is given: “New Internet Shortcut” if you miss entering your own)

8. The short cut is created and looks like below, when you click on it the Test Plan opens.

Testa

TesTrek 2010 – Effectively Managing the Testing Process through Collaboration

Dave Lloyd & I did a workshop at TesTrek today. What a great group of people, lots of interaction, and no one left early, thanks everyone. If your interested in learning more about Microsoft Visual Studio and ALM's contact ObjectSharp. Tell them you were at TesTrek with us.

You can download our PowerPoint presentation here.

thanks everyone,

Testa

Microsoft Test Manager – hot fix available

On Oct. 12th Microsoft has published a hot fix for issues that were causing Test Manager to crash and memory leaks that caused data and productivity loss. After downloading the hot fix I noticed that in the Test Case iterations section that are new buttons and bugs fixed. See what I’ve found so far below:

1. You can now delete test case iterations without crashing MTM. And working with data in the iterations seems smoother.

2. You will now have a “Remove parameter” button which allows you to remove a parameter from you iterations and/or test steps.

3. You will also have a “Rename parameter” button which allows you to rename a parameter in your test case iterations and/or shared steps.

4. Last you will have a “Delete parameter” button which allows you to delete a parameter in your test case iterations and/or shared steps. Deletions with this button only removes the @ sign from the parameter in the steps. You can still highlight the parameter in the step and click the delete button on your keyboard, this removes all instances of the parameter in your test case.

Note: shared step parameters display in associated test cases however, you can only Rename, Remove or Delete parameters from within the shared step itself. At the moment you are able to select the shared step parameter and proceed with selecting the test case remove, delete or rename button and proceed again as if executing the button feature right to the end. The action does not actually work!

Note: the issue with deleting a parameter in a shared step and the associated test cases not being updated still exists. This hot fix does not include a fix for this.

Click this link: hot fix KB2387011  to download.

Microsoft Test Manager – Video Recordings

Did you know that you have the option of either saving all Video Recording or just saving when you create a bug?

Check the Video Recorder configuration button and set up when you want to save recording plus the quality of the video.

The default is set to save all video recording.

Testa

Test Manager & VS2010 automation playback in FireFox

Microsoft has released a plug-in for Mozilla FireFox that allows you to record a action recording, a coded UI or web test in Internet Explorer 7 or 8 and play the same test back against FireFox. The current plug-in supports FireFox 3.5 and 3.6. The plug-in can be downloaded and installed on top of an existing 2010 install.

Download here: Test Package for Mozilla FireFox Power Tool

Note: after you download read the ReadMe file there is some configuration setup required before you can use this tool.

Testa

TesTrek 2010 in Toronto this October.

TesTrek in Toronto is coming this October check out the website at this link for more information.

Dave Lloyd and I are doing a workshop on October 20th at 11:00am - 12:30am

Effectively Managing the Testing Process through Collaboration
Debra Forsyth and Dave Lloyd,
Object Sharp Consulting

Communications is the key to the success or failure of any team, whether it’s a sports team or a software development team creating the next app. Without communication, the team members have no idea how best to direct their efforts. With communication, the team can overcome almost any challenge. Unfortunately, ensuring that the level of collaboration required in generating success can be a difficult goal to achieve. Solid development managers have a bag of tricks that they utilize to foster communications among the different roles. In this session, you will learn some of the tricks that can be used in your own environment to create better communications across all of the team roles.

• How can the other team roles more effectively communicate with the testing team?
• What does the test team need to know to complete the testing effort?

Special 15% Promotion Discount to TesTrek:

To receive a special 15% speaker promotion discount, please contact Darrin Crittington and give the special promotion code of ‘Spkpromo2010’ along with our names either by emailing dcrittington@qaiworldwide.org or calling 1-866-724-6013.

Testa

Microsoft Test Manager: How to manage test case/test configuration assignments …

In my last blog there where questions about how to assign or unassigned  test cases to test configurations (browsers, operating systems).

How to manage test case/test configuration assignment:

• In the Plan Tab select a test suite then test case.
• In the Test Suite panel toolbar click on the Configurations button.
• The Select Test Configurations window opens where you can add and remove test configurations associated to a test case.
• Simply click button All configurations, all test configurations you have created are added to the test case.
• Next click in the test configuration cell to add or remove.
• Last, click the Apply Changes button.

Whether you add test configurations before or after setting up Test Suites and Test Cases you can manage test case assignment by using the Test Suite | Test Case, Configurations button.

Note: Test Suite/Test Cases are automatically assigned to all test configurations created prior to there own existence.However, if you create additional test configurations after creating Test Suites/Test Cases they are not automatically assigned. In this case you can assign using the above steps.

Thanks to Jeff Levinson (Team System, MVP) for helping out with this question.

Testa

Microsoft Test Manager – Test Configurations & Test Cases

Duplication of tests for configurations does have a flow that you need to be aware of …

Microsoft Test Manager – Test Plan Configurations

If you need to test against different test configurations you are going to love Microsoft Test Manager.

Test Configurations are different setups that any application requires testing against. Example: Microsoft Test Manager would have been tested against different operating systems like XP, Vista, or Windows7. A web application may require testing against different browsers like IE7, IE8, Fire Fox, Chrome or Safari.

The ability to set up test configurations is only limited to your needs. One of the best features is the ability of Microsoft Test Manager to simulate a duplication of test cases for test configurations assigned to a test plan for test runs only.

It does require management to some extend however;

• it does save your team time in creating duplicate test cases or excel spreadsheets of what configurations will or have been tested.
• it is only really one instance of the test case so an changes affect all instances in the test run.
• the test plan documents what configurations will been tested and by which test cases.

There is significant savings to the test team for this feature to be one of the top.

Test Plan – test case shows once only.

Test tab >Run Tests – test case is shown for each test configuration set up in our test plan.

Over the next few blogs I will tell you more about Test Configurations and how they affect test cases and test runs.

Visual Studio 2010 – Web Performance Tests

Learn about Web Performance Tests authoring & debugging with Visual Studio 2010. Ed Glas is the Load Test product team manger at Microsoft and has posted some very good reading material on the subject of web tests.

Web Test Authoring and Debugging Techniques for Visual Studio 2010

Great reading Ed … Testa

Microsoft Test Manager – bugs being fixed in future update

When attempting to paste (using Ctrl-V) data from the clipboard into a parameter within an Iteration you must use the spacebar or manually click into the field that you are populating. If you don't have a cursor blinking even thought the box is highlighted (has focus) using Ctrl-V creates another Iteration instead of pasting the copy data.

This bug has been report to Microsoft and should be address in a future update.

___________________________________________________________________________________

Created a Shared Step that has 37 parameters. We added the Shared Step to a Test Case and loaded  the data for one iteration. We then decided to remove two parameters. In the Shared Steps we deleted the two parameters and removed the steps. Saved and Closed. We then opened the Test Case expected the two parameters to be gone, they are not.

We have also had issues with changing the data in the parameters (Test Case)

1 remove the data then click the enter key

2. add the new data then tab out

3. save.

Work a Round's:

1. Deletion of parameter in shared step does not delete parameters in the test case.

There is definitely a bug with refresh here. I will file this issue. To workaround this, insert a dummy step and delete it. You will see that parameters go away. Let me know if this works for you.

2. Deletion of data

I agree, we do not have a good experience of editing data. You will have to hit the enter key to take the test case to dirty state. Only after that the data gets saved.

Testa

Microsoft Test Manager – adding parameters to Shared Steps

Shared steps can have parameters however, before adding iterations of data there are considerations you need to take into account.

If you consider logically how this works, adding a bunch of parameters and data to a Shared Step will have that step repeat running through the data you added. Then once all iterations of the shared step complete your next test case step would run. Make sense.

• What if I want the Shared Step parameters to run with other parameters in my Test Case?
• What if I want to add data to my Test Case Shared Step parameters months later?

The cleanest and most flexible way I believe and have found is to add data (iterations) to my Shared Step parameters from within the Test Case. Remember a Shared Step can be used in any Test Case and therefore needs to be flexible.

Example:

A Shared Step with Parameters > no data (iteration) added.

Shared Step with Parameters (userid & password) inside the Test Case

Above there are three parameters of which two belong to the Shared Step, the other the test case. You can add many parameter iterations to this test case without affecting any other test case that contains the same shared step. (If the UserID and Pswd’s were in the shared step that step would keep opening and login for each iteration before doing the test case step two.)

I have seen cases where testing always uses the same UserID and Pswd for login. In that case having these as parameters set within a Shared Step makes sense. If a new set of credentials is issued you can easily change them in one place, quick and simple.

Note: now this is confusing. I set my parameters in a Shared Step added the Shared Step to a Test Case. In the Test Case the shared step parameters display however they do not show any data.

[I will try a scenario where parameters are set in the Shared Step and in the Test Case to see what happens!  - will let you know the outcome]

Comments on this subject are welcome. If you have found a different way to handle a combination of parameters in shared steps and test cases or situations you have come across  – lets learn together.

Testa :-)

Microsoft Test Manager 2010 – Test Plan & Test Configurations & Test Cases

This is a very cool feature and a big test effort time saver.

In your test plan you have set up more then one configuration. Maybe you need to test in different browser types, or different operating systems, even different languages. You create one test case to test a feature/requirement and attach it to a test suite, setting suite state to In Progress. When you go to the tab Test and open Run Tests there is an instance of your one test case for every configuration assigned to the Test Plan.

For very large applications or web sites that require testing in many different configurations this feature not only saves time in the creation of many test cases but it creates a marker that the testing has been done.

Feature Highlight:

Many Test Configurations – One Test Case – Instance of Test Case for every Test Configuration.

Test Cases in Visual Studio

Creating or editing a Test Case in Visual Studio:

If you create a Test Case in Visual Studio you cannot enter steps, first enter a Title and save, then click the Open for edit button. Test Manager opens with the test case open ready for you to enter steps.

You can add links, file attachments and/or add to the summary from within Visual Studio.

Opening for edit a test case in Visual Studio does not allow you to edit the steps. Again you have to click Open for edit which opens the test case in Test Manager.

Creating or editing Shared Steps in Visual Studio works the same way as the Test Case.

Bugs

Website Not Invented Here had a great comic worth sharing.

Microsoft Test Manager - Test Suites

The MTM Tests Suites are how you organize your test cases. There are three test suites to select from they are, Requirement query-based, the query-based, or the static based. The two query based test suites will automatically add new test cases to a suite if they meet the query criteria.

When would we use the different test suites?

1. Requirement query-based is pretty straight forward. For each requirement you have at least one if not more test cases.

2. Query-based has to include a test case category condition which guarantees only test cases are added then you can query by any field in the work item.

•    Area Path of requirement work item
•    Project Iterations
•    By work item ranking or priority

1. 3. Static test suites  are multi functional. You either add test cases to the suite manually or you add query based test suites to the static test suite. This gives you additional options for organizing the test effort. Examples:

•       Iteration One (static test suite)
• Requirements based query – Requirement 1
• Requirements based query – Requirement 2
• etc…..

Work Item Categories is new in VS2010 and as you see are used in MTM for query-based test suites. Work Items up till now had one name for example the Bug. In VS2010 the out of the box work item bug can be renamed. In MTM work item categories are used to restrict the type of work item to show in a given list. The categories are:

I am sure there are lots of different ideas as to how to use Test Suites. I would love to hear them. please add comments with your Test Suite ideas.

Test Manager 2010 (MTM) - Test Plans and Suites

Microsoft Test Manager 2010 is now available, thank you Microsoft. MTM like Visual Studio connects to Team Foundation Server where all artefacts are stored. Like the bug, test cases are work items. What else does MTM have, lots. Lets look at the concept of a test plan first. You can have one test plan to many for a project. It is totally up to you the test team how you set up your testing effort on a project.

The Test Plan documents the test effort  thru:

• Properties: test environment, settings, configurations, assigned build, and details like state.

Test Suites: organize test cases. There are three types of test suites; one is based on a requirement, another is a query of work item fields and the third is static. The possibilities are endless.

Microsoft has taken the test plan document to the next level. You can update a test plan as test efforts change. You can copy it as a starting point for the next project. You can if needed extract the information into a word document using an add on tool. If your printing for management or auditor put it on the network and direct people to it – save the tree’s. Plus, I bet most people do not even read them.

Look out world the Test Plan has evolved.  :-)

Visual Studio 2010 At The Movies

ObjectSharp and Microsoft teamed up to launch Visual Studio 2010 and .NET 4 in Toronto on April 24th. I had the privilege of show casing Microsoft Test Manager 2010 (MTM), the new testing tool. With MTM you create Test Plans outlining test settings and environments, test suites that let you organize your Test Cases and assign a build for testing. If you have steps that are repeated in many test cases you can create a Shared Step. Add shared steps to many test cases however having only one place to manage these steps. When you want to test different sets of data against a Test Case parameters lets you do this. Showcasing my favourite feature Manual Testing Fast Forwarded. Hands up this test is running on its own now! Microsoft has created a testing tool that makes sense and will change the future of manual testing. Did you know that today 70% of all testing is done through manual testing? If you want to find out more about Test Manager, Coded UI Testing, Web Performance Testing or Load Testing get in touch with us.

Never Test Alone – TASSQ presentation in TO

Andre King & I presented “Never Test Alone” at TASSQ in Toronto yesterday evening. After the presentation Joe Larizza (TASSQ President) addressed the group saying that with the economy businesses are looking for ways to reduce costs. Outsourcing testing will be one of them. People need to be proactive in finding ways to reduce cost and produce quicker. Joe encouraged people to talk to their management teams, present the concept outlines in “Never Test Alone” and volunteer to give it a try. See how it works, what it saves and in the end you may be saving more then you expect.

In Never Test Alone testers are involved in the Requirements & Design reviews and validations. This allows the test team to prepare for testing early, creating test cases, test scenario's and other documentation at the start of the project. Plus testers add value to the process from their experience and perspective. This reduces the cost of the test group weeding through requirements later to get an understanding that may not even be a correct one of the system. Enhancements that testers log during testing will come out during the requirement review phase. If deemed required can save money by being incorporated before development starts versus later.

Team up testers with developers during the creation of unit tests, the more tested at this stage the less cost in fixing, less bugs later, and we can reduce the duplication of testing being done between the two groups. Testers now know what has been tested therefore will not have to retest later. A testers perspective in what needs tested will differ from a developers so the combination of the two creates very rigorous unit tests.

If you are doing continuous builds (http://en.wikipedia.org/wiki/Build_automation) the unit tests are run against each build becoming regression tests. So we now have during unit testing, the functional testing and automated regression testing happening very early in the project.

Unit tests can be used in load testing. One example is login where UserID and Password are entered. The unit test can be build to retrieve from a listing of many UserID/Password combinations to execute login many times. What is the savings in knowing during development that the system is not going to have problems with 1,000 concurrent logins? Add this test to the build regression tests and you know throughout your project how well it is standing up to load. Why leave load testing to the end, do it early and save time, money and possibly the project.

Andre demonstrated a tool for automated white box testing, Microsoft Research PEX (http://research.microsoft.com/en-us/projects/Pex/). Very impressive and it can be run against code by testers saving development time for development. Check out the link above for a white paper on PEX. Maybe Microsoft will make a version of PEX that testing can use which does not require us to have access to the actual source code.

This is a list of some effects from following this presentation:

• testing a lot earlier
• saving money on bug management
• saving money on system changes late in the project
• reducing timelines
• building a team relationship
•  great feeling of accomplishment

And you are not testing alone, it is a group effort and it is the route to success.

 Link to TASSQ

Presentation Slides

Testa

Never Test Alone - KWSQA Conference

Bruce Johnson and I attended the KWSQA Conference yesterday in Waterloo. We teamed up doing a session at the conference. Our topic was "Never Test Alone - the Route to Success". Bruce wowed the crowd with a demo of his test-driven development skills and the Microsoft Research tool Pex (Automated Whitebox .Net testing tool). As we promised the presentation slides can be downloaded see link below. Congratulations to the KWSQA group for a great job.

Never Test Alone presentation slides 

Research Microsoft Pex

If we can be of help or answer any questions please don't hestitate to contact us.

Testa

CAST 2008 will be held in Toronto

There are not many conferences on Software Testing that are held here in Canada, well this year a big one is. For the first time ever CAST 2008 conference is being held right here in Toronto.  The show theme is Beyond the Boundaries: Interdisciplinary Approaches to Software Testing.

Mark you calendar for July 14-16th.

If your interested in speaking the Call for Papers deadline is February 4th, 2008.

To see more information on the conference click this link CAST 2008

VSTS Edition for Testers Presentation at TASSQ

Tuesday Jan. 29th, 2008 at the Toronto Association of Systems and Software Quality (TASSQ) meeting join myself and Barry Gervin for an evening presentation on VSTS Edition for Testers. Click the link for details about this presentation including a link for registration.

Hope to see you there ... Testa

Community for Software Testers @ Microsoft

Microsoft has launched a new community for software testers to share and learn about everything testing. There are and will be videos, articles, and blogs all dealing with software testing.

Join the Software Testing Discussion Forum and participate through out conversations.

Check it out -> Microsoft Tester Center 

VS Team System 2008 (Orcas) Web Test Correlation Helper

Sean Lumlev from Microsoft has done an excellent job explaining Web Test Correlation and how VS Team System 2008 (Orcas) new helper feature works.

Check out Sean’s blog by clicking this link Web Test Correlation and read all about it.

VS Team System 2008 (Orcas) Test-level Validation Rules

       VS Team System 2008 (Orcas) has improved validation rules.

Test-level validation rules allow you to add the same rule to every request by simply clicking the Web test node and selecting Add Validation Rule. A dialog box opens select the validation rule, in the properties you can edit or add options and parameters. Click Ok and you’re done.

Reasons for using a test-level rule:

·         Errors in the server application can cause incorrect flow causing the playback of the Web test to vary from the recorded sequence

·         Validate content that must appear on every page

·         Validate that all pages conform to a specific criterion

       VS Team System 2008 (Orcas) has improved validation rules.

Listed below are new validation rules:

·         Stop test on error

·         Search request and response

·         Add validation rule for title

·         Redirect validation

·         Provide test level validation rules

·         Expected HTTP code

·         Warning level for errors on dependents

VS Team System 2008 – Web Testing New Features

AJAX Requests & JavaScript Pop-ups

Currently VSTS 2005 requires an add-in tool called Fiddler to record AJAX requests. If you are not sure what AJAX is watch this video Measuring the Business Value of AJAX . As shown in the video using Fiddler we have the ability to capture AJAX requests, then using VSTS we are able to manage the Fiddler recordings in VSTS, see the requests in results and add the tests to our Load tests. If you watch the video the value AJAX adds is pretty impressive.

VSTS 2008 (ORCAS) now gives you a new feature, it captures AJAX requests with the web test recorder. 

Along with AJAX the web test recorder also is able to record JavaScript pop-ups.

Fiddler is no longer required – VSTS 2008 Web test recorder does the job.

New Features coming in Orcas - VS Team System 2008

VS Team System Performance and Load Testing - What is all the other terminology in comparison?

Surfing the internet can be mind boggling when searching industry terminology. Try searching terminology for testing that a solution is able to handle its data obligations while conducting itself in a suitable fashion. How do you like those words?

Terminology I found and have experienced:

Stress Testing

Load Testing

Performance Testing

Volume Testing

Concurrency Testing

Endurance Testing

Scalability Testing

Soak Testing

What does all this mean and why has Microsoft in Team System only used two of these?

I believe Microsoft in Team System has done the right thing. How many terms do we need to identify testing a solution can handle real life usage with peak times and large amounts of data?

VS Team System Performance testing is designed for developers to measure, evaluate, and target performance issues in their code. Profiling is used in performance testing; it is the process of observing and recording metrics about the behavior of solution code. Issues normally stem from code that performs slowly, or uses too much system memory.

Load testing is used to simulate many users accessing a server at the same time. In load testing, the web tests simulate multiple users opening simultaneous connections to a server and making multiple HTTP requests. Unit tests added to a load test can be used to test data access.

With all the features of the Load tool you are definitely stressing the solution, adding volume, testing concurrency, endurance, scalability and soak so why all these terms. With a combined performance test of developer code and load testing of the solution, we are testing the “Performance” of the solution in my opinion. Let’s keep it to the two terms so everyone understands what’s being tested and why.

Quote for today.

The ability to simplify means to eliminate the unnecessary so that the necessary may speak. (Hans Hofmann)

Have a great weekend … Testa J

Load Testing Early and Often with VSTS

I have blogged about testing early, getting your test team involved at the start of the project one item I neglected to include was Load testing. So let’s catch up and talk about when load testing should be done. A lot of projects wait till either UAT or just prior to implementation to perform Load testing, does that make sense?

What is Load Testing?  --  It is the process of creating demand on a system or devise and measuring its response. It is modeling the expected usage of a software program by simulating multiple users accessing the program’s services concurrently.  

With VS Team System Load Tester you are grouping and running your existing unit and web tests. When a build is ready the unit tests are run against that build. We are verifying the build quality and regression testing that any fixes made did not break something else. The QA team gets a quality build and run their web and manual tests and report any bugs. If the bug’s reported are not high severities then let’s put the unit and web tests into a load test and run it. You’ll have to do some planning ahead like a clean environment and data set up for the load testing. An important factor during load testing is having an appropriate data size to find issues like missing indexes, to many indexes, locks, bad query plans, the list goes on and on.

Load testing early will find performance issues sooner which in most instances are due to architectural issues in the application. Finding these just before implementation can turn into a No Go decision. Finding them early allows lots of time to analyze and make the necessary changes, and you can retest the fix without delaying other testing, development or implementation.

This is a best practice that with VS Team System allows us to do so easily it’s crazy not too.

Quote by Andy Warhol (1928-1987): “They always say time changes things, but you actually have to change them yourself.”

Lets change Load Testing start early don’t leave it till the end.

Enjoying a sunny summer … Testa J

VSTS - How to associate work items to tests

Every test you create whether it is Unit, Manual, Web or Load can have work items associated to them. If you’re working with the MSF Agile process the work item Scenario is a detailed requirement, a bug is a work item, risk is documented in a work item. You have the ability to associate work item(s) to your tests.  Remember that work items are stored in the TFS as is test results, you can run queries against anything stored in the TFS, you therefore have the ability to report on the testing of requirements, risk, bugs. Think about knowing that a bug has been retested and that it’s passed. I know prior to VSTS we used a lot of excel spreadsheets and human intervention to document this information, now it’s a click of your mouse, select the work item to associate and your done. Run your queries and the information displays.

How to associate a work item to a test …

To access a tests property open the window Test View which gives you a view of all your tests created. Right click on a test and select properties, the Properties window will open under “Misc” you will find the property Associate Work Item. Note that you can move the Properties window to the right of your screen under Team Explorer/ Solution/Project and Test View.   You can also do this using Test Manager.

Next Steps:

1.       Click the button icon in the Associate Work Item property field

2.       Window opens with an Add button enabled

3.       Click the Add button

4.       The Work Items Picker window opens

5.       Select the Team Project your working on

There are three ways to get the work item(s) you want to associate you can select a query, key in the work item identifier number, use the work item title and type. The easiest way would be to find work items by query.

6.       By default the option “Saved query:” is selected click the dropdown and select query named “All Work Items”

7.       Click the Find button

The Select items to add to the work item list displays all work items in your project and by default they are all selected.

8.       Click the Unselect All button

9.       Pick a scenario or bug or any work item you please

10.   Click OK

Your returned to the previous window with the work items identifier’s displayed. If you’ve selected incorrectly you can click on an id and remove or click Details to see the actual work item.

11.   Click OK again

The test property Associated Work Item now populates with the work items identifier number you selected.

Picture of the windows talked about above:

Manual Test Property:

Work Item Picker:

Run queries and see what scenarios your tests are covering, if your real good query to see if what scenarios don’t have test associated.

Try it out … Testa

Visual Studio Team System Add-in: Scenario Coverage Analyser for TFS

We've heard of code coverage which is normally used by the developer's to verify how much of the code they've written was tested by their Unit tests. Microsoft took it a step beyond developers and gave the QA the ability to run code coverage against our tests. You can with any test type including Manual Tests have code coverage running during execution and see how much of the code is being tested. It's a great way to know whether or not the QA test suites created are covering all the code. With both developers and testers running code coverage against a Build we get a real good idea of how good that Build really is.

Check out Eric Sink's blog on Code Coverage at -> http://www.ericsink.com/articles/Code_Coverage.html

A fellow by the name of Paul Stovell did some creative thinking and came up with a tool called Scenario Coverage Analyser for TFS, which basically is an MSBuild task. Scenario Coverage Analyser allows you to summarize your code coverage statistics in terms of the Scenarios they cover. We are assuming you’re using the work item Scenario to document your requirements in VSTS.

A quick explanation of how this works, you modify the Team Builds to call Scenario Coverage Analyser. You run your Team Build and Unit Tests with code coverage Scenario Coverage Analyser does three things;

1. Looks at the code coverage results generated
2. Maps the individual code coverage results to the work items that the code was written for
3. Produces an HTML report called Traceability Matrix

This add-in tool only runs with Builds so it does not work with the QA Test execution, but it gives the QA Team a report on how much of each Scenario (requirement) was tested by the Unit Tests.

Thoughts to comment on:

If a Scenario shows that the unit test tested 90% of its related code how much more testing of that Scenario would QA really need to do?

What % of Scenario Coverage would be acceptable?

Would this be a report you'd share with your customer?

Is there a like tool that could do the same analysis of code coverage run against the QA Tests?

Wouldn't that be a nice add-in for the QA Team?

To see more on this great add-in tool here is a link to Paul Stovell's blog ->

http://www.paulstovell.net/blog/index.php/scenario-coverage-analyser-for-tfs/

Paul’s how to set up Scenario Coverage Analyser ->

http://www.paulstovell.net/blog/index.php/scenario-coverage-analyser-for-tfs-setting-it-up/

To download the tool -> http://readify.net/Default.aspx?tabid=269

Try it out and let us know how it works for you - add a comment to my blog with your thoughts.

Thanks to Paul for this great addition to VSTS.

Another sunny day ... Testa

Team System (VSTS) Course for the Quality Assurance Team

ObjectSharp is now offering a Team System Quality Assurance Best Practices course of which I blogged about a few blogs ago.

 

 The course and labs cover the following topics:

 

• QA Role in the Software Development Life Cycle
• Overview of Visual Studio Team System
• Team Communications
• Working with Visual Studio
• Version Control
• Manual Testing
• Web Testing
• Other Testing
• Load Testing
• Builds
• Defect Tracking

For more details on the course go to:  http://www.objectsharp.com/training/CourseDetail.aspx?id=5030

 

Link to the public course schedule:  http://www.objectsharp.com/training/ClassSchedule.aspx

 

We offer a public course in Toronto or we can host your team’s private, customized course at our fully equipped facility.  We also offer onsite, customized training with our portable laptops.

 

I want to also share with you a great place for a weekend away. I was at a family wedding this past weekend and stayed at the “Kingbridge Centre” which is a conference centre that is open for anyone to stay.  I highly recommend it, great place with lots to do and it’s just 45 minutes from downtown TO. Check it out at www.kingbridgecentre.com . The wedding was great too!

 

On the phone in our room at Kingbridge was a quote by Alexander Graham Bell it says alot about what I believe is the key to success in IT projects.

 

“Before anything else, preparation is the key to success” 

 

Talk to you later … Testa

VSTS: QA & Project Community Early - documents, requirements, work items

VSTS Team Explorer about ....

What better day to blog then a rainy one, Visual Studio Team Systems Team Explorer is the topic today.

What is VSTS Team Explorer about?

In my mind I think of it as “The Communicator”. Team Explorer is the central point of Visual Studio Team System projects it contains the artifacts that are shared between project teams. Team Explorer gives you a view of what is stored in the Data Warehouse (Team Foundation Server - TFS) in a sorted, organized tree view. Access to information is controlled by roles and set role permissions. Team Explorer can be configured to display all Projects you are a member of or specific ones, it’s up to you. You are given a My Favourites where you can upload documents or move existing documents to for easy quick access.  Project Portal is accessed through Team Explorer. Team members can use the project portal to save/retrieve documents, view reports, exchange information by posting messages, and use other SharePoint collaborative features like calendars and lists. Alerts are emails sent from TFS advising the subscriber that an event has happened. An example of alerts is the status of a work item change, a check-in occurs, a build is completed, or when a build status changes etc… Alerts can be subscribed to through the Project Portal.

Below is a view of Team Explorer (VSTS Edition for Testers & MSF Agile process).

What are the VSTS Team Explorer artifacts?

1.    Project Process

• A Process is a collection of defined processes, activities that yield a result, product, or service; usually a continuous operation. A series of actions or operations designed to achieve an end. 
• VSTS comes with two process templates that you can select from they are MSF for Agile Software Development and MSF for CMMI Process Improvement.
• Both process templates can be customized to meet your business needs.
• The project process you select shapes how your project is set up in VSTS. It will affect other default artifacts that come with VSTS.

2.    Project Guidance

• The Project Guidance is a document accessible from Team Explorer that defines and explains the project process.
• Within the Project Guidance the processes, the roles and the artifacts used in the process are documented along with how they interact. 
• It is a definition and mapping of the whole process, the roles, the processes and the default artifacts.
• If the project process has been customized the Process Guidance will need to be updated to reflect any changes.

3.    Work Items:

• Each project process comes with pre-defined defaulted work items.
• Work items are items that define work to be done in the project.
• Pre-defined defaulted work items can be customized, or new work items can be added. Note that customizations can affect the project process, queries, and/or reports.
• Work items can be assigned to project teams and specific people in a team.
• Examples of work items: tasks, bugs, scenario’s, risk
• Each project process creates default work items that relate to the process an example is a Task work item to create the Test Approach. There are many pre-defined Task work items that are automatically created for work that needs to be done at the team project level. 
• Work items have states, which can be used to monitor the progress of the project at any given time in real time.
• Work items can be used to create to do lists for project teams, and/or team members.
• Work items can be associated to each other and other artifacts for example a scenario (requirement) can be associated to it’s related test (unit, web, manual are a few). Another example is a bug which can be associated to a test, a scenario, a build, even a task. The capabilities are endless.
• Work items are powerful communication tools to knowing what’s happening in a project, what’s left to be done, where problems exist, what’s left to be done, are all requirements developed and tested.  

4.    Queries:

• Process templates come with pre-defined default queries.
• You can create your own queries that you set up either for yourself or all team members to access.
• Query example would be My Task to Do List which would query for Task work items that are assigned to you and with a state equal to open. A Task work item is defined as a piece of work that has to be completed, an example; Create the Test Approach, the task work items defines what has to be done, who is responsible and the status.

5.    Documents:

• Process templates come with pre-defined default documents that are sorted in Team Explorer by project team.
• There is a place holder for Development, Requirements, Security, Test (QA), Project Management, Templates, and the Process Guidance, of which some have pre-defined document templates.
• Any document created outside of VSTS can be uploaded to Team Explorer; an example would be specific Test Plans.
• Your own company document templates can be uploaded into Team Explorer.
• A Test Approach template is a pre-defined template document that is available for the Test Team.

6.    Reports:

• Process templates come with pre-defined default reports that have been designed to allow reporting on the project.
• The pre-defined reports depend on the Edition of VSTS you have and the project process selected.
• Reports can be customized or created from scratch.
• Creating your own reports does require knowledge and understanding of Analysis Services which is the Data Warehouse and Reporting Services.

Below is an example of pre-defined default reports that come with VSTS Edition for Testers and the process template MSF Agile.

7.       Team Builds:

·         Process Template creates a folder where project builds are created, run and viewed.

·         By default the All Build Types is added which when opened displays all created builds, showing each build by name, status, quality and comments.

8.       Source Control:

·         Process Template provides access to the Project Source Control files.

Quote of the day: “The single biggest problem in communication is the illusion that it has taken place.”

Till next time … Testa

Inside Information - ObjectSharp to Announce: VSTS Testers course

Duration: 3 days

Description: This 3 day hands-on class will equip students with the techniques and best practices for using Visual Studio Team Edition for Testers and Team Foundation Server to manage the quality of software within the development life cycle.

This course is intended for Software Quality Assurance Professionals (a.k.a. “Testers, Leads & QA Managers”) who are either using or interested in learning about Team Foundation Server and Visual Studio Team System Edition for Testers.

This course will train students how to use the tools to author and execute various types of testing supported by Team System, and how to effectively use the Team Foundation Server defect tracking features as part of a holistic approach to software development. Students will learn about work items, builds, version control, managing tests, code coverage, executing test runs, reporting and how all these items are part of an integrated solution for Quality Assurance teams and the whole development life cycle process.

This course goes beyond teaching the basics of automated testing and defect management. It covers quality assurance best practices related to analyzing requirements, understanding the best testing approach, authoring effective tests (both automated and manual) and the integration of the quality assurance role within the larger development team.

The course content is a combination of lectures, slides, demonstrations, hands-on labs, group discussions and exercise from instructors who are not only experienced with the Team System tools, but also seasoned veterans of the quality assurance discipline. 

More News: 

ObjectSharp now has a Facebook Group check it out. If your not a Facebooker join in on the fun. It’s a great way to share stuff with friends, family, find old school chums, join groups of interest to you. 

A love affair with knowledge will never end in heartbreak. by Michael Garrett Marino
 
Have a good weekend … Testa   

Visual Studio 2005 Team Edition for Testers ... cool stuff for us tester's

As tester’s some of our worst project nightmares are:

 Forgot to tell you we just made changes to Requirement 439

Here’s a build, What's in it?, I’ve no idea

Project is behind, QA has 3 weeks to test

That’s not a defect – show me!

I could go on but I think you get the picture, well QAer’s Microsoft has come to our aid with VSTS and it’s our own edition.

Below are just some of the cool stuff thats in our edition and may help to eliminate some of your nightmares:

• Access to the Team Project
• Our own Project/Solution with document templates, everyone on your QA Team can share and communicate stuff about the project
• Version Control for our documents and all test scripts
• Work Items -> scenario’s, bugs, tasks, risk, and you can create your own customized work items and you can customize the out of box work items
• Ability to assign work items to other team member's
• Ability to associate a work items to other work items
• Ability to associate work items to your tests
• Ability to use Unit tests, improve them, write them, help development write them, good unit tests make great regression tests
• Build Reports that tell you about code changes, fixes, new functionality, the quality
• Workspace where your daily stuff is stored
• Ability to publish your work to a team data warehouse
• Capability to query the data warehouse
• Queries already built for you
• Reports already built for you or create your own
• Tools like Test Manager for organizing your tests into test lists
• Use Code Coverage – run along side your tests and see how much your tests are covering the actual development code
• Ability to share your test results, analyze them thru reports
• Multiple test types, Manual, Web, Load, Ordered, Generic, Unit
• The concept of our tests runs associated to a build and reported on for all to see
• Ability to track the status of our projects, know when you are done, know the state of things in real time
• Project Portal where we can put information for all to see, test results, reports, how are we doing stuff, metrics and much much more

My hope in posting this listing is to intrigue you to want to know more .. if  you have specifics you’d like to know more about just ask.

Did you know VSTS Edition for Testers won the Software Test and Performance Tester’s Choice award for .NET Test/Performance Solution category and was runner up for three other categories: Load/Performance Test, Defect/Issue Management, and Integrated Test Suite.

“Everybody's a mad scientist, and life is their lab. We're all trying to experiment to find a way to live, to solve problems, to fend off madness and chaos.”  by David Cronenberg ( Canadian Film Director)

Have a good evening … Testa

IT Project Success: Starting Early - How, Who, What, When and Why

Yesterday was about IT Projects being successful only 34% of the time. Today I’m blogging on how to being a IT Project with success in the air, our goal in the IT industry should be to increase that 34% to 90 plus percent.

To succeed we need to get all the IT project members involved early in the project. We need to plan the resources early, get them on board during the planning. Team leads should be involved in planning out and setting up the project team. Business Analysis, Architects, System Designers, Developers, QA testers/engineers, Release Manager, Trainers, Technical Writers, Help Documenters, there maybe other’s that your projects require whoever you need get those resources planned. If there are tools needed by any of the teams or the whole team plan for training. Plan your process whether it be Agile or XP you’ll find processes always vary depending on the project, the resources and the project type.

Once project planning has is done, get the resources on board for the first step, requirements review or as I like to call it the Design meeting. All teams should have representation including the business in the Design meeting. This meeting(s) is where all teams participate to;

• discussed and learn the requirements put forth by the business,  – build expertise of requirements across all teams
• validate requirements - can we do? - are they valid?
• learn about the proposed design and architecture
• identify requirements that are system needs versus functionality
• create a listing of scenarios from the requirements
• work out an initial build schedule and what scenarios will be in each build
• how can we best test, what types of testing will be needed
• what are the time-lines, are they realistic, if not now is the time to flag
• is there resources we don’t have, expertise we need

A lot of discussions along with learning happens in the design meeting(s). The list below is what you can expect out of these design meeting(s):

• team of experts in knowing and understanding requirements
• team relationship building, respect for other team/people expertise
• communicating to obtain a shared understanding
• eliminating miss interpretation of requirements
• teams going away with needed information to further plan
• better time-line estimates – more realistic
• proposed build schedule(s)
• identified missing resources, tools
• issues identified early that can be solved early
• extensive listing of scenarios
• elimiation of scope creeping - business user’s gain a comfort level that their needs are understood

 To quote Rod Stewart, “I wish I knew what I know now before.”

Anyone want to add to the list I’d love to hear from you, I can guarantee there are other positive affects from starting early with all teams involved.

Testa

Back to blogging

It's been awhile since I last blogged however my passion in my absence for Quality Assurance has not diminished; it's alive, excited and ready to go. I'd like to share with you my experiences and research on ways the QA Team can help change the success rate of IT projects. How we need to change the perception of the QA team, get involved sooner in projects, help our mates find more bugs earlier, change the concept of what’s tested and when, become "test developer's" (I'll explain that one later), automate our testing correctly, take advantage of developer unit testing, and last but not least what can VSTS Edition for Testers do for you, the team and the project.

So they say only 34% of IT projects are successful, yep out of 100 IT Projects 66 are not successful. What makes a project successful, completed on schedule, all valid requirements are meant, within budgets, testing has been done, etc… Why are projects not successful, we believe there are four major factors, one around requirements, the next around scope and the third involves bad estimates and four not testing early enough in the project. Requirements are poorly recorded, misinterpreted, missed completely, invalidate, not validated, and not visible to everyone. The business user’s are not involved after passing over their needs. Scope gets pushed out but time-lines don’t how many times has someone promised some new functionality or expanded functionality that was not part of the original requirements. Estimates, how long do you need to complete a task, we are not good at this either you see under estimating or over estimating. Being in QA which is the end of the line, these bad estimations of time affect our team most. Last testing to late or not early enough, bugs are cheapest to fix early in the project, as are requirements that are misinterpreted, not valid or causing scope creeping. The earlier testing is started the better chance of project success. So how do we address these four issues, better still how do we work towards increasing that 34% success rate to a higher number?

Over the next while I will blog on each of these issues, incorporating them into not only VSTS as the integrated tool that helps us manage our projects, our requirements, our time-lines, our tasks and help us to succeed. Whether you use VSTS or some other tools the concepts are the same so stay tuned for tomorrows blog "Starting Early - who, how, what, when and why.

See you tomorrow ... Testa

Blogging about VS2005 Team System

Ok, I said I was going to blog more on the testing features that are coming out with VS2005 Team System and I am. However, now that the Beta1 version is out I'm in the process of getting a copy hoping the Testing features are in this beta and going to do some exploring at which time I will blog on my findings.

So for now I'm on hold - hope by next week to get started. Just for interest I am also going to be trying out Compuware's testing tools that integrate into VS2005 and will blog about them.

TDD - some cheese with that wine?

I am seeing alot of talk about what's going to happen to the QA department with TDD - good news - it's going to be better - you might learn some new skills - your going to help development (it's a whole new relationship) - your no longer doing nitty gritty testing - start by reading as much as you can about TDD and the process get a head start it'll be worth it.

I've been doing alot of research on TDD and how it affects the whole System Development Life Cycle. I am excited. I've been in the Software QA world for some years doing both manual and automated testing, I really believe that TDD is going to be the best thing to happen for the QA team. In saying that I also believe QA people will need to start a new learning curve in using tools like XUnit in my case NUnit, MbUnit, and a bunch of others.

Developer's are not tester's nor are they suppose to be ... with that said in TDD they maybe writing a test before they actually write code but that does not mean that every possible assertion has been included nor does it mean that they have interpreted the requirement correctly. The test may pass but does that test cover all possible scenario's - probably not. Was the requirement interpreted correctly, was anything left out or missed. Keep in mind that requirements are not alwas telling the whole story or are they always complete. How many times as QA people have you had to ask questions about a requirement which resulted in adding more test cases/functionality to that requirement? This is still going to happen.

FYI - Assertion definition: A statement in a program that a condition is true at this point in the program. Useful for reasoning about how a program is supposed to behave.

Can we say each assertion is equal to a test case?

One approach in fitting QA into the TDD process which is my favorite is to pair with developers during the test code phase. QA people can work with the developer's making sure all assertions (scenario's) are covered in each test. Help in interpreting the requirement and what's required of it. It has been said that the developer's have to write their own tests they cannot wait for someone else too, I agree but a tester can help. I do not see pairing as slowing down development it's been shown that two minds are better then one and you have two different skill sets working together to create what will end up being more right then wrong. In fact in this approach the developer is writing the test code so the tester is doing their thing coming up with all the possibilities that need to be addressed by the requirement. For testers there maybe a side effect in this they'll likey learn how to write the test code not a bad thing to know.

Another approach is to pass over  QA a copy of the developers test code (often, as testing earlier is more efficent and less costly for all) at which point QA examines the assertions in the test code and makes sure that all scenario's have been covered. In this approach either the tester passes over missing assertions for development to add and make sure that they pass, if not they'll need to refactor the test code and implementation code. Or, have tester's that can add the assertions missing themselves run the test suite reporting to development any test failures.  I like the idea of adding the assertions myself makes my job more interesting.

Either approach is an asset to the whole project in that QA knows what has been tested therefore possibly eliminating test duplication which in return allows for more high level testing like performance, load, stress etc...  Note that if the test code covers all scenario's and are build as a Test Suite they can be run along side of each build therefore regression testing is being done daily. It is also possible to write xUnit integration tests they are more difficult but it can be done.

In thinking about this I have a suggestion for QA teams - learn how to code assertions in tools like xUnit it's not that hard then your team has the ablility to help development and take advantage of using these tools for further testing.

TDD does not replace QA there is still the usability of a system to test, depending on how involved in the test code you are a little or alot of functionality testing, another testing style I can see happening more openly is exporatory testing. QA can help out with the User Acceptance Testing which in TDD it should happen with each build not at the end, it's being called Customer or Business Facing tests - does this meet the expectation of the “Customer”. I call it looking for requirement omissions.

I had the experience last fall of using TDD with the system architects to test first using Microsoft Visual Studio ACT  the architecural design approach for performance and load ability. The design was proved to be able to handle performance and load requirements then it was built.

QA people do not be afraid that TDD is going to replace you - it's not - your going to end up with more interesting tasks and a new found respect for your skills.

Visual Studio 2005 Team System testing features ...

My goal is to spend some time trying out all of Visual Studio 2005 Team Systems new testing feature:

• Test Project creation
• Record a Web Test: records browser activity and contains a list of Web requests and various properties for each request
• Load Test:  is a test that is designed to put a server application under heavy user load to pinpoint performance and/or scalability problems. In Visual Studio, a load test can be based on a Unit Test or a Web Test.
• Web Test : is a test that is used to verify functionality of a Web application
• Unit Test: tests written to prove that production code will work or works, they map very closely to development code in name and scope (one Unit test, one method).
• Test Format Manual Test
• Word Format Manual Test

I will break down each feature into seperate blogs should anyone like to get all my notes I'll post them as articles (they include snagits of the screens) they are very detailed step by step notes.

Now getting back to the creation of a test project if you've worked with .Net development it is no different you can select either a Test Project or an Empty Test Project you can select the language you want your test scripts to be created in (C#, VB, J# and C++) etc... Once created you have a Solution with References and a Project where you can add items to your project, the items being the above test features.

Recording a Web Test

This feature is very cool, it captures all User interactions with the browser and creates a script for each request. Each script has properties like Think Time and others that I'd like to investigate more on and blog about later. Other features is the ability to bind data to a recorded input field, to add validation rules to a script. You can validate that a text exists on a web page by simply entering for instance “My WebPage“ in the validation rule text.

 Once tests are recorded you can playback, modify, the interesting feature in running these scripts is the display of not only the data gathered but also the Web page requested per script. All this information is kept with the script for review at any time after being run.

These Web tests are not being considered for usage as true QA Functional tests but so far I do not see in some instances why they couldn't be, the purpose of these Web Test recordings is to use them for Load testing.

I am just getting into trying out VS 2005 Team System and if you stay tuned I'll be blogging on more of my findings as I go along.  I am hoping to also have a try at Compuware's TestPartner (GUI - functional test tool) integrated into VS 2005 Team System which I'm looking forward and will blog on also. 

Visual Studio 2005 Team System

I watched  Visual Studio Team System end-to-end demo given at TechEd 2004 and found it to be very informative on how Unit, Web, Load and Manual Testing has been integrated into Visual Studio. I also learned that Compuware's TestPartner which is a functional testing tool can be integrated into Visual Studio 2005 Team System. The video is fairly lengthy but well worth watching.