Microsoft Test Manager – Test Plan Configurations

If you need to test against different test configurations you are going to love Microsoft Test Manager.

Test Configurations are different setups that any application requires testing against. Example: Microsoft Test Manager would have been tested against different operating systems like XP, Vista, or Windows7. A web application may require testing against different browsers like IE7, IE8, Fire Fox, Chrome or Safari.

The ability to set up test configurations is only limited to your needs. One of the best features is the ability of Microsoft Test Manager to simulate a duplication of test cases for test configurations assigned to a test plan for test runs only.

It does require management to some extend however;

  • it does save your team time in creating duplicate test cases or excel spreadsheets of what configurations will or have been tested.
  • it is only really one instance of the test case so an changes affect all instances in the test run.
  • the test plan documents what configurations will been tested and by which test cases.

There is significant savings to the test team for this feature to be one of the top.

Test Plan – test case shows once only.

image

Test tab >Run Tests – test case is shown for each test configuration set up in our test plan.

image

Over the next few blogs I will tell you more about Test Configurations and how they affect test cases and test runs.

Testa

Visual Studio 2010 – Web Performance Tests

Learn about Web Performance Tests authoring & debugging with Visual Studio 2010. Ed Glas is the Load Test product team manger at Microsoft and has posted some very good reading material on the subject of web tests.

Web Test Authoring and Debugging Techniques for Visual Studio 2010

Great reading Ed … Testa

 

Microsoft Test Manager – bugs being fixed in future update

When attempting to paste (using Ctrl-V) data from the clipboard into a parameter within an Iteration you must use the spacebar or manually click into the field that you are populating. If you don't have a cursor blinking even thought the box is highlighted (has focus) using Ctrl-V creates another Iteration instead of pasting the copy data.

This bug has been report to Microsoft and should be address in a future update.

___________________________________________________________________________________

Created a Shared Step that has 37 parameters. We added the Shared Step to a Test Case and loaded  the data for one iteration. We then decided to remove two parameters. In the Shared Steps we deleted the two parameters and removed the steps. Saved and Closed. We then opened the Test Case expected the two parameters to be gone, they are not.

We have also had issues with changing the data in the parameters (Test Case)

1 remove the data then click the enter key

2. add the new data then tab out

3. save.

Work a Round's:

1. Deletion of parameter in shared step does not delete parameters in the test case.

There is definitely a bug with refresh here. I will file this issue. To workaround this, insert a dummy step and delete it. You will see that parameters go away. Let me know if this works for you.

2. Deletion of data

I agree, we do not have a good experience of editing data. You will have to hit the enter key to take the test case to dirty state. Only after that the data gets saved.

Testa

Microsoft Test Manager – adding parameters to Shared Steps

Shared steps can have parameters however, before adding iterations of data there are considerations you need to take into account.

If you consider logically how this works, adding a bunch of parameters and data to a Shared Step will have that step repeat running through the data you added. Then once all iterations of the shared step complete your next test case step would run. Make sense.

  • What if I want the Shared Step parameters to run with other parameters in my Test Case?
  • What if I want to add data to my Test Case Shared Step parameters months later?

The cleanest and most flexible way I believe and have found is to add data (iterations) to my Shared Step parameters from within the Test Case. Remember a Shared Step can be used in any Test Case and therefore needs to be flexible.

Example:

A Shared Step with Parameters > no data (iteration) added.

image

Shared Step with Parameters (userid & password) inside the Test Case

image

Above there are three parameters of which two belong to the Shared Step, the other the test case. You can add many parameter iterations to this test case without affecting any other test case that contains the same shared step. (If the UserID and Pswd’s were in the shared step that step would keep opening and login for each iteration before doing the test case step two.)

I have seen cases where testing always uses the same UserID and Pswd for login. In that case having these as parameters set within a Shared Step makes sense. If a new set of credentials is issued you can easily change them in one place, quick and simple.

Note: now this is confusing. I set my parameters in a Shared Step added the Shared Step to a Test Case. In the Test Case the shared step parameters display however they do not show any data.

[I will try a scenario where parameters are set in the Shared Step and in the Test Case to see what happens!  - will let you know the outcome]

Comments on this subject are welcome. If you have found a different way to handle a combination of parameters in shared steps and test cases or situations you have come across  – lets learn together.

Testa :-)

Microsoft Test Manager 2010 – Test Plan & Test Configurations & Test Cases

This is a very cool feature and a big test effort time saver.

In your test plan you have set up more then one configuration. Maybe you need to test in different browser types, or different operating systems, even different languages. You create one test case to test a feature/requirement and attach it to a test suite, setting suite state to In Progress. When you go to the tab Test and open Run Tests there is an instance of your one test case for every configuration assigned to the Test Plan.

image

For very large applications or web sites that require testing in many different configurations this feature not only saves time in the creation of many test cases but it creates a marker that the testing has been done.

Feature Highlight:

Many Test Configurations – One Test Case – Instance of Test Case for every Test Configuration.

OSharp_MG_9190-1_jpg.jpg

Test Cases in Visual Studio

Creating or editing a Test Case in Visual Studio:

If you create a Test Case in Visual Studio you cannot enter steps, first enter a Title and save, then click the Open for edit button. Test Manager opens with the test case open ready for you to enter steps.

image 

You can add links, file attachments and/or add to the summary from within Visual Studio.

Opening for edit a test case in Visual Studio does not allow you to edit the steps. Again you have to click Open for edit which opens the test case in Test Manager.

Creating or editing Shared Steps in Visual Studio works the same way as the Test Case.

Bugs

Website Not Invented Here had a great comic worth sharing.

Tester Comic Strip

Microsoft Test Manager - Test Suites

The MTM Tests Suites are how you organize your test cases. There are three test suites to select from they are, Requirement query-based, the query-based, or the static based. The two query based test suites will automatically add new test cases to a suite if they meet the query criteria.

When would we use the different test suites?

1. Requirement query-based is pretty straight forward. For each requirement you have at least one if not more test cases.

2. Query-based has to include a test case category condition which guarantees only test cases are added then you can query by any field in the work item.

    •    Area Path of requirement work item
    •    Project Iterations
    •    By work item ranking or priority
  1. 3. Static test suites  are multi functional. You either add test cases to the suite manually or you add query based test suites to the static test suite. This gives you additional options for organizing the test effort. Examples:
    •       Iteration One (static test suite)
        • Requirements based query – Requirement 1
        • Requirements based query – Requirement 2
        • etc…..

Work Item Categories is new in VS2010 and as you see are used in MTM for query-based test suites. Work Items up till now had one name for example the Bug. In VS2010 the out of the box work item bug can be renamed. In MTM work item categories are used to restrict the type of work item to show in a given list. The categories are:

image 

I am sure there are lots of different ideas as to how to use Test Suites. I would love to hear them. please add comments with your Test Suite ideas.

 

Test Manager 2010 (MTM) - Test Plans and Suites

Microsoft Test Manager 2010 is now available, thank you Microsoft. MTM like Visual Studio connects to Team Foundation Server where all artefacts are stored. Like the bug, test cases are work items. What else does MTM have, lots. Lets look at the concept of a test plan first. You can have one test plan to many for a project. It is totally up to you the test team how you set up your testing effort on a project.

The Test Plan documents the test effort  thru:

  • Properties: test environment, settings, configurations, assigned build, and details like state.

image

Test Suites: organize test cases. There are three types of test suites; one is based on a requirement, another is a query of work item fields and the third is static. The possibilities are endless.

image

Microsoft has taken the test plan document to the next level. You can update a test plan as test efforts change. You can copy it as a starting point for the next project. You can if needed extract the information into a word document using an add on tool. If your printing for management or auditor put it on the network and direct people to it – save the tree’s. Plus, I bet most people do not even read them.

Look out world the Test Plan has evolved.  :-)