Microsoft Test Manager– What is my test state at the Test Plan level?

Check out the Visual Studio Team Test blog on using Excel reports with MTM to identify test state at a test plan level. This is a step by step on how to create the report in excel using information on your test state from Test Manager.

Testa

Microsoft Test Manager (MTM) – action recordings

MTM 2010 has opened up the world of manual testing with data-driven and fast forward capabilities. Plus, they have taken extra steps to ensure your successful here is an example of one.

Your are opening an application from a short cut on your desktop. Test Manager will capture and store the path to where you have placed the application executable and the path based on the %USERPROFILE%. In my example it looks like this:

 Launch '%USERPROFILE%\Documents\Visual Studio 2010\Projects\Calculator\Calculator\bin\Debug\Calculator.exe'

In this example if I remove the executable from my desktop and pin it to my start menu the test will execute successfully. If I then remove it from my start menu and pin it to my task bar the test again executes successfully. In addition the test actions is opening the executable not following the human interaction of double-clicking. This makes your test even less subset able to failure. Note that these types of failures are not application issues and therefore we’d really like to not have to deal with them.

Thanks to Microsoft there is less chance of this slowing our testing down due to setup issues.

Testa Smile

Microsoft Test Manager, Coded UI (CUIT), Web Performance & Load, Bugs, using TFS2010 – 3 day course in Toronto 10% off

If you have never been to Toronto here is your chance plus while here you can learn about Microsoft Test Manager, Coded UI testing, Web & Load testing, the new Bug, Fast Forwarded Manual Testing, and all you need to know about TFS2010 the communicator. Here is your chance. Check out our 3 day course offering being held March 9th to 11th and get 10% off.

Contact: trainging@newhorizon.ca 

Tell them Testa send you …

Microsoft Test Manager–test case multi steps tool

Testers have been asking for multi-step capability in test cases. Shai Raiten has created a tool that enables test case step multi lines. The Test Steps Editor enables editing of actions and expected results to add multi-lines. The tool works with both Shared Steps and Test Cases.

Here is an example of the tool:

You need to know your work item id, enter then click Load, the test case steps displays.

Put your cursor at the end of the line above where you want to add a line.

Alt+Enter adds a line below the selected line.

image

image

Click to download the Test Steps Editor.

Thanks to Shai,

Testa

Microsoft Test Manager – hot fix available

On Oct. 12th Microsoft has published a hot fix for issues that were causing Test Manager to crash and memory leaks that caused data and productivity loss. After downloading the hot fix I noticed that in the Test Case iterations section that are new buttons and bugs fixed. See what I’ve found so far below:

1. You can now delete test case iterations without crashing MTM. And working with data in the iterations seems smoother.

2. You will now have a “Remove parameter” button which allows you to remove a parameter from you iterations and/or test steps.

image

3. You will also have a “Rename parameter” button which allows you to rename a parameter in your test case iterations and/or shared steps.

image

4. Last you will have a “Delete parameter” button which allows you to delete a parameter in your test case iterations and/or shared steps. Deletions with this button only removes the @ sign from the parameter in the steps. You can still highlight the parameter in the step and click the delete button on your keyboard, this removes all instances of the parameter in your test case.

Note: shared step parameters display in associated test cases however, you can only Rename, Remove or Delete parameters from within the shared step itself. At the moment you are able to select the shared step parameter and proceed with selecting the test case remove, delete or rename button and proceed again as if executing the button feature right to the end. The action does not actually work!

Note: the issue with deleting a parameter in a shared step and the associated test cases not being updated still exists. This hot fix does not include a fix for this.

Click this link: hot fix KB2387011  to download.

Microsoft Test Manager: How to manage test case/test configuration assignments …

In my last blog there where questions about how to assign or unassigned  test cases to test configurations (browsers, operating systems).

How to manage test case/test configuration assignment:

    • In the Plan Tab select a test suite then test case.
    • In the Test Suite panel toolbar click on the Configurations button.
    • The Select Test Configurations window opens where you can add and remove test configurations associated to a test case.
    • Simply click button All configurations, all test configurations you have created are added to the test case.
    • Next click in the test configuration cell to add or remove.
    • Last, click the Apply Changes button.

image

Whether you add test configurations before or after setting up Test Suites and Test Cases you can manage test case assignment by using the Test Suite | Test Case, Configurations button.

Note: Test Suite/Test Cases are automatically assigned to all test configurations created prior to there own existence.However, if you create additional test configurations after creating Test Suites/Test Cases they are not automatically assigned. In this case you can assign using the above steps.

Thanks to Jeff Levinson (Team System, MVP) for helping out with this question.

Testa

Microsoft Test Manager – Test Configurations & Test Cases

Duplication of tests for configurations does have a flow that you need to be aware of …

  • Test Configurations have to be set to active and selected in the Test Plan properties.
  • Test Configurations set up prior to creating test cases are automatically duplicated for each configuration assigned to the test plan.
    • Note: you have no ability to indicate which configurations affect which test cases. Therefore you will have to use the test case run status of “blocked”. If you leave “active” a tester would be able to run.

    image

  • Adding a new test configurations to an existing Test Plan will not affect any existing Test Suites. ( I believe this is a bug and have reported it to Microsoft) However, if I add a new test suite and associate existing or new test cases to that suite all the Test Plan test configurations are used.

When I get an answer on whether this the correct behaviour or if there is a work around will post it.

Testa

  •  

Microsoft Test Manager – Test Plan Configurations

If you need to test against different test configurations you are going to love Microsoft Test Manager.

Test Configurations are different setups that any application requires testing against. Example: Microsoft Test Manager would have been tested against different operating systems like XP, Vista, or Windows7. A web application may require testing against different browsers like IE7, IE8, Fire Fox, Chrome or Safari.

The ability to set up test configurations is only limited to your needs. One of the best features is the ability of Microsoft Test Manager to simulate a duplication of test cases for test configurations assigned to a test plan for test runs only.

It does require management to some extend however;

  • it does save your team time in creating duplicate test cases or excel spreadsheets of what configurations will or have been tested.
  • it is only really one instance of the test case so an changes affect all instances in the test run.
  • the test plan documents what configurations will been tested and by which test cases.

There is significant savings to the test team for this feature to be one of the top.

Test Plan – test case shows once only.

image

Test tab >Run Tests – test case is shown for each test configuration set up in our test plan.

image

Over the next few blogs I will tell you more about Test Configurations and how they affect test cases and test runs.

Testa

Microsoft Test Manager – bugs being fixed in future update

When attempting to paste (using Ctrl-V) data from the clipboard into a parameter within an Iteration you must use the spacebar or manually click into the field that you are populating. If you don't have a cursor blinking even thought the box is highlighted (has focus) using Ctrl-V creates another Iteration instead of pasting the copy data.

This bug has been report to Microsoft and should be address in a future update.

___________________________________________________________________________________

Created a Shared Step that has 37 parameters. We added the Shared Step to a Test Case and loaded  the data for one iteration. We then decided to remove two parameters. In the Shared Steps we deleted the two parameters and removed the steps. Saved and Closed. We then opened the Test Case expected the two parameters to be gone, they are not.

We have also had issues with changing the data in the parameters (Test Case)

1 remove the data then click the enter key

2. add the new data then tab out

3. save.

Work a Round's:

1. Deletion of parameter in shared step does not delete parameters in the test case.

There is definitely a bug with refresh here. I will file this issue. To workaround this, insert a dummy step and delete it. You will see that parameters go away. Let me know if this works for you.

2. Deletion of data

I agree, we do not have a good experience of editing data. You will have to hit the enter key to take the test case to dirty state. Only after that the data gets saved.

Testa

Microsoft Test Manager – adding parameters to Shared Steps

Shared steps can have parameters however, before adding iterations of data there are considerations you need to take into account.

If you consider logically how this works, adding a bunch of parameters and data to a Shared Step will have that step repeat running through the data you added. Then once all iterations of the shared step complete your next test case step would run. Make sense.

  • What if I want the Shared Step parameters to run with other parameters in my Test Case?
  • What if I want to add data to my Test Case Shared Step parameters months later?

The cleanest and most flexible way I believe and have found is to add data (iterations) to my Shared Step parameters from within the Test Case. Remember a Shared Step can be used in any Test Case and therefore needs to be flexible.

Example:

A Shared Step with Parameters > no data (iteration) added.

image

Shared Step with Parameters (userid & password) inside the Test Case

image

Above there are three parameters of which two belong to the Shared Step, the other the test case. You can add many parameter iterations to this test case without affecting any other test case that contains the same shared step. (If the UserID and Pswd’s were in the shared step that step would keep opening and login for each iteration before doing the test case step two.)

I have seen cases where testing always uses the same UserID and Pswd for login. In that case having these as parameters set within a Shared Step makes sense. If a new set of credentials is issued you can easily change them in one place, quick and simple.

Note: now this is confusing. I set my parameters in a Shared Step added the Shared Step to a Test Case. In the Test Case the shared step parameters display however they do not show any data.

[I will try a scenario where parameters are set in the Shared Step and in the Test Case to see what happens!  - will let you know the outcome]

Comments on this subject are welcome. If you have found a different way to handle a combination of parameters in shared steps and test cases or situations you have come across  – lets learn together.

Testa :-)