Microsoft Completed Build Deletion – where has our test results gone?

Did you know that the files and data that are part of a completed build can delete?

Did you know that the data deleted cannot be recovered?

Did you know by default Test Results from any automated tests run against the build are deleted by default?

If you have the right permissions you can right-click in Build Explorer on a completed build and select delete. When you do that by default all the items associated with that completed build are deleted.

·         Details: Information about the completed build that is displayed in Build Explorer. This information includes build steps, requestor, and date and time queued.

·         Drop: File and folders output by the build and copied to the drop location.

·         Test Results: Results of any automated tests executed during the build process or results of any test published against this build.

·         Label: The version control marker associated with the specific file versions used by the build process.

·         Symbols: The debugging symbols published to a symbol server during the build.

You can also configure the retention policy and set auto deletion rules . Nothing wrong with that however is the person responsible for the “Build” setup and maintenance deleting could be deleting Test Results?

There is an option that can be set to stop the deletion of your Test Results. 

Make sure your teams understands what happens when deleting completed builds. Set that option to keep your test results, unless you don’t want them!

 

Testa Smile

Mobile Testing plugin to Visual Studio

A new Visual Studio plug in SeeTest for mobile testing. You can use Visual Studio (all versions) to test real physical phones including iOS, Android, Blackberry, Windows Mobile and Symbian. This includes Windows 8 phones. Just connect your device using a standard USB cable to your computer and record tests. SeeTest auto generates a script that you can copy & paste into Visual Studio. Then you can edit, run & view results in Visual Studio.

Check out Charles Sterling blog all about Visual Studio & SeeTest plugin. There is a video on Charles blog that you can watch.

Testa Smile

Agile, Scrum webcasts to join

Agile.org has some create webcasts on subjects concerning Agile, Scrum. You can either sign up to them or go to the archive listing. Couple of interest are:

Agile Practices in a Traditional Organization

Adopting Test-First Development

Release Duration and Enterprise Agility

On May 15th is Agile and Quality: It is not an Oxymoron but a Necessity

Click here to check out the public webcast series.

Testa Smile

ALM User Group in Toronto meets May 24th

It is Methodology May at the TALMUG

Methodology. In the world of software development there are not many words that raise contention quite as quickly as this. But why is that? What are the differences between Agile, Iterative, and Rigorous software development methodologies? There has been buzz about Scrum, XP, Lean, Waterfall, Kanban, and RUP for years; how do they fit into this discussion? But most importantly, why should you care? What does the test team think of all this?

In Methodology May the TALMUG brings you a panel of ALM professionals to discuss and debate these very questions and maybe help you see what methodology could work best at your company.

Pizza and Pop will be available at 5:30pm - Come out and join in on the discussions.

Being held at 40 University, Suite 1301, Toronto meeting starts at 6pm.

Follow on twitter @TOALMUG

Click here to Sign Up 

Testa Smile

Microsoft seeing work item history made easier

Dave Lloyd has updated GetHistory a power tool for displaying work item history. Now you can export the work item history to excel. If your company has a audit process that has to be followed this tool may help you report on changes to requirements, test cases, bugs in just a few steps.

Click Get History to get the newest version. Keep your eye out on this tool you never know what features Dave may add!

Testa Smile

Create Reports for TFS2010 Test Results

Check out instructions on how to create reports for your TFS2010 Test Results. There is also an example that you can follow.

Testa Smile

Microsoft Visual Studio 2010 Test Vs. QTP

Karthik K.K has posted on LinkedIn (click to see) a chart that compares the features of Visual Studio & Test Manager to Quick Test Professional (QTP).

I like the last item:

Visual Studio 2010 Test QTP Who’s Best
VSTS is cheaper and can be used for both development & testing. QTP is costlier and can be used ONLY for testing. VSTS

VSTS can also be used by Business Analyst, Project Managers, and Stakeholders. It can assist teams being Agile or Scrum or Waterfall thru a process template. The process template can be customized to meet your company need. VSTS reports on all aspects of a project and can tell you at any time where in the project you are at, the quality of the project to date, the status of your requirements/user stories. You can have a “Requirement to Test Matrix” in seconds at anytime.

If this alone has got your attention and you want to know more contact me directly.

Testa Smile

Toronto VS ALM User Group kick off meeting April 12th

The first Toronto VS ALM User Group kick off meeting is being held on April 12th @6:30pm sign up at TALMUG to get the details and register.

The User Group is for all roles within the Application Life Management team. Topics presented will vary from generic ALM practices to ALM with Visual Studio. If your involved in product management, a stakeholder, a business analysis or product owner, a developer or a tester this User Group is for you.

The goal of the first meeting will be to:

  • Define the group’s mission
  • Select an appropriate name
  • Document the roles of the executive
  • Recruit volunteers
  • Funding
  • Discuss how to reach out for sponsors
  • Discuss meeting locations
  • Discuss ideas for the first few meeting topics

Come out on April 12th and help us to kick-off this new user group code named TALMUG.

Testa Smile

Requirements Traceability in Visual Studio

Team Foundation Server (TFS) and Visual Studio(VS) excels when it comes to requirements traceability. Depending on what process you are using in TFS your stakeholders needs are documented in a work item called one of the following: requirements, user stories, use cases or backlog items. For this blog I am using the term requirement.

What is requirements traceability?

Continuous knowledge of the life of a requirement from conception to creation to design to development to verification to implementation and change. The ability to trace a requirements state and/or status at any time during a project.

First step in making requirements traceable.

In Visual Studio other work items are used to identify work that needs to be done to a requirement before it can be deemed done. There are different work item types depending on the process template you are using, however all contain the following: Task, Test Case, Shared Step, Bug. Additional work items can be added if not in your process template like Review, Issue, Change Request to name just a few. All these work items can be linked to the requirement they are helping to fulfill. Visual Studio 2010 introduced the concept of hierarchy by which a work item is linked to another work item using a linked type. The work item task is a child of a requirement, two requirements can be related, a test case tests a requirement. Work items like tasks may be linked to show predecessor or successor of another task. To learn more about choosing link types to effectively track your project click here MSDN Library.

Linked type listing:

image

Example of a requirement with linked work items:

image

What can be traced?

In Visual Studio during diagram modeling requirements and other work item types can be attached to objects in the model. During modeling you can either create or link requirements. This allows you to trace work items like requirements from a model diagram.

Example of a Use Case model with requirements linked:

image

Within the requirement work item I can see all other linked work items and information about them like state, assigned to. Knowing the state of work items linked to a requirement tells me the state and status of the actual requirement itself.

Development when checking in code to VS source control can be forced to associate what requirement or other work item their code relates too. Code check-ins create a Change Set that contains information about the check in and gets linked to the work item(s) associated during a check-in. When fixing bugs this is a nice feature.

In Visual Studio I can create queries to show any type of traceability report. Examples: Requirements to Tests Matrix, Requirements to Tasks, Requirements to Issues, Test Cases with Bugs to name just a few. Queries can also be exported or opened in excel, sent to someone thru email or opened in Microsoft Project. Queries exported to excel allows for adding additional excel reporting features like a Pivot Charts.

Requirements to Test Matrix – is there test cases for each requirement

image

Visual Studio comes with many reports that trace Requirements showing linked tasks and remaining work left, test case results state and many more scenario’s. Remember depending on what process template you are working with will determine the reports you see.

Example:

image

Checkout MSDN library for lots more information on Requirements Traceability in TFS & VS.

Testa Smile

Functional/System testing with Visual Studio/Test Manager

If your reading this blog you likely understand what functional testing and you may use the term system testing.

Wikipedia defines these terms as:

System testing of software or hardware is testing conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements. System testing falls within the scope of black box testing, and as such, should require no knowledge of the inner design of the code or logic”

Functional testing is a type of black box testing that bases its test cases on the specifications of the software component under test. Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered.”

If you have read Agile Software Engineering with Visual Studio (by Sam Guckenheimer & Neno Loje) you will have heard about “reducing waste”. Identified as tasks that reduce waste are functional and system testing. These two tasks can be done during code development through unit tests reducing the cost of bug fixes, bug analysis, creating a suite of automated tests and automated regressions tests and reducing the number of people involved in testing and the bug. In some teams developers and testers have been testing the same thing one through unit tests and then again later by a testers. This is duplication of work effort that is a expensive waste.

In Visual Studio there are unit testing tools and third party add in tools that developers can use to create very robust unit tests. You maybe thinking “the developers do not test the same “stuff” that testers do”. Your right, I agree. However since the team is encouraged to make changes to reduce waste why don’t we testers help them. In Visual Studio we can create test cases that are associated to the requirement/user story work item that describes what needs tested. We can pair up with developers to help them write robust unit tests to cover all the testing including boundary, error and data testing.

There are people that believe the future of the “software testers” is about to make a big change. Testers will need to be able to write and execute unit tests themselves therefore requiring the basics of coding and the ability to add assertions (validation) to the unit tests. (Check out MSDN’s Verifying Code by Using Unit Tests topics. ) I believe this will be a reality in the future but I also believe it will evolve.  If you want to start now pair up with your developers to help them create unit tests that execute both the “happy path” and boundaries of an individual method, class or component. Help them to create system integration tests. Getting expose to how unit tests are designed and coded will help you move into the future. In addition having knowledge of what has been unit tested reduces test duplication later. TFS and Visual Studio help us with all this through work item traceability.

Example of work item traceability:

image

Example of a Test Case and Associated Automation:

imageimage

Visual Studio has the tools that will help us move into the future with confidence and the security we’ll need. Humans in general are not adapt to change but change we must. I am one of those people that embraces change and excels in change but then I have had Visual Studio in my pocket!

- Kent Beck

The role of professional testing will inevitably change from “adult supervision” to something more closely resembling an amplifier for the communication between those who generally have a feeling for what the system must do and those who will make it do.

Kent Beck author of Test-Driven Development (Addison Wesley 2002), 86.

Visual Studio – my companion, my mentor, my stability, my aid, my reporter, my success

Testa Smile 

(stay tuned, next blog  I will show you how easy it is to create a unit test!)