Never Test Alone – TASSQ presentation in TO

Andre King & I presented “Never Test Alone” at TASSQ in Toronto yesterday evening. After the presentation Joe Larizza (TASSQ President) addressed the group saying that with the economy businesses are looking for ways to reduce costs. Outsourcing testing will be one of them. People need to be proactive in finding ways to reduce cost and produce quicker. Joe encouraged people to talk to their management teams, present the concept outlines in “Never Test Alone” and volunteer to give it a try. See how it works, what it saves and in the end you may be saving more then you expect.

In Never Test Alone testers are involved in the Requirements & Design reviews and validations. This allows the test team to prepare for testing early, creating test cases, test scenario's and other documentation at the start of the project. Plus testers add value to the process from their experience and perspective. This reduces the cost of the test group weeding through requirements later to get an understanding that may not even be a correct one of the system. Enhancements that testers log during testing will come out during the requirement review phase. If deemed required can save money by being incorporated before development starts versus later.

Team up testers with developers during the creation of unit tests, the more tested at this stage the less cost in fixing, less bugs later, and we can reduce the duplication of testing being done between the two groups. Testers now know what has been tested therefore will not have to retest later. A testers perspective in what needs tested will differ from a developers so the combination of the two creates very rigorous unit tests.

If you are doing continuous builds (http://en.wikipedia.org/wiki/Build_automation) the unit tests are run against each build becoming regression tests. So we now have during unit testing, the functional testing and automated regression testing happening very early in the project.

Unit tests can be used in load testing. One example is login where UserID and Password are entered. The unit test can be build to retrieve from a listing of many UserID/Password combinations to execute login many times. What is the savings in knowing during development that the system is not going to have problems with 1,000 concurrent logins? Add this test to the build regression tests and you know throughout your project how well it is standing up to load. Why leave load testing to the end, do it early and save time, money and possibly the project.

Andre demonstrated a tool for automated white box testing, Microsoft Research PEX (http://research.microsoft.com/en-us/projects/Pex/). Very impressive and it can be run against code by testers saving development time for development. Check out the link above for a white paper on PEX. Maybe Microsoft will make a version of PEX that testing can use which does not require us to have access to the actual source code.

This is a list of some effects from following this presentation:

• testing a lot earlier
• saving money on bug management
• saving money on system changes late in the project
• reducing timelines
• building a team relationship
•  great feeling of accomplishment

And you are not testing alone, it is a group effort and it is the route to success.

 Link to TASSQ

Presentation Slides

Testa

Never Test Alone - KWSQA Conference

Bruce Johnson and I attended the KWSQA Conference yesterday in Waterloo. We teamed up doing a session at the conference. Our topic was "Never Test Alone - the Route to Success". Bruce wowed the crowd with a demo of his test-driven development skills and the Microsoft Research tool Pex (Automated Whitebox .Net testing tool). As we promised the presentation slides can be downloaded see link below. Congratulations to the KWSQA group for a great job.

Never Test Alone presentation slides 

Research Microsoft Pex

If we can be of help or answer any questions please don't hestitate to contact us.

Testa

CAST 2008 will be held in Toronto

There are not many conferences on Software Testing that are held here in Canada, well this year a big one is. For the first time ever CAST 2008 conference is being held right here in Toronto.  The show theme is Beyond the Boundaries: Interdisciplinary Approaches to Software Testing.

Mark you calendar for July 14-16th.

If your interested in speaking the Call for Papers deadline is February 4th, 2008.

To see more information on the conference click this link CAST 2008

VSTS Edition for Testers Presentation at TASSQ

Tuesday Jan. 29th, 2008 at the Toronto Association of Systems and Software Quality (TASSQ) meeting join myself and Barry Gervin for an evening presentation on VSTS Edition for Testers. Click the link for details about this presentation including a link for registration.

Hope to see you there ... Testa

Community for Software Testers @ Microsoft

Microsoft has launched a new community for software testers to share and learn about everything testing. There are and will be videos, articles, and blogs all dealing with software testing.

Join the Software Testing Discussion Forum and participate through out conversations.

Check it out -> Microsoft Tester Center 

VS Team System 2008 (Orcas) Web Test Correlation Helper

Sean Lumlev from Microsoft has done an excellent job explaining Web Test Correlation and how VS Team System 2008 (Orcas) new helper feature works.

Check out Sean’s blog by clicking this link Web Test Correlation and read all about it.

VS Team System 2008 (Orcas) Test-level Validation Rules

       VS Team System 2008 (Orcas) has improved validation rules.

Test-level validation rules allow you to add the same rule to every request by simply clicking the Web test node and selecting Add Validation Rule. A dialog box opens select the validation rule, in the properties you can edit or add options and parameters. Click Ok and you’re done.

Reasons for using a test-level rule:

·         Errors in the server application can cause incorrect flow causing the playback of the Web test to vary from the recorded sequence

·         Validate content that must appear on every page

·         Validate that all pages conform to a specific criterion

       VS Team System 2008 (Orcas) has improved validation rules.

Listed below are new validation rules:

·         Stop test on error

·         Search request and response

·         Add validation rule for title

·         Redirect validation

·         Provide test level validation rules

·         Expected HTTP code

·         Warning level for errors on dependents

VS Team System 2008 – Web Testing New Features

AJAX Requests & JavaScript Pop-ups

Currently VSTS 2005 requires an add-in tool called Fiddler to record AJAX requests. If you are not sure what AJAX is watch this video Measuring the Business Value of AJAX . As shown in the video using Fiddler we have the ability to capture AJAX requests, then using VSTS we are able to manage the Fiddler recordings in VSTS, see the requests in results and add the tests to our Load tests. If you watch the video the value AJAX adds is pretty impressive.

VSTS 2008 (ORCAS) now gives you a new feature, it captures AJAX requests with the web test recorder. 

Along with AJAX the web test recorder also is able to record JavaScript pop-ups.

Fiddler is no longer required – VSTS 2008 Web test recorder does the job.

New Features coming in Orcas - VS Team System 2008

VS Team System Performance and Load Testing - What is all the other terminology in comparison?

Surfing the internet can be mind boggling when searching industry terminology. Try searching terminology for testing that a solution is able to handle its data obligations while conducting itself in a suitable fashion. How do you like those words?

Terminology I found and have experienced:

Stress Testing

Load Testing

Performance Testing

Volume Testing

Concurrency Testing

Endurance Testing

Scalability Testing

Soak Testing

What does all this mean and why has Microsoft in Team System only used two of these?

I believe Microsoft in Team System has done the right thing. How many terms do we need to identify testing a solution can handle real life usage with peak times and large amounts of data?

VS Team System Performance testing is designed for developers to measure, evaluate, and target performance issues in their code. Profiling is used in performance testing; it is the process of observing and recording metrics about the behavior of solution code. Issues normally stem from code that performs slowly, or uses too much system memory.

Load testing is used to simulate many users accessing a server at the same time. In load testing, the web tests simulate multiple users opening simultaneous connections to a server and making multiple HTTP requests. Unit tests added to a load test can be used to test data access.

With all the features of the Load tool you are definitely stressing the solution, adding volume, testing concurrency, endurance, scalability and soak so why all these terms. With a combined performance test of developer code and load testing of the solution, we are testing the “Performance” of the solution in my opinion. Let’s keep it to the two terms so everyone understands what’s being tested and why.

Quote for today.

The ability to simplify means to eliminate the unnecessary so that the necessary may speak. (Hans Hofmann)

Have a great weekend … Testa J