September, 2008

Role of tester while verifying the closure of bugs

When a product gets completed, it is released for testing to QC department along with the relevant documents. The QC department based on the scope of testing, availability of testers and the time at which the product is estimated to be released to customer, prepares its test plan. After studying the business and customer requirements, test cases and test scenarios are built by testers, based on which bugs or defects are reported. Once the bugs report (or defects report) is released to development team by the testing team, it is the development team that comes into the action. They based on the category, validity of a bug divide among themselves the bugs to be fixed and inform the testing team the estimated time required to close or address all the bugs/ defects. Once all the defects are fixed, the product goes back to testing team, for verification of closure of bugs.

The question arises here is that what is now the scope of testers for a re-released product. Does it suffice the purpose if testers just validate the closure of bugs? No, it is never going to be a fool-proofed product. What about the complete re-testing of the product for the two main purposes:

This is very crucial phase and the testing in this phase needs to be more exhaustive and extensive than the earlier round of testing.

What next after Bug Report (or Defect report) is released?

As soon as a bug of defect report is released by Quality department to Development Team, the first task of the development team is to call for a meeting inviting all those involved in the development of the product, the project head, the quality head and the testers who have performed testing. Prior to this meeting, it is good if the development team just have a go through of the list submitted to them. The purpose of this meeting is – to clarify if there is any understanding gap regarding a bug reported, to assign the tasks to developers (who is going to fix the bugs), the time frame. And at the same time mark the bugs that need simulation, clarification, need not be fixed, duplicate bugs reported, false bugs, incorrect bugs (the design is meeting the customer requirements but falsely reported as a bug by testers).

Spending time in this meeting with most of the people required is going to be very fruitful. This meeting places all involved on the same mindframe.

Immediately after the meeting (or during this meeting, if possible), the development team should fill in the estimate time required for fixing each bug (time required is to be filled against each bug). Then arrive at a conclusion when the product tentatively is going to be handed over to the test team again for verification of closure of bugs, or finding out any new bugs.

Is your defect (or bugs) report serving its purpose?

Let us understand what is the purpose of bugs report or defects report? Is it serving its purpose? How much?

The purpose of a bugs report or defects report is to make the intended recipients of this report to understand each and every bug reported in its perspective view. A bugs report has to be quite elaborative in explaining the developers if any business or customer requirement has been failed to meet or has been built wrongly.

Logically defects or bugs report is a report card of testing process. This is the only way to find out how extensively and perfectly testing has been performed. How well have the business rules and customer requirements been understood by testers is reflected by the test cases performed. It also reflects how well Business requirements or customer requirements been documented.

The bugs or defects report not only reflects the total health of the system built but also shows how perfectly has been the testing performed, how well the requirements have been documented and how well the documented requirements been built into the system.

The quality of the product fails its purpose if the testing is not performed well. On the other hand a good testing of a bad system will not be of much use in ensuring a perfect system meeting customer’s requirements.

The bugs should be written so elaborative that they are quite clear to the development team to understand and act on it. These bug reports can be referred to any time in the future too. The more is the clarity or simulation required by the development team from testing team, the less is the purpose of bugs report being served. All it indicates is that the report needs improvement.

Components of a Bugs or Defects Report

Based on the above facts, the bugs or defects report’s content has to be very effective in conveying the intent of the testers performing the testing.

The components of an effective bugs report would be:
Reference Number: Give a unique number for referring to it any time in the future. Treat it like a ticket number generated against a complaint.
Reference Documents: Mention all the documents being referred to, for building your test cases, scenarios and test plan.
Scope of Testing
Initiated by: Write the name of the person who has initiated the test process
Initiated on (date)
Test Setup prepared by: Name of the person
Test setup prepared on (date)
Handed over by: Name of the person who handed over the product and documents to test team
Handed over on (date)
Schedule of testing (from what date to what date)
Test team composition: Persons involved in testing and their roles
Test Plan
Test cases/ Scenarios
Testing started on
Testing completed on
Reviewed by
Reviewed on
Summary of bugs: Total bugs, bugs count category wise (like show stopper, severe, critical, desirable etc.)
List of bugs with their reference to test cases/ scenarios (against one test case there could be no. of bugs): the list would have the columns – S.No., Module, Sub Module, Description, Screenshot reference no., test case reference no., Category, Status (to be filled later by development team once they start acting on bugs), Remarks.

What is a bugs report in software testing?

A bugs report or defects report is a list of bugs found out by testers while testing a software product in testing phase under a testing environment. The test environment is created at development site similar to the actual environment in which the software is supposed to work or run in live situation at customer site. Environment built for testing is supposed to be an independent entity meant only for testing team to perform testing without any development changes happening on it.

Bugs or defects report is the most vital tool for developers to understand where the product that has been built by them is lacking its functionality or performance. Testers have to be very careful in studying business and customer requirements while building their test cases and test plan. The quality of test cases will clearly enhance the core performance and functionality of the product built.

5 essentials while building test environment for software testing

What is a test environment? – I have already explained in my earlier blog  http://itknowledgeexchange.techtarget.co…). Also I stated in my previous blog – “why build a separate test environment”  http://itknowledgeexchange.techtarget.co…) from which two major parties being benefitted are customer and quality testing team of the vendor. Here let us try to understand five essentials to be adhered to while building a test environment for software testing. Let us also understand that these are not the only essentials, there could be many more, I am taking the five top priority factors of them.

1. Customer’s Environments: Understand clearly the environment in which the customer is going to run this software. This has to be checked not only for server but also for the user’s machines. The environments factors could be the hardware, OS, Database, Front end tools, browsers etc. Take care of all the versions of OS, browser the customer machines that are going to run this application.
2. Test Server: Build your test environment as much as possible a replica of customer environment. This is to be applicable to Server and client machines as well.
3. Separate Test Server: Build the test environment on a separate server free from development and dedicated exclusively for testing purposes.
4. Understand business requirements well: The testers and test lead should be very much clear about the customer requirements based on which the test cases are to be built. More understanding, more coverage. Much clearer understanding, wider coverage.
5. Documentation: Aesthetically document each and every test that the testers perform for a unit, module or integration testing of the product.

Why build a test environment for software testing?

In continuation to my previous blog – “what is a test environment?”, let us understand here why a test environment is required for testing a software product? I see two major purposes for this, one purpose catering to customer end, and the second purpose catering to the vendor end quality testing team.

The vendor or supplier who is preparing or supplying a software product to its customer must be very clear that in what environment his customer is planning to run this product. The environment could be the hard side – i.e. the hardware configuration comprising of hardware server and users PC components viz – hard disk size, its speed, RAM – size and speed, Processor, bus speed etc. in the soft side it will comprise of the Server/ Users PCs Operating System and its version, OS Patches requirements, Browser – which all browsers are supported and which versions, IIS server specifications at server end, Database Server – What database and which version or release. Any other software components required at server or users computers. The purpose is to run this software as if it is being used by the client at his site. In a way it is the simulation of client site.

The second purpose to have the test environment separate to development environment is to restrict developer’s intervention in test environment during the test phase. This is to be done so that the test results are clear, real and non-ambiguous.

Any other reason coming to the mind while reading this blog that comes to my readers – is most welcome in the comments section.

What is a Testing environment for software testing?

A testing environment is a setup of software and hardware on which the testing team is going to perform the testing of the newly built software product. This setup consists of the physical setup which includes hardware, and logical setup that includes Server Operating system, client operating system, database server, front end running environment, browser (if web application), IIS (version on server side) or any other software components required to run this software product. This testing setup is to be built on both the ends – i.e. the server and client.

I remember a case where an application was built strictly as per customer requirements by a software vendor without understanding or taking it seriously that what database version the client is intends to use for this software. The vendor developed the application on Microsoft SQL Server x version and the client purchased the new server with the next version of MS SQL. When the product was completed at vendor side, he did a fantastic testing and made the software functionally very strong meeting customer’s all business needs. And when the product was launched at the customer location on the next version of OS and database, the product misbehaved (or did not perform) in few of the instances for which the technical team had to be sent to the customer site to fix the gaps and make it run hassle free. Now this not only affected the product cost but delayed the project deadlines in a major way. This small ignorance at both client and vendor end made a recursive effect at the vendors profits and next few projects.

The production or development team when completes a product development at their end, have to produce this test environment for testing team prior to loading their newly developed product on that environment for testing.

Why Project Manager is sceptical to testers, if at all he is!

A Project Manager is mostly sceptical to the testers that they unnecessarily try to increase the bugs list just to keep quality department on top. He also feels that it is intentional and many bugs can be avoided to be mentioned which are not very important. Many Project managers in general also feel that quality department is not necessary to exist in an organization. The quality can be maintained by hiring good developers, is what they feel. Then why a separate battalion of testers is required in most of the software organizations across the globe.

Many factors, I would say are responsible for this. Customer getting more cautious towards quality and delivery of product, timelines, competition, increase in internal and external expectations, benchmarks, awareness, prospects, continuous improvement, survival of the fittest, revenue growth and timely returns are few of the top most of them.

There is another side to this coin – A project manager’s skepticism initially may be due to a fear that his team’s development weaknesses will come into light. One more weak area could be the incomplete or inaccurate documentation or study of customer requirements. But with positive signals from Testing Team, Project manager has to gradually understand and admit that all the efforts being done by testing team is not to highlight the weaknesses of his team or product, but to strengthen his team and product.

Birth of Quality with a need to wear multiple Hats!

The switch over or additional responsibility may arise out of different scenarios. Small software organizations initially have no quality assurance department, but with their gradual growth and to meet project commitments, the quality assurance emerges as a separate department.

At this juncture the organization may not be ready to appoint somebody from outside as quality lead, due to various reasons. The reasons could be that the new person will take his own time to settle down in the organization and a couple of months to absorb organization culture. Other reasons could be since this is an experimental start of a new department for which organization may itself be unclear about the department size and its growth.

A new person may refuse to join the organization as a quality lead with too less people in the team. The requirement initially could be too small that one or two persons are required to start the new department. And most important factor is that the internal person working in say production, development or implementation will understand the gravity of emergence of the newly formed quality assurance department and hence will be able to move the department in the serious direction management expects it to.