July, 2008

Software Testing has made his life tough but secured

3 Years back there was no QA department in the organization. The development manager, John, was the king. Nobody was there to check at what stage his development projects were, all used to go by his words. One day he was asked – hey, what is the status of your software that you are developing for Canada? Oh Sir, he reported his boss, it is all ready. When do you want to send it to Canada with my implementation team? Ok, if it is ready, let us book tickets for coming weekend so that your people land there, settle down in the hotel, enjoy weekend, and start implementation on Monday – his boss told him. Ok Sir, he replied.

John called his development team responsible for Canada project, takes stocks of situation, gets to know that still many screens, sub-modules, units are under development and at no cost finish by weekend. Oh, but I have told boss that everything is ready – he tells his team. Don’t worry sir, his team tells him, we will finish it there. Ok, he replied.

So the team goes to Canada, the target of finishing the project is not met, users are not happy with the product (it was not a complete product), management got angry, stay is increased, but even with the extended stay, the project does not take proper shape and hence finally the team is sent back.

What now! An emergency meeting is called. Reasons for project failures are found out. It comes out that John had projected a rosy picture to the management without showing the existing thorns in it.

And the QA department took birth with a resolution that no software will move out to any country without a final clearance from QA.

3 Years have passed, John’s life got miserable in the beginning. Each of his products had to undergo 5, 6, 7… cycles of testing and re-testing by QA. Initially QA reported why the product has come for testing when it is not complete? Why it is declared as complete? Why during testing, the development team keep doing something or the other in the same product instead of giving time to QA etc. etc.

Ultimately maturity arrived. The products only are shipped when they are sensibly completed and verified/tested.

The Development Manager finally says to QA Head – You exposed me, no problem, your team tortured me (my testing and re-testing), no problem, ultimately I am the happiest man now because now when I declare about a product that it is complete, and my boss stares at me, I add immediately, and it has been cleared by QA after x number of testings finally. I am sure it will not fail now as usual.

John’s life has become tough but he is the happiest man on this earth, you know why, because now he is sure that none of his projects will overrun or fail.

Why Software Developers are not Doctors?

Software Developers are not doctors and that is why are not in the habit of doing ‘first time right’ coding. All newly developed software contains bugs, bugs and bugs. Developers have this mindset that no software can be written without a bug in it. They are convinced that bugs are integrated part of coding and involuntarily that are bound to place them here and there in their code while writing a software. If doctors start working like this that in every operation they leave some important activity ‘undone’ and have a privilege of getting each operation audited by another doctor (specially meant for auditing a patient immediately after operation), the auditing doctor will find out the shortcomings in the operation, report it to the doctor who operated upon the patient, the patient gets operated again, again auditing, again operation, so on and so forth. Think of what will happen to the patient in this case. And I think the patient will prefer to die on his own than in this manner.

The same thing happens to the health of software when some bugs are embedded in it, found out by testers, reported to developers. The developers fix those bugs, in turn produce some other bugs, again it comes to testers. Another bug report, another fixing, another different set of bugs production, so on and so forth. What happens to the health of the coding in this case?

If software could shout and cry!

A point to think upon and act upon!

A Poultry Farm and Software

To me sometimes it appears, may appear funny to some, and logical to others, that there is a strong resemblance in a Poultry Farm and Software. As different Hen in a poultry farm keep producing eggs day in and day out similarly different modules in the software keep producing bugs.

Do we agree then that as hatching eggs produce more hens, similarly fixing of a bug produces more bugs?

Eggs which are not hatched but consumed may then be related to bugs which are there in the system but have not been identified and remain floating in the system. As is known that many bugs cross testing phase without being identified and come into light during live run (i.e. post implementation phase), sometimes after months of successful completion of a project.

Do we require such software – productive (producing bugs) like a poultry farm? Or we need good programmers to make good software and not turn it into a poultry farm.

I invite your valuable Feedback/Comments please.

Equivalence Partitioning (or Class) Testing Method

Equivalence Partitioning Testing Method is a method used in Black Box Testing (see: http://itknowledgeexchange.techtarget.co…) for the purpose of finding out minimum set of data for testing for valid and invalid inputs.

The purpose of this testing is to:
1. Ensure that the software acts according to the type of data is entered, i.e. for a valid entry it should accept the data and for an invalid entry it should return the appropriate message/alert.
2. To ascertain that minimum data is used for similar results (the result of a test case could be valid or invalid)

For a group of values, if the software behaves in same manner, the group or set of value is called ‘equivalence class’ or ‘equivalence partition’. So primarily for an equivalence class the software will behave in the same manner and shall produce the identical result. Different equivalence classes will make the software behave differently and shall produce different result.

The tester has to be careful in carving out the different equivalence classes so that one value is taken from each class to ensure complete testing. The testing in this way is completed with least efforts/ test cases.

Let us take an example to illustrate:
In a particular Airline based on number of flights taken in a year, a membership is offered with different privileges as below:
If the number of flights taken is between 5 and 10, the membership offered is ‘Silver’
If the number of flights taken is between 11 and 15, the membership offered is ‘Gold’
If the number of flights taken is between 16 and 20, the membership offered is ‘Platinum’
If the number of flights taken is between 21 and 40, the membership offered is ‘Diamond’

Based on above all inputs between 5 and 40 are valid, less than 5 and more than 40 are invalid. And also any non-numeric value is also invalid.

So there will be 7 equivalence classes viz: 5-10, 11-15, 16-20, 21-40, 40 and non numeric and thus one value from each equivalence partition will give you the complete test coverage.

What is Black Box Testing

Black Box testing of a software is based on two primary things –
1. Customer Requirements Document
2. Functionality of the software

Purpose of the Black Box Testing is to ensure, that means, that –
1. The software meets are customer requirements as specified in the Customer Requirements Document (business needs)
2. And the software meets user requirements (user needs)
By user requirements we don’t mean that it has to meet every user’s requirements but more appropriately the appropriate usage requirements.

Black Box Testing is the most widely used testing and is used to used to check if the product meets per se customer’s all requirements. The tester is not worried about how the software has been built, what queries have been written, whether the coding is technically optimal or not, but focuses more as if the customer is working on the system and feels like customer when he tests the software.

A basic requirement of Black Box Testing is that nothing should be presumed about the functionality of the software while testing the software, rather the tester is supposed to check each and every functionality of a unit, module and the product as specified in the customer requirement document. So the fundamental of this testing is the Customer Requirement Document, based on which the appropriate test plan, test cases and test scenarios are built and ultimately test results are drawn as a Test Report (or Bugs Report).

For building various test cases in Black Box Testing certain techniques are used like:
Boundary Value Analysis (or Boundary Value Testing), see my earlier blog on this:
http://itknowledgeexchange.techtarget.com/quality-assurance/bvt-or-bva-%e2%80%93-boundary-value-testing-or-analysis/
Equivalence Partitioning: This is a technique of testing with minimum of test classes or data input to identify between valid and invalid input.

Prior to starting actual testing, a tester must verify and validate:
1. Customer Requirements
2. Functional Specifications of the software
3. Test Plan
4. Test Cases
5. Test Data
6. Test case execution

After above validation, the test cases are executed, results are recorded and reported to development team.

How Software Testing took birth!

Way back in 1960s softwares were being produced but there were no established standards or software engineering practices. Poor design and Regular hardware upgradation left many types of software useless and thus a strong need arose in the industry to provide effective and reliable software developers who could pace up with regular hardware advancement to produce faster and highly reliable softwares.

All this, plus the increasing dependency on softwares increased not only demand of good developers, but also a requirement of software testing took birth, may be, not initially in a very strong and structured manner but it did.

Software engineering practices that were taking place at that time were not formulated, or structured in an efficient manner. Some of the bad practices were no measurements or metrics were in shape to measure up -
Development lifecycle or product lifecycle
Lack of planning, monitoring
No control on development schedules
No control on customer requirements changes
No documentation
No reviews of documents
Failures at customer end during or post implementation due to product incapabilities, schedule delays, revenue losses

Therefore a need to test the software came into existence to:
enhance all these processes,
increase the reliability of product,
deliver a quality product,
deliver in time

BVT or BVA – Boundary Value Testing or Analysis

Let us take an example that a code is to be written for entering employee details in employee master table. There are certain conditions for validating while entering a new employee for a specific post that his minimum experience at the time of joining has to be 5 years but not more than 10 years. The coder has written the code and designed the screen to input data satisfying these conditions. Now this code comes to tester for functional testing to verify if it meets the desired requirements.

How tester is going to confirm it?
A tester has to write test cases by understanding the following:
1. Boundary
2. Valid entries
3. Invalid entries
4. Inside edges of the boundary
5. Outside edges of the boundary
6. Boundary points

Let us see what each one is and why it is important as the test cases are going to be built on this basis:
1. Boundary: The boundary as per requirement here is 5 to 10 including both values.
2. Valid Entries: The valid entries shall be all values between 5 and 10.
3. Invalid entries: All values other than in point no. 2 are invalid
4. Inside edges of the boundary: Inside edges of the boundary in this case are 6 and 9
5. Outside edges of the boundary: Outside edges will be 4 and 11
6. Boundary points: Boundary points shall be 5, 6, 7, 8, 9 and 10.

So the test input data shall be: 4, 5, 6 to 9, 10, 11

Definition of BVA or BVT: Boundary Value Analysis is the process of selecting test cases (or test data) by understanding boundaries that differentiate between valid and invalid conditions. Tests are run to check the inside and outside edges of these boundaries, in addition to the actual boundary points.

What is SRS?

During Software Development certain business rules and disciplines need to be embedded in the software based on the customer and business requirements as specified by customer during the initial business study. These are required so that the software meets customer expectations and needs. A Document is prepared based on this needs which is known as SRS (Software Requirements Specifications).

To prepare SRS a group of professional from the organization who is responsible for this software development, visit the customer site to have a first hand look at their processes, documents, functions and requirements. SRS is prepared as detailed and explanatory as possible because this document is handed over to the development team who will be responsible for developing a good software matching customer needs and meeting customer requirements.

SRS in a way is the foundation stone for good software that has to take birth based on it. It is very important to meet the right people at customer location who are deeply involved in these processes in their respective fields. A detailed study of each process, supported by relevant documents, forms, reports need to be collected. Process owner must devote ample time to the team member who has come for business study so that he in turns documents it well as part of SRS.

Finally each Team member compiles, edits, verifies and prepare a document that is called SRS.

Progressive Software Testing Approach by acquiring Soft Skills - Step by Step

In Software Testing we need to devise an approach that features a gradual progression from the simplest criteria of testing, to more sophisticated criterion via many planned and structured steps, each of which brings incremental benefits to the Project as a whole. That way, as a tester masters one skill or area of testing, there will always be something worthwhile to try next.

Important Aspects are:
Trust: Testing and development teams have to build a trust-trust relationship in each other and in the product under development.
Constructive Communication: The initial communication of testing team with the Development team has to be quite open, transparent, constructive and encouraging. This is the phase that decides about the course of action for testing and any alternatives. The future of software, its accuracy and the successful implementation at customer site afterwards will be the key factors emerging out of it. Don’t make this communication too formal.
Documentation: Ensure that testers have sufficient documents in hand before starting the analysis and testing of the product. Appropriate analysis of the documents is very important to remove and ambiguity/confusion/loop-hole in understanding the customer requirements. Test cases and scenarios will emerge out of these requirements.
Open Mindedness: A tester has to go with open mind in all discussions only then he will be able to understand developers, development process, customer requirements and then he will plan on the testing requirements accordingly.
Variables: Keep all variables in the loop that are important to the testing and successful product development. Different variable could be developers working on different modules, documents, testers etc.
Learn from Past: Keep learning from past experience and try not getting caught in any mistakes that have been made earlier. A common saying is – “Never repeat the same mistake again”. It is a mistake to go for testing of a product in a crude or unstructured manner. It is a way of improvising your testing model over a period of time with more and more experience.
Indentify Shortfalls in advance: Be clear about what is in scope of testing and what is not. Clearly define what is not going to be covered in testing and what could be the impact of them on the product. Foresee and uncertainties/delays. It is important to understand the benefits you are going to get well in advance rather than getting no benefits at the end of this whole exercise.
Keep Interacting: A tester has to keep interacting with the developer throughout the testing process. Ultimately the goal of both sides is to have a better product at the end crossing all barriers of conflicts, bugs, faults or defects.

Your valuable comments are most welcome.
Thanks
Jaideep
 jaideepkhanduja at yahoo.co.in

Linear Approach of Cost Estimation of Bug fixing for various Software Projects

Assume that after functional testing of a software following is the summary of count of bugs:
Total Bugs : Tn
Severe Bugs : Sn
Critical Bugs : Cn
Desirable : Dn

Here Sn + Cn + Dn = Tn

And let us assume that the estimated expenses of removing of those bugs (based on developer’s direct and indirect cost) are:
Average Cost of removing a Severe Bug : $S
Average cost of removing a Critical Bug : $C
Average cost of removing a Desirable Bug: $D
Then we can say that the Average cost of removing a bug (Severe/ Critical/ Desirable) : ($S + $C + $D)/3 = let us say $B

Here Total cost ($T) of removing all bugs is: $T = ($S * Sn) + ($C * Cn) + ($D * Dn)

This will be for one project and similar data van be obtained for various other software projects under development in the organization.

The graphical or tabular comparison of Tn * $B to $T for all these projects is the linear approach of Cost Estimation of Bug Fixing or Defect Removal.