On his blog by the famous Leader on Project Management Bas de Baar, he asked a question to all his blog readers – “What is the best advice you can give to project managers supervising offshore development projects?” Various readers posted their opinion as per their perception, experience and learning. His focus was more on Offshore development projects, whereas I would like to point to any offshore project that has to be developed implemented on site. So in context to offshore development and implementation of a project, a project manager has to essentially develop some soft skills other than his/her standard project management tools and methodologies. According to me for an offshore project management following would be the essential key requirements as I am sure every one of you will agree that an offshore project is definitely different to an onshore project. The basic things to be taken care of in context of that country, management, users shall be:
Language: Understand what their prime language is in day-to-day use. I went to a country where language was a big barrier and I was heading the project. Before going there, I learned few of the commonly used words, downloaded a dictionary to translate from English to their language. Although very few of my users were fluent in English, and I used them extensively as a medium to translate what I used to say in my training sessions. But there were instances when I am sitting with a couple of users who don’t know English or who know very little of English, then this offline/online dictionary was very helpful. Also I learned before hand that they speak English very slowly and can understand only if spoken in low speed and a different accent (like “show” will be pronounced as “cho”). Ascent was another criterion that I had to gradually master upon. This will also depend on deciding upon the pace of the project. I had to change my plan accordingly to cover up the backlog at times.
Culture: They will love if you talk about their country and culture, and that is the best way to make them your best friends. Then it was easier to understand them faster. Learn about their likings and disliking, what they don’t like is very important, or what annoys them easily, so avoid doing that or using those words. Don’t refuse to attend any of their customs when invited. After all it is the people that make any project success. And most of the people outside will be interesting to know your language or culture, do not hesitate in sharing your knowledge/skills and spending some time on this, although it is not the part of your project but it will develop into a very strong tool to progress your project in right direction with right pace.
Be as them: Don’t act as a foreigner. Act as one of them. Show the management that you are the part of their organization and it is not only their but your project also. Keep top management involved in all your decisions and actions. Seek their help where required.
Celebrate: Celebrate any small success that will motivate them and speed up the implementation.
Do let me know anything that I might have missed to mention.
Roberts: Project Manager
Laura: Project Manager
Julia: Project Leader
Sandy: Quality Head
Toby, Andy, and Victor: Software Testers
Linda, Lisa, Paula, Edger and Alex: Module Leaders
Storyline: Project Manager, Roberts has newly been shifted to manage various projects (running or in pipeline) as this post got vacant after the resignation of Laura. Roberts prior to this was head commercials and this is a new avenue for him, he now handles many projects at a time and is okay with it. Julia his Project Lead for the project in question is leading a project for the first time. She has been involved in project implementation earlier but as a module leader and not the project leader. Sandy is the Quality Head and is responsible for the QC of any software that goes to any customer. Toby, Andy and Victor are reporting to Sandy. Linda, Lisa, Paula, Eder and Alex are working on different modules of the project lead by Julia.
Laura was a cherished leader a year back, as she was able to grab quite a good number of overseas projects. But somehow she failed to manage the projects as each project got overscheduled due to one reason or the other. Ultimately, Laura had to admit her defeat, and she resigned from the company. But does it solve the problem? No, the company by now is in deep troubled waters and has a big question mark on its reputation for the existing and future projects. Julia and the team were already designated by Laura and were sent to the client site for implementation. Sandy and Laura were not in good terms as Laura always wanted Sandy to bypass the defects in the product and wanted him to give a go-ahead for implementation without worrying about the completion of the product (or ignoring unfixed severe bugs reported by Sandy’s team).
Roberts did not disturb or made any changes in this particular project team lead by Julia. Julia and team are back from customer site without completion of the project as some issues arose which could not be handled by Julia. Few of the outcomes of the team and team leader that are back from the project (still incomplete) are:
1. Toby, Andy and Victor were not happy with the state of the software when it was taken for the implementation at customer site
2. Sandy had clearly told management in advance that there are chances of project failure
3. Linda, Lisa, Paula, Edger and Alex are not happy with their leader Julia as they feel that Julia who is also heading a module of this project is always over occupied in her module and has no time to address project or other module issues
4. Julia was not able to convince customer management about the product
5. Many business rules that were built in the product shockingly absolved as the wrong requirements understood by business study team.
6. Julia is not happy with her team members – Linda, Lisa, Paula, Edger and Alex as she feels they are not listening to her and doing whatever they want to do without consulting her
7. Julia and her team were all sailing their individual boats in different directions at customer site.
8. Julia feels she is the team lead just on papers otherwise it is being lead by someone else.
9. Roberts feels that the initial system study done at the time of Laura was not satisfactory and accurate.
10. Roberts also feels that even now when the team goes back for implementation of the same project, there will certainly be many issues.
11. Roberts is not sure about the timeline or schedule of the project, that whether this time the team lead by Julia will be able to finish the project in scheduled timeframe, but is not worried as he knows that the management will accept his failure because he is not the one who started it. He feels secured even if this project fails as he is sure that the management will not him fully responsible for this, or even if it happens he can pass on all the blame to Laura who is not with the organization now.
Ultimately how do we ensure the successful completion of this project in stipulated time is a big question (although it is not disturbing anyone as everyone has already a bundle of reasons in his pocket)?
Does someone have the answer?
Testing does not ensure risk-free software. There is nothing called hundred percent testing. Be it manual or automated testing but all testing is in a way manual (even in automated testing the test scripts are altered manually) and hence it is for sure that the testing will never be able to cover all possible test cases in a practical situation if compared to an ideal situation. Every time my team comes to me with a final report of bugs of any software tested, I am able to point out many cases that they say “yes, it should have been done! But we missed it or skipped it!”. Sometimes some cases or scenarios are intentionally dropped by the tester based on just his imagination that this case has no significant role. But if looked from another angle, the same intentionally dropped case appears to be a very important test case that is must to conduct to understand the result of an important behavior of the software. Like security, testing also does not ensure zero percent risk after it has been done. Only thing it ensures is minimizing the risks and vulnerabilities to the best possible level. Software Testing is the process to reduce or minimize the bugs or defects hidden in the software.
For further tips on how to achieve this you can go through my previously posted blog as “Twenty ways to ensure complete coverage of software testing”: https://itknowledgeexchange.techtarget.com/quality-assurance/
To ensure complete coverage of software testing, testing team must be careful about certain activities that are part of the process. If the software testing is not complete as per the business and customer requirements, it could have a severe adverse effect during or post implementation of the software at the customer site. The more is the coverage, the less are the chances of any bugs passing to the implementation phase. So, to ensure the complete coverage of software product and to find the maximum possible bugs or defects, testers have to ensure following steps:
1. Ensure that the documents defining the business and customer requirements are complete and correct. How to do that can be taken up in a separate topic.
2. Ensure that the development team has clearly understood the documents.
3. Ensure that testers themselves have thoroughly read and understood the documents.
4. Prepare a clear cut scope of testing based on product documents.
5. The strategy and Test Planning is as per system requirements.
6. Decide test methodology and test tools (if any), and test schedule.
7. Prepare Test Cases based on business rules and customer requirements.
8. Ensure that the test cases are extensive and sensible to cover the complete requirements testing.
9. Ensure that during testing no changes in the test environment (coding etc.) is done by development team.
10. Ensure that development team representatives (1 or all) are present during the complete testing.
11. Create Test Scenarios based on test cases.
12. Observe the result of each test case and record it accordingly.
13. Prepare a comprehensive and detailed test report explaining each of the test case, scenario and its result elaborately.
14. Ensure that all bugs reported should make sense (no duplication/overlapping of scenarios and no repetition of bugs reported)
15. Ensure that the complete testing finishes as per schedule.
16. The final report submitted should clearly state the areas not covered under testing, reason for the same and its impact on the product.
17. Simulate any bugs that are not clear to development team.
18. Ensure that you have a tentative plan from the development team when they are fixing all bugs and submitting it back to testing team.
19. Verify all bugs fixed and ensure that the development team is sitting with testers during verification.
20. Prepare the final report and submit back to Development team, giving the status of each bug fixed, as verified as fixed or not fixed. Report any new bugs arrived in the software while fixing these bugs.
3 Years back there was no QA department in the organization. The development manager, John, was the king. Nobody was there to check at what stage his development projects were, all used to go by his words. One day he was asked – hey, what is the status of your software that you are developing for Canada? Oh Sir, he reported his boss, it is all ready. When do you want to send it to Canada with my implementation team? Ok, if it is ready, let us book tickets for coming weekend so that your people land there, settle down in the hotel, enjoy weekend, and start implementation on Monday – his boss told him. Ok Sir, he replied.
John called his development team responsible for Canada project, takes stocks of situation, gets to know that still many screens, sub-modules, units are under development and at no cost finish by weekend. Oh, but I have told boss that everything is ready – he tells his team. Don’t worry sir, his team tells him, we will finish it there. Ok, he replied.
So the team goes to Canada, the target of finishing the project is not met, users are not happy with the product (it was not a complete product), management got angry, stay is increased, but even with the extended stay, the project does not take proper shape and hence finally the team is sent back.
What now! An emergency meeting is called. Reasons for project failures are found out. It comes out that John had projected a rosy picture to the management without showing the existing thorns in it.
And the QA department took birth with a resolution that no software will move out to any country without a final clearance from QA.
3 Years have passed, John’s life got miserable in the beginning. Each of his products had to undergo 5, 6, 7… cycles of testing and re-testing by QA. Initially QA reported why the product has come for testing when it is not complete? Why it is declared as complete? Why during testing, the development team keep doing something or the other in the same product instead of giving time to QA etc. etc.
Ultimately maturity arrived. The products only are shipped when they are sensibly completed and verified/tested.
The Development Manager finally says to QA Head – You exposed me, no problem, your team tortured me (my testing and re-testing), no problem, ultimately I am the happiest man now because now when I declare about a product that it is complete, and my boss stares at me, I add immediately, and it has been cleared by QA after x number of testings finally. I am sure it will not fail now as usual.
John’s life has become tough but he is the happiest man on this earth, you know why, because now he is sure that none of his projects will overrun or fail.
Software Developers are not doctors and that is why are not in the habit of doing ‘first time right’ coding. All newly developed software contains bugs, bugs and bugs. Developers have this mindset that no software can be written without a bug in it. They are convinced that bugs are integrated part of coding and involuntarily that are bound to place them here and there in their code while writing a software. If doctors start working like this that in every operation they leave some important activity ‘undone’ and have a privilege of getting each operation audited by another doctor (specially meant for auditing a patient immediately after operation), the auditing doctor will find out the shortcomings in the operation, report it to the doctor who operated upon the patient, the patient gets operated again, again auditing, again operation, so on and so forth. Think of what will happen to the patient in this case. And I think the patient will prefer to die on his own than in this manner.
The same thing happens to the health of software when some bugs are embedded in it, found out by testers, reported to developers. The developers fix those bugs, in turn produce some other bugs, again it comes to testers. Another bug report, another fixing, another different set of bugs production, so on and so forth. What happens to the health of the coding in this case?
If software could shout and cry!
A point to think upon and act upon!
To me sometimes it appears, may appear funny to some, and logical to others, that there is a strong resemblance in a Poultry Farm and Software. As different Hen in a poultry farm keep producing eggs day in and day out similarly different modules in the software keep producing bugs.
Do we agree then that as hatching eggs produce more hens, similarly fixing of a bug produces more bugs?
Eggs which are not hatched but consumed may then be related to bugs which are there in the system but have not been identified and remain floating in the system. As is known that many bugs cross testing phase without being identified and come into light during live run (i.e. post implementation phase), sometimes after months of successful completion of a project.
Do we require such software – productive (producing bugs) like a poultry farm? Or we need good programmers to make good software and not turn it into a poultry farm.
I invite your valuable Feedback/Comments please.
Equivalence Partitioning Testing Method is a method used in Black Box Testing (see: https://itknowledgeexchange.techtarget.com/quality-assurance/what-is-black-box-testing/) for the purpose of finding out minimum set of data for testing for valid and invalid inputs.
The purpose of this testing is to:
1. Ensure that the software acts according to the type of data is entered, i.e. for a valid entry it should accept the data and for an invalid entry it should return the appropriate message/alert.
2. To ascertain that minimum data is used for similar results (the result of a test case could be valid or invalid)
For a group of values, if the software behaves in same manner, the group or set of value is called ‘equivalence class’ or ‘equivalence partition’. So primarily for an equivalence class the software will behave in the same manner and shall produce the identical result. Different equivalence classes will make the software behave differently and shall produce different result.
The tester has to be careful in carving out the different equivalence classes so that one value is taken from each class to ensure complete testing. The testing in this way is completed with least efforts/ test cases.
Let us take an example to illustrate:
In a particular Airline based on number of flights taken in a year, a membership is offered with different privileges as below:
If the number of flights taken is between 5 and 10, the membership offered is ‘Silver’
If the number of flights taken is between 11 and 15, the membership offered is ‘Gold’
If the number of flights taken is between 16 and 20, the membership offered is ‘Platinum’
If the number of flights taken is between 21 and 40, the membership offered is ‘Diamond’
Based on above all inputs between 5 and 40 are valid, less than 5 and more than 40 are invalid. And also any non-numeric value is also invalid.
So there will be 7 equivalence classes viz: 5-10, 11-15, 16-20, 21-40, 40 and non numeric and thus one value from each equivalence partition will give you the complete test coverage.
Black Box testing of a software is based on two primary things –
1. Customer Requirements Document
2. Functionality of the software
Purpose of the Black Box Testing is to ensure, that means, that –
1. The software meets are customer requirements as specified in the Customer Requirements Document (business needs)
2. And the software meets user requirements (user needs)
By user requirements we don’t mean that it has to meet every user’s requirements but more appropriately the appropriate usage requirements.
Black Box Testing is the most widely used testing and is used to used to check if the product meets per se customer’s all requirements. The tester is not worried about how the software has been built, what queries have been written, whether the coding is technically optimal or not, but focuses more as if the customer is working on the system and feels like customer when he tests the software.
A basic requirement of Black Box Testing is that nothing should be presumed about the functionality of the software while testing the software, rather the tester is supposed to check each and every functionality of a unit, module and the product as specified in the customer requirement document. So the fundamental of this testing is the Customer Requirement Document, based on which the appropriate test plan, test cases and test scenarios are built and ultimately test results are drawn as a Test Report (or Bugs Report).
For building various test cases in Black Box Testing certain techniques are used like:
Boundary Value Analysis (or Boundary Value Testing), see my earlier blog on this:
Equivalence Partitioning: This is a technique of testing with minimum of test classes or data input to identify between valid and invalid input.
Prior to starting actual testing, a tester must verify and validate:
1. Customer Requirements
2. Functional Specifications of the software
3. Test Plan
4. Test Cases
5. Test Data
6. Test case execution
After above validation, the test cases are executed, results are recorded and reported to development team.
Way back in 1960s softwares were being produced but there were no established standards or software engineering practices. Poor design and Regular hardware upgradation left many types of software useless and thus a strong need arose in the industry to provide effective and reliable software developers who could pace up with regular hardware advancement to produce faster and highly reliable softwares.
All this, plus the increasing dependency on softwares increased not only demand of good developers, but also a requirement of software testing took birth, may be, not initially in a very strong and structured manner but it did.
Software engineering practices that were taking place at that time were not formulated, or structured in an efficient manner. Some of the bad practices were no measurements or metrics were in shape to measure up –
Development lifecycle or product lifecycle
Lack of planning, monitoring
No control on development schedules
No control on customer requirements changes
No reviews of documents
Failures at customer end during or post implementation due to product incapabilities, schedule delays, revenue losses
Therefore a need to test the software came into existence to:
enhance all these processes,
increase the reliability of product,
deliver a quality product,
deliver in time
Let us take an example that a code is to be written for entering employee details in employee master table. There are certain conditions for validating while entering a new employee for a specific post that his minimum experience at the time of joining has to be 5 years but not more than 10 years. The coder has written the code and designed the screen to input data satisfying these conditions. Now this code comes to tester for functional testing to verify if it meets the desired requirements.
How tester is going to confirm it?
A tester has to write test cases by understanding the following:
2. Valid entries
3. Invalid entries
4. Inside edges of the boundary
5. Outside edges of the boundary
6. Boundary points
Let us see what each one is and why it is important as the test cases are going to be built on this basis:
1. Boundary: The boundary as per requirement here is 5 to 10 including both values.
2. Valid Entries: The valid entries shall be all values between 5 and 10.
3. Invalid entries: All values other than in point no. 2 are invalid
4. Inside edges of the boundary: Inside edges of the boundary in this case are 6 and 9
5. Outside edges of the boundary: Outside edges will be 4 and 11
6. Boundary points: Boundary points shall be 5, 6, 7, 8, 9 and 10.
So the test input data shall be: 4, 5, 6 to 9, 10, 11
Definition of BVA or BVT: Boundary Value Analysis is the process of selecting test cases (or test data) by understanding boundaries that differentiate between valid and invalid conditions. Tests are run to check the inside and outside edges of these boundaries, in addition to the actual boundary points.
During Software Development certain business rules and disciplines need to be embedded in the software based on the customer and business requirements as specified by customer during the initial business study. These are required so that the software meets customer expectations and needs. A Document is prepared based on this needs which is known as SRS (Software Requirements Specifications).
To prepare SRS a group of professional from the organization who is responsible for this software development, visit the customer site to have a first hand look at their processes, documents, functions and requirements. SRS is prepared as detailed and explanatory as possible because this document is handed over to the development team who will be responsible for developing a good software matching customer needs and meeting customer requirements.
SRS in a way is the foundation stone for good software that has to take birth based on it. It is very important to meet the right people at customer location who are deeply involved in these processes in their respective fields. A detailed study of each process, supported by relevant documents, forms, reports need to be collected. Process owner must devote ample time to the team member who has come for business study so that he in turns documents it well as part of SRS.
Finally each Team member compiles, edits, verifies and prepare a document that is called SRS.
In Software Testing we need to devise an approach that features a gradual progression from the simplest criteria of testing, to more sophisticated criterion via many planned and structured steps, each of which brings incremental benefits to the Project as a whole. That way, as a tester masters one skill or area of testing, there will always be something worthwhile to try next.
Important Aspects are:
Trust: Testing and development teams have to build a trust-trust relationship in each other and in the product under development.
Constructive Communication: The initial communication of testing team with the Development team has to be quite open, transparent, constructive and encouraging. This is the phase that decides about the course of action for testing and any alternatives. The future of software, its accuracy and the successful implementation at customer site afterwards will be the key factors emerging out of it. Don’t make this communication too formal.
Documentation: Ensure that testers have sufficient documents in hand before starting the analysis and testing of the product. Appropriate analysis of the documents is very important to remove and ambiguity/confusion/loop-hole in understanding the customer requirements. Test cases and scenarios will emerge out of these requirements.
Open Mindedness: A tester has to go with open mind in all discussions only then he will be able to understand developers, development process, customer requirements and then he will plan on the testing requirements accordingly.
Variables: Keep all variables in the loop that are important to the testing and successful product development. Different variable could be developers working on different modules, documents, testers etc.
Learn from Past: Keep learning from past experience and try not getting caught in any mistakes that have been made earlier. A common saying is – “Never repeat the same mistake again”. It is a mistake to go for testing of a product in a crude or unstructured manner. It is a way of improvising your testing model over a period of time with more and more experience.
Indentify Shortfalls in advance: Be clear about what is in scope of testing and what is not. Clearly define what is not going to be covered in testing and what could be the impact of them on the product. Foresee and uncertainties/delays. It is important to understand the benefits you are going to get well in advance rather than getting no benefits at the end of this whole exercise.
Keep Interacting: A tester has to keep interacting with the developer throughout the testing process. Ultimately the goal of both sides is to have a better product at the end crossing all barriers of conflicts, bugs, faults or defects.
Your valuable comments are most welcome.
Assume that after functional testing of a software following is the summary of count of bugs:
Total Bugs : Tn
Severe Bugs : Sn
Critical Bugs : Cn
Desirable : Dn
Here Sn + Cn + Dn = Tn
And let us assume that the estimated expenses of removing of those bugs (based on developer’s direct and indirect cost) are:
Average Cost of removing a Severe Bug : $S
Average cost of removing a Critical Bug : $C
Average cost of removing a Desirable Bug: $D
Then we can say that the Average cost of removing a bug (Severe/ Critical/ Desirable) : ($S + $C + $D)/3 = let us say $B
Here Total cost ($T) of removing all bugs is: $T = ($S * Sn) + ($C * Cn) + ($D * Dn)
This will be for one project and similar data van be obtained for various other software projects under development in the organization.
The graphical or tabular comparison of Tn * $B to $T for all these projects is the linear approach of Cost Estimation of Bug Fixing or Defect Removal.
At times Testers need to make the door open (friendly or forcefully), shut by the Project Manager. If door is not opened at appropriate time it may hamper/delay the progress (or ill-effect even the successful implementation) of a project. And those times could be:
Initial Documents: When the Initial documents are incomplete or ambiguous, not explaining completely about the user requirements or the business rules to be embedded in the software. This is the most crucial situation and if not handled immediately – will build up a wrong product – built by right people and tested by right people.
Test Cases: At the time of preparation of test cases and test scenarios, a tester may consult Development Team members to confirm if all aspects are covered and nothing is missing.
Testing: During testing it is always better if the respective developer sits along with the testing team, so that the bugs are confirmed hand-to-hand. This shall not require simulating the bugs at a later stage (and this will definitely save time).
Testing Report: After submission of testing report, do confirm with the concerned developers/ Project Head if any clarification is required.
Testing Feedback: Development Team is supposed to send its feedback on test report in a stipulated time frame, remind them if it does not happen.
Feedback: Check the feedback thoroughly and if there is any point raised by testers which is missing feedback from developers or if any feedback is not clear, do check it with the developer’s team.
Verification: Insist on development team sitting with testers while verification of closure of bugs. Record all closures and any new bug generated or old bug reopened during verification.
Creator vs Destroyer: As a law of nature – creator can not be destroyer. Similarly in software, a programmer can not be critic of his own code.
Constructive Criticism: A good programmer will always love and admire his code, but will never have that ‘third eye’ (that a tester has) to locate bugs in the code. A programmer is fond of listening to ‘praises’ for a well written code than listening about the shortcomings in the program written by him.
Vision of a tester: A programmer will always think of the pass conditions in most of the cases and will bother less about the negative scenarios. A simple example could be that the programmer has to write a code to accept a password that has to have minimum one special character, two numerals and length not less than 8 characters. Once this condition is fulfilled, the programmer will be more than satisfied – but what about n number of different test cases that a tester creates just to check this simple requirement.
Fear Factor: Even sometimes when a programmer knows about a flaw in his coding, seeking a major change and coming into his notice at a later stage, he will like to oversee it and also try it to be to be overseen by others. He may fear that highlighting a shortcoming in a code written by him at this critical juncture may loosen up his superiors trust in him, which is usually dear all everyone at any cost.
Thinking Pattern: Walking on the Same Road everyday makes one perfect on that road but what about other road patterns, walking backwards etc. Continuous Coding, Coding and Coding make him a perfect coder but only a perfect coder.
Documentation: Usually a coder writes code perfectly but with very little or no documentation. At times, he himself forgets what logic has he written for a specific requirement and then he has to refer to his own code (and waste some time) to find it out.
Self Testing: The confidence (or over confidence) stops a coder to introspect or examine his own coding by doing smoke testing or jotting down some test cases and scenarios to run them and get a pass through for his own satisfaction.
Tight Schedules: A programmer is always having his task basket full and has delivery schedule tight. That also becomes one of the reasons for passing on some bugs (in a hurry).