A bugs report or defects report is a list of bugs found out by testers while testing a software product in testing phase under a testing environment. The test environment is created at development site similar to the actual environment in which the software is supposed to work or run in live situation at customer site. Environment built for testing is supposed to be an independent entity meant only for testing team to perform testing without any development changes happening on it.
Bugs or defects report is the most vital tool for developers to understand where the product that has been built by them is lacking its functionality or performance. Testers have to be very careful in studying business and customer requirements while building their test cases and test plan. The quality of test cases will clearly enhance the core performance and functionality of the product built.
What is a test environment? – I have already explained in my earlier blog (https://itknowledgeexchange.techtarget.com/quality-assurance/what-is-a-testing-environment-for-software-testing/). Also I stated in my previous blog – “why build a separate test environment” (https://itknowledgeexchange.techtarget.com/quality-assurance/why-build-a-test-environment-for-software-testing/) from which two major parties being benefitted are customer and quality testing team of the vendor. Here let us try to understand five essentials to be adhered to while building a test environment for software testing. Let us also understand that these are not the only essentials, there could be many more, I am taking the five top priority factors of them.
1. Customer’s Environments: Understand clearly the environment in which the customer is going to run this software. This has to be checked not only for server but also for the user’s machines. The environments factors could be the hardware, OS, Database, Front end tools, browsers etc. Take care of all the versions of OS, browser the customer machines that are going to run this application.
2. Test Server: Build your test environment as much as possible a replica of customer environment. This is to be applicable to Server and client machines as well.
3. Separate Test Server: Build the test environment on a separate server free from development and dedicated exclusively for testing purposes.
4. Understand business requirements well: The testers and test lead should be very much clear about the customer requirements based on which the test cases are to be built. More understanding, more coverage. Much clearer understanding, wider coverage.
5. Documentation: Aesthetically document each and every test that the testers perform for a unit, module or integration testing of the product.
In continuation to my previous blog – “what is a test environment?”, let us understand here why a test environment is required for testing a software product? I see two major purposes for this, one purpose catering to customer end, and the second purpose catering to the vendor end quality testing team.
The vendor or supplier who is preparing or supplying a software product to its customer must be very clear that in what environment his customer is planning to run this product. The environment could be the hard side – i.e. the hardware configuration comprising of hardware server and users PC components viz – hard disk size, its speed, RAM – size and speed, Processor, bus speed etc. in the soft side it will comprise of the Server/ Users PCs Operating System and its version, OS Patches requirements, Browser – which all browsers are supported and which versions, IIS server specifications at server end, Database Server – What database and which version or release. Any other software components required at server or users computers. The purpose is to run this software as if it is being used by the client at his site. In a way it is the simulation of client site.
The second purpose to have the test environment separate to development environment is to restrict developer’s intervention in test environment during the test phase. This is to be done so that the test results are clear, real and non-ambiguous.
Any other reason coming to the mind while reading this blog that comes to my readers – is most welcome in the comments section.
A testing environment is a setup of software and hardware on which the testing team is going to perform the testing of the newly built software product. This setup consists of the physical setup which includes hardware, and logical setup that includes Server Operating system, client operating system, database server, front end running environment, browser (if web application), IIS (version on server side) or any other software components required to run this software product. This testing setup is to be built on both the ends – i.e. the server and client.
I remember a case where an application was built strictly as per customer requirements by a software vendor without understanding or taking it seriously that what database version the client is intends to use for this software. The vendor developed the application on Microsoft SQL Server x version and the client purchased the new server with the next version of MS SQL. When the product was completed at vendor side, he did a fantastic testing and made the software functionally very strong meeting customer’s all business needs. And when the product was launched at the customer location on the next version of OS and database, the product misbehaved (or did not perform) in few of the instances for which the technical team had to be sent to the customer site to fix the gaps and make it run hassle free. Now this not only affected the product cost but delayed the project deadlines in a major way. This small ignorance at both client and vendor end made a recursive effect at the vendors profits and next few projects.
The production or development team when completes a product development at their end, have to produce this test environment for testing team prior to loading their newly developed product on that environment for testing.
A Project Manager is mostly sceptical to the testers that they unnecessarily try to increase the bugs list just to keep quality department on top. He also feels that it is intentional and many bugs can be avoided to be mentioned which are not very important. Many Project managers in general also feel that quality department is not necessary to exist in an organization. The quality can be maintained by hiring good developers, is what they feel. Then why a separate battalion of testers is required in most of the software organizations across the globe.
Many factors, I would say are responsible for this. Customer getting more cautious towards quality and delivery of product, timelines, competition, increase in internal and external expectations, benchmarks, awareness, prospects, continuous improvement, survival of the fittest, revenue growth and timely returns are few of the top most of them.
There is another side to this coin – A project manager’s skepticism initially may be due to a fear that his team’s development weaknesses will come into light. One more weak area could be the incomplete or inaccurate documentation or study of customer requirements. But with positive signals from Testing Team, Project manager has to gradually understand and admit that all the efforts being done by testing team is not to highlight the weaknesses of his team or product, but to strengthen his team and product.
The switch over or additional responsibility may arise out of different scenarios. Small software organizations initially have no quality assurance department, but with their gradual growth and to meet project commitments, the quality assurance emerges as a separate department.
At this juncture the organization may not be ready to appoint somebody from outside as quality lead, due to various reasons. The reasons could be that the new person will take his own time to settle down in the organization and a couple of months to absorb organization culture. Other reasons could be since this is an experimental start of a new department for which organization may itself be unclear about the department size and its growth.
A new person may refuse to join the organization as a quality lead with too less people in the team. The requirement initially could be too small that one or two persons are required to start the new department. And most important factor is that the internal person working in say production, development or implementation will understand the gravity of emergence of the newly formed quality assurance department and hence will be able to move the department in the serious direction management expects it to.
During my 20 plus years of experience in IT, from time to time I received questions from career beginners (or seekers) having spent 1 to 4 years in one stream of career with a query related to their career growth. This query came from different individuals having spent some time successfully as a developer, or tester, or project leader, or quality leader etc. Streams like development, testing, implementation, support and so on keep prompting or attracting the professionals to peep on the other side of the career that appear to them greener than their own. You never know the comfort and pain of that green appearing side unless you put your feet in their shoe and feel the pinch yourself.
Similar questions I received from others were asking, if not about the career growth, then about the possibility of diversion in the career of one of these persons and the scope of that change. Certain other questions were related to a person wearing a special Hat in the way of switch over or in the form of additional Hat. That person may be a quality lead told by the organization to switch over his role to development manager, or vice versa. Or sometimes someone queries if it is possible for a person to wear different Hats at different times during the day performing multiple roles at the same tenure of his career.
So it does not imply that the hat change or additional hat is always wishful, many times it may be imposed one too. I would say that there is no harm in accepting the challenge and getting into it although it will definitely ask for extra efforts and involvement.
On the other side it is very true that a strong developer may not turn out to be a strong tester and vice versa. Same is true for project managers and quality managers.
What are my reader’s views on the same?
Have you experienced a change in career from one stream to another, if yes, what was the reason.
It may not have happened to you but to someone around you, but still share your points?
It is not only the coding or development that accounts for a successful project. The journey starts right from the release of order from customer after which the customer requirements need to be studied accurately and extensively. This is going to be the foundation of the project. One small slippage or one wrong business requirement in customer requirement document may lead to a big cause of failure of a project. Now there are two ways of going about it. Either the organization has a product in place which is to be delivered with minor or major customization depending on how much customer requirements are meeting with the product already in place. In that case, the next step of mapping it with the existing product and simultaneously finding out the gaps is very crucial. As the clarity of customer requirements and business rules is important to build a right product, equally important is the clarity of gaps. Once gaps are clear, the role of developers comes into picture. Project Leader has to prepare a plan to finish off the job through his developers and under his guidance. The plan should be aesthetically adhered to, to complete the development in time.
Next comes the role of quality control department that has to ensure that the product finalized and finished is meeting customer requirements in all respects. Testing starts at various times in various projects and various organizations depending on the seriousness, methodology and awareness of the organization. Many organizations now involve testing team right from the beginning of a project at the time of its inception stage when not a single document has been prepared. Belief and fact is that involving testing team at this level always enhances the development and helps in meeting the deadlines better.
Documentation is a very important factor – be it user requirements, gap analysis, technical architecture of the product, database design document, ERD, users manual or any other relevant document.
Last but not the least comes the implementation team’s role. Each member plays a vital role especially when it is an overseas project and the success is very crucial. Going to a new environment, adjusting to a different culture, away from family, daily tight targets, late hours working are all the factors that have to be taken care of.
This is the last in my “Project Management” series. To enjoy this blog you will have to go through Project Management I (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-i/), Project Management II (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-ii/), Project Management III (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-iii/) and Project Management IV (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-iv/).
Nothing is constant in this world except “CHANGE”. One this was clear by now that we have to change – immediately and without any further delay. To enhance our Project Management Process, we had already identified our weak areas, we knew what is to be addressed, and when it had to be done also was very clear – it had to be NOW. How? We had to think and analyze ourselves.
We had to transform ourselves in following respect:
Project Requirements: Not only the customer and business requirements but all project requirements need to be specific, clear cut, vetted by customer, documented – including any communication taking place thereafter – in form of email, telephone or otherwise.
Quality: Involvement: Testing team right from the beginning of the project and make them a part of development team, let them participate in all development discussions. Let them be very clear to identify the risks involved if the product is developed as per the project specifications and design finalized. Testing team has to be an integral part of the project now onwards, testing not at the final stage, but at each and every stage of development, by clearly identifying the scope of testing (unit, module, product, functional etc.) and highlighting the defect or bug at the moment it is encountered without waiting for submission of a test report (which in any case will happen but not after each defect identification!).
Sizing and Right Candidature: Go for the right size and right people while building a team for a project.
Backlog: Be clear about the requirements still pending to be catered to, built into the product. Count it as backlog, cut backlog into smaller pieces, and assign each element of backlog to a team member with the time plan (hours or max day).
Monitor: Regular monitoring of this backlog planned versus the completed is required atleast once a day. Involve only the concerned in the backlog as with the backlog resizing, the team also have been divided into smaller parts.
Involve Customer: Keep customer updated on each and every backlog completed. Let the customer, if they wish, to run through the backlog completed. Let the customer, if they wish, to check the balance backlog.
Release the Product: Release the product as per your commitment date. Any pending jobs (if at all!) may become the part of next release after their completion.
Hope you enjoyed the series.
In my earlier blogs on Project Management – Project Management – I (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-i/), Project Management – II (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-ii/) and Project Management – III (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-iii/), I tried to spot on certain defects in our project management process. All these defects or shortcomings were initially not visible due to small teams, small size of projects and different management perspective. But a requirement to relook into ‘project management’ arose when management had to change their perspective due to change in customer perspective.
This asked for introspection, analysis and thought process in which we identified certain factors as stated on my earlier blog “Project Management – III”.
All these factors were causing:
Missed Schedule – Most of the projects were either getting overdelayed by dying to their natural death prior to completion or implementation.
Missed Functionality – Lack of customer requirements, lack of documentation, isolation were few of the factors that caused the build of an improper functionality or totally missing it from the product. The customer living on another island was in a different imagination world dreaming about a different functionality and product as compared to the one being built (or missed) by the development team.
Missed Budget: Obviously – any delay in schedule, or building of a functionality that was skipped during initial built will ask for a fresh expenditure. Is customer ready to pay this additional amount?
Brittle Product: The product is too fragile or brittle that it is loosely threaded in such a manner that it will lose its flow or functionality during actual usage.
Quality Aspect: The product that is seeking changes during the later stages is always prone to weak quality due to time and budget constraints and various other pressures from customer side.
One thing was clear by now that we need to change, as there is no alternative. We need to change to strive, to exist, and to deliver. We need to change to get out of our static system. We need to change to meet customer requirements. We need to TRANSFORM – OURSELVES, OUR PROCESSES, OUR THINKING.
Change – What and How – wait and hop on to my next blog in the series – “Project Management – V”
To carry on further from my earlier blogs on Project Management I (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-i/)and Project Management II (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-ii/), this is the third in series. Project Management in my opinion, is not a static Management, it is a dynamic state. It keeps on giving a new learning in every project, and seeks changes in the project management process. There are instances where the earlier project management methodology of freezing customer requirement with a document preparation, a sign off from customer, stating that the document is sufficient to cater to our requirements and the software be built based on these requirements mentioned in the document. Now the requirements may change during development, or even implementation, during any stage of project management. You can not say ‘no’ to those requirements asking for change in software, as you know very well that ultimately a ‘good software’ is the one that caters to the customer requirements and customer business than to those mentioned in the ‘business study document’.
What we lacked in our methodology of project management mentioned in my earlier blog “Project Management – II” was – a clear cut ownership of the project, involvement of all concerned throughout the project life cycle. Documentation was another weak area – documentation of not only the requirements, but of system design, database design, communication (internal and external) associated with the project. And above all, the random and isolated involvement of different stakeholders, like QA will enter into the project only when the product is complete and ready for testing.
Now, of above, imagine the time spent on QA to make them understand the complete product right from scratch, that too in absence of complete documents.
As discussed in my earlier blog on project management (Project Management – I, https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-i/), different organizations adopt different procedures, processes and policies for their Project Management, and then keep improvising them until they reach at the level of ‘best’. Best practices in ‘Project Management’ are not a single set of practices that are used by various organizations – but rather I would say any ‘good’ practice when used to its best of effort, sincerity and energy becomes a best practice. Any best practice in Project Management asks for involvement, from all levels, starting from top management to the junior most level. Here, I would discuss some practices in software project management, which we adopted earlier, then brainstormed on those practices that where we were as compared to the best practices being adopted and followed by various other global organizations, analyzed where we need to change and how we need to change with clarity of visible benefits out of those changes adopted.
Initially it was a purely waterfall scenario – with the sequence as – Requirements and System Analysis, System Design, Testing, Implementation and then support finally. This was going well for years, without any much of issues. But later when we grew and analyzed we found that it goes well with small team sizes, small projects, fixed requirements, in scenarios where reporting of apparent success to management was sufficient.
But we found a resistance in out own system – team sizes were not small now, projects were of all sizes – small, medium and large, requirements were never definitive and we have to be open for any requirement change asked by customer during a project, and above all management is not at all interested in any offline report showing progress of the project – rather they are interested in ‘progress’ if it goes well without any ‘reporting’. That means the management wanted to see the progress – visible and in unanimity with the customer (internal as quality, and external as the Product Owner).
Project Management is nothing but adoption of a set of processes to ensure the smooth running of a project development passing through various stages and its successful and timely completion. It is the crux of any project as I too believe that it is not the people who make the projects successful, it is the processes and procedures adopted by the management responsible for the project. Project Management depends on how matured a company’s processes and procedures are. Broadly in terms of Project Management, organizations can be divided into three categories: First category will be of organizations having no or a very little system in place for Project Management. The organizations falling in second category would be those who have processes and procedures in place but not adequate for 100% successful Project Management and hence those organizations keep on striving for improvements by continuously analyzing, trying, good practices for enhancements, optimization, and successful project management. The third category of organization would be those who are matured, balanced, and a showcase for other organizations. When I say balanced, I mean they are neither too rigid nor too flexible in their practices. These organizations have a set of best practices in their sleeves being used for years, time and tested again and again. They would decide upon the best suitable practices depending on a specific project. When I say matured, I mean their success rate is higher than their peer organizations, they are the best in Project Management. Their employee turnover ratio is quite less, their employee satisfaction level, energy level and passion level is higher. They have not only adopted the best practices in their system, but in their organization culture, and in the blood of each employee.
For first and second category of organizations, the journey is long to reach to the final level, but equally higher is the scope to achieve it. Provided they have an urge, zeal and fire to achieve it, they stay on their levels for years. It is the management that has to drive it and for management it is sooner the better looking at the competition at global level and the expectations level of the customer. That does not mean that for the third category of organizations it is the end of the journey, rather it is difficult to maintain, sustain and keep improvising as everyone knows it is easier to reach at the top once than reaching every time, and maintaining it.
This is in continuation to my earlier blog “Project Management – A Sad Story” (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-a-sad-story/). Keeping all the character names same as described there, I will update my readers on what happened to this particular project headed by Roberts. Although Roberts is new to these projects but is trying his best to get into each issue and address it appropriately keeping ‘common sense’ part intact while making a decision. Roberts knows and often tells others that project management is not only about the management of a project, it is about managing people working under you, it is also about customer. The story takes a couple of twists and turns and moves altogether differently.
Julia, the project leader, accepts her defeat, and resigns from the organization. Linda, Lisa, Paula, Edger and Alex are doubtful about the success or completion of this project. The issues that arose at customer site were majorly two as stated by the customer – one is that Julia is not competitive enough to understand business requirements and embed them into the software, and second is that Team has no purpose in staying at customer site till the product is complete. So they sent the team back to their country. Julia’s resignation is accepted as management also feels she is one of the reasons of this failure. Sandy, the QA head, is given an additional charge as project leader of this project. Sandy has a good record or project management and has been 100% successful in all the project he handled so fat prior to becoming the QA Head. One would wonder when he was so successful in Projects, why he became QA Head. Probably he knew that this Quality of ‘quality; of a product in him is one of the key factor for all his project management success. Sandy accepts this challenge but with an apprehension that before Julia stops coming to the organization, she atleast explains him the complete picture at customer site about project, requirements, and customer. Sandy has to not only take care of all customer needs to be built in the product, but also to ensure the quality aspect of it. The project is delayed by 20 odd days. Development team is working in the customer requirements. Sandy is overall now is the linking part between offshore customer and his development team (and also for his testing team).
Only apprehension Sandy has now is that if the fresh requirements brought by Julia were complete and exact or still if he has to face the similar situation again!
Let us wait and watch!
By the way – what is your opinion on this – what would happen to this project – SUCCESS or FAILURE!
STLC (Software Testing Life Cycle) is an integral component of SDLC (Software Development Life Cycle). Gone are the times when any software was made on the basis of its requirements and the moment it used to get completed by the development team, it got released to the customer. But now, Testing has become a distinct phenomenon during and after the development of software. No software is released to the customer without a comprehensive testing by QC or Testing team in the organization. The Scope and Methodology may vary from product to product, customer to customer, and organization to organization. There are certain aspects of Software Testing Life Cycle. To name top few among them, I would like to list twelve essential steps of Software Testing Life Cycle.
The Steps are to be followed from the start of Testing of software to the end of the testing as follows:
1- Before the dynamic testing, there is a static testing. Static testing includes review of documents required for the software development. This includes following activities:
(a) All the documents related to customer requirements and business rules that are required
for software design and development should be handed over to QA. These documents
should be complete and dually signed by client and representative of the company
(usually Project Manager).
(b) QA reviews these documents. The reviewing of documents includes comprehensive and
thorough study of the documents. If any discrepancy is found then it should be noted
and raised in the review meeting with the Development team.
(c) After this there should be a formal meeting between the QA and development team
regarding these documents, the agenda of this meeting mainly includes what is missing in
the document, QA queries to be answered by Development/Project Team and/or
clarification required for any confusions.
2- After the Software development or build of a module, QA starts dynamic testing. If during the development the requirement has been changed on customer demand or due to any other reason, then that should be documented and a copy of this revised document is given to the QA and also discussed as explained in point 1 above.
3- Development and Testing environment should be made clear to the QA by the Development team. It include the following activities:
(a)- Server to hit for Testing
(b)- Installation of latest build on the test server.
(c)- Modules/Screens to test.
(d)- Test duration as decided by test manager and project manager mutually based on scope
of work and team strength.
(e)– Demo of the software on test server by development team to the QC members.
4- After this Test cases and test scenarios are prepared and then the Test execution by QC.
5- A comprehensive Report of Bugs is prepared by the Testers and a review/verification by QC/QA/Testing Head takes place. Before handing over this report to Development Team there is a thorough review of Bugs List by Test Manager and in case of any clarification required on a bug submitted, the Testing Head discusses the bugs with the assigned tester.
6- Release of bug report by QC Team to Development Team.
7- Discussion/simulation of bugs by QC with development team if development team requires and time required for fixing the bugs should be made clear by Dev team at this stage.
8- Feedback from Development team on reported bugs with the stipulated time frame required to fix all bugs.
9- Any changes in the software being made in respect to fix these bugs should be made clear to the QA team by the Development team.
10- Testing team then Retests or verifies the bugs fixed by the development team.
11- Submitting the retesting bug report to the Test manager and after this the step 5 to step 10 are followed until the product has reached a stage where it can be released to customer.
12- Criteria for ending the testing should be defined by management or Test Manager Like when all major bugs are reported and fixed. Major bugs mean the bugs that affect the Business of the Client.
Testing is a process of facilitating development team/ project team in improving the quality of software before it is released to the customer for use. Some key essential steps are always there that need to be followed by the Testers during Software Testing to streamline the process. The most important checkpoints for testers during software testing, in my opinion, would be:
(1)- Complete document of customer and business requirements specifying all the requirements for the development of the product.
(2)- Latest completed build and URL of the application to hit for Testing.
(3)- Software requirements for installing the application on PCs for testing or for database connectivity.
(4)-Training/Demo of the project by development team to Testing Team so as to understand the flow/functionality of the software.
(5)- Scope of testing should be made clear by the development team/Head.
(6)- If the build comes for retesting, then it should be accompanied by the revised document which includes the updated changes incorporated in the software.
(7)- Clarity regarding which member of development team should be contacted in case of any clarification required during the testing phase regarding the functionality of the module or if testers encounter a showstopper in the Software.
(8)- After release of the bugs list to development team, how much time they will require for fixing the bugs.
On his blog by the famous Leader on Project Management Bas de Baar, he asked a question to all his blog readers – “What is the best advice you can give to project managers supervising offshore development projects?” Various readers posted their opinion as per their perception, experience and learning. His focus was more on Offshore development projects, whereas I would like to point to any offshore project that has to be developed implemented on site. So in context to offshore development and implementation of a project, a project manager has to essentially develop some soft skills other than his/her standard project management tools and methodologies. According to me for an offshore project management following would be the essential key requirements as I am sure every one of you will agree that an offshore project is definitely different to an onshore project. The basic things to be taken care of in context of that country, management, users shall be:
Language: Understand what their prime language is in day-to-day use. I went to a country where language was a big barrier and I was heading the project. Before going there, I learned few of the commonly used words, downloaded a dictionary to translate from English to their language. Although very few of my users were fluent in English, and I used them extensively as a medium to translate what I used to say in my training sessions. But there were instances when I am sitting with a couple of users who don’t know English or who know very little of English, then this offline/online dictionary was very helpful. Also I learned before hand that they speak English very slowly and can understand only if spoken in low speed and a different accent (like “show” will be pronounced as “cho”). Ascent was another criterion that I had to gradually master upon. This will also depend on deciding upon the pace of the project. I had to change my plan accordingly to cover up the backlog at times.
Culture: They will love if you talk about their country and culture, and that is the best way to make them your best friends. Then it was easier to understand them faster. Learn about their likings and disliking, what they don’t like is very important, or what annoys them easily, so avoid doing that or using those words. Don’t refuse to attend any of their customs when invited. After all it is the people that make any project success. And most of the people outside will be interesting to know your language or culture, do not hesitate in sharing your knowledge/skills and spending some time on this, although it is not the part of your project but it will develop into a very strong tool to progress your project in right direction with right pace.
Be as them: Don’t act as a foreigner. Act as one of them. Show the management that you are the part of their organization and it is not only their but your project also. Keep top management involved in all your decisions and actions. Seek their help where required.
Celebrate: Celebrate any small success that will motivate them and speed up the implementation.
Do let me know anything that I might have missed to mention.
Roberts: Project Manager
Laura: Project Manager
Julia: Project Leader
Sandy: Quality Head
Toby, Andy, and Victor: Software Testers
Linda, Lisa, Paula, Edger and Alex: Module Leaders
Storyline: Project Manager, Roberts has newly been shifted to manage various projects (running or in pipeline) as this post got vacant after the resignation of Laura. Roberts prior to this was head commercials and this is a new avenue for him, he now handles many projects at a time and is okay with it. Julia his Project Lead for the project in question is leading a project for the first time. She has been involved in project implementation earlier but as a module leader and not the project leader. Sandy is the Quality Head and is responsible for the QC of any software that goes to any customer. Toby, Andy and Victor are reporting to Sandy. Linda, Lisa, Paula, Eder and Alex are working on different modules of the project lead by Julia.
Laura was a cherished leader a year back, as she was able to grab quite a good number of overseas projects. But somehow she failed to manage the projects as each project got overscheduled due to one reason or the other. Ultimately, Laura had to admit her defeat, and she resigned from the company. But does it solve the problem? No, the company by now is in deep troubled waters and has a big question mark on its reputation for the existing and future projects. Julia and the team were already designated by Laura and were sent to the client site for implementation. Sandy and Laura were not in good terms as Laura always wanted Sandy to bypass the defects in the product and wanted him to give a go-ahead for implementation without worrying about the completion of the product (or ignoring unfixed severe bugs reported by Sandy’s team).
Roberts did not disturb or made any changes in this particular project team lead by Julia. Julia and team are back from customer site without completion of the project as some issues arose which could not be handled by Julia. Few of the outcomes of the team and team leader that are back from the project (still incomplete) are:
1. Toby, Andy and Victor were not happy with the state of the software when it was taken for the implementation at customer site
2. Sandy had clearly told management in advance that there are chances of project failure
3. Linda, Lisa, Paula, Edger and Alex are not happy with their leader Julia as they feel that Julia who is also heading a module of this project is always over occupied in her module and has no time to address project or other module issues
4. Julia was not able to convince customer management about the product
5. Many business rules that were built in the product shockingly absolved as the wrong requirements understood by business study team.
6. Julia is not happy with her team members – Linda, Lisa, Paula, Edger and Alex as she feels they are not listening to her and doing whatever they want to do without consulting her
7. Julia and her team were all sailing their individual boats in different directions at customer site.
8. Julia feels she is the team lead just on papers otherwise it is being lead by someone else.
9. Roberts feels that the initial system study done at the time of Laura was not satisfactory and accurate.
10. Roberts also feels that even now when the team goes back for implementation of the same project, there will certainly be many issues.
11. Roberts is not sure about the timeline or schedule of the project, that whether this time the team lead by Julia will be able to finish the project in scheduled timeframe, but is not worried as he knows that the management will accept his failure because he is not the one who started it. He feels secured even if this project fails as he is sure that the management will not him fully responsible for this, or even if it happens he can pass on all the blame to Laura who is not with the organization now.
Ultimately how do we ensure the successful completion of this project in stipulated time is a big question (although it is not disturbing anyone as everyone has already a bundle of reasons in his pocket)?
Does someone have the answer?
Testing does not ensure risk-free software. There is nothing called hundred percent testing. Be it manual or automated testing but all testing is in a way manual (even in automated testing the test scripts are altered manually) and hence it is for sure that the testing will never be able to cover all possible test cases in a practical situation if compared to an ideal situation. Every time my team comes to me with a final report of bugs of any software tested, I am able to point out many cases that they say “yes, it should have been done! But we missed it or skipped it!”. Sometimes some cases or scenarios are intentionally dropped by the tester based on just his imagination that this case has no significant role. But if looked from another angle, the same intentionally dropped case appears to be a very important test case that is must to conduct to understand the result of an important behavior of the software. Like security, testing also does not ensure zero percent risk after it has been done. Only thing it ensures is minimizing the risks and vulnerabilities to the best possible level. Software Testing is the process to reduce or minimize the bugs or defects hidden in the software.
For further tips on how to achieve this you can go through my previously posted blog as “Twenty ways to ensure complete coverage of software testing”: https://itknowledgeexchange.techtarget.com/quality-assurance/
To ensure complete coverage of software testing, testing team must be careful about certain activities that are part of the process. If the software testing is not complete as per the business and customer requirements, it could have a severe adverse effect during or post implementation of the software at the customer site. The more is the coverage, the less are the chances of any bugs passing to the implementation phase. So, to ensure the complete coverage of software product and to find the maximum possible bugs or defects, testers have to ensure following steps:
1. Ensure that the documents defining the business and customer requirements are complete and correct. How to do that can be taken up in a separate topic.
2. Ensure that the development team has clearly understood the documents.
3. Ensure that testers themselves have thoroughly read and understood the documents.
4. Prepare a clear cut scope of testing based on product documents.
5. The strategy and Test Planning is as per system requirements.
6. Decide test methodology and test tools (if any), and test schedule.
7. Prepare Test Cases based on business rules and customer requirements.
8. Ensure that the test cases are extensive and sensible to cover the complete requirements testing.
9. Ensure that during testing no changes in the test environment (coding etc.) is done by development team.
10. Ensure that development team representatives (1 or all) are present during the complete testing.
11. Create Test Scenarios based on test cases.
12. Observe the result of each test case and record it accordingly.
13. Prepare a comprehensive and detailed test report explaining each of the test case, scenario and its result elaborately.
14. Ensure that all bugs reported should make sense (no duplication/overlapping of scenarios and no repetition of bugs reported)
15. Ensure that the complete testing finishes as per schedule.
16. The final report submitted should clearly state the areas not covered under testing, reason for the same and its impact on the product.
17. Simulate any bugs that are not clear to development team.
18. Ensure that you have a tentative plan from the development team when they are fixing all bugs and submitting it back to testing team.
19. Verify all bugs fixed and ensure that the development team is sitting with testers during verification.
20. Prepare the final report and submit back to Development team, giving the status of each bug fixed, as verified as fixed or not fixed. Report any new bugs arrived in the software while fixing these bugs.
3 Years back there was no QA department in the organization. The development manager, John, was the king. Nobody was there to check at what stage his development projects were, all used to go by his words. One day he was asked – hey, what is the status of your software that you are developing for Canada? Oh Sir, he reported his boss, it is all ready. When do you want to send it to Canada with my implementation team? Ok, if it is ready, let us book tickets for coming weekend so that your people land there, settle down in the hotel, enjoy weekend, and start implementation on Monday – his boss told him. Ok Sir, he replied.
John called his development team responsible for Canada project, takes stocks of situation, gets to know that still many screens, sub-modules, units are under development and at no cost finish by weekend. Oh, but I have told boss that everything is ready – he tells his team. Don’t worry sir, his team tells him, we will finish it there. Ok, he replied.
So the team goes to Canada, the target of finishing the project is not met, users are not happy with the product (it was not a complete product), management got angry, stay is increased, but even with the extended stay, the project does not take proper shape and hence finally the team is sent back.
What now! An emergency meeting is called. Reasons for project failures are found out. It comes out that John had projected a rosy picture to the management without showing the existing thorns in it.
And the QA department took birth with a resolution that no software will move out to any country without a final clearance from QA.
3 Years have passed, John’s life got miserable in the beginning. Each of his products had to undergo 5, 6, 7… cycles of testing and re-testing by QA. Initially QA reported why the product has come for testing when it is not complete? Why it is declared as complete? Why during testing, the development team keep doing something or the other in the same product instead of giving time to QA etc. etc.
Ultimately maturity arrived. The products only are shipped when they are sensibly completed and verified/tested.
The Development Manager finally says to QA Head – You exposed me, no problem, your team tortured me (my testing and re-testing), no problem, ultimately I am the happiest man now because now when I declare about a product that it is complete, and my boss stares at me, I add immediately, and it has been cleared by QA after x number of testings finally. I am sure it will not fail now as usual.
John’s life has become tough but he is the happiest man on this earth, you know why, because now he is sure that none of his projects will overrun or fail.
Software Developers are not doctors and that is why are not in the habit of doing ‘first time right’ coding. All newly developed software contains bugs, bugs and bugs. Developers have this mindset that no software can be written without a bug in it. They are convinced that bugs are integrated part of coding and involuntarily that are bound to place them here and there in their code while writing a software. If doctors start working like this that in every operation they leave some important activity ‘undone’ and have a privilege of getting each operation audited by another doctor (specially meant for auditing a patient immediately after operation), the auditing doctor will find out the shortcomings in the operation, report it to the doctor who operated upon the patient, the patient gets operated again, again auditing, again operation, so on and so forth. Think of what will happen to the patient in this case. And I think the patient will prefer to die on his own than in this manner.
The same thing happens to the health of software when some bugs are embedded in it, found out by testers, reported to developers. The developers fix those bugs, in turn produce some other bugs, again it comes to testers. Another bug report, another fixing, another different set of bugs production, so on and so forth. What happens to the health of the coding in this case?
If software could shout and cry!
A point to think upon and act upon!
To me sometimes it appears, may appear funny to some, and logical to others, that there is a strong resemblance in a Poultry Farm and Software. As different Hen in a poultry farm keep producing eggs day in and day out similarly different modules in the software keep producing bugs.
Do we agree then that as hatching eggs produce more hens, similarly fixing of a bug produces more bugs?
Eggs which are not hatched but consumed may then be related to bugs which are there in the system but have not been identified and remain floating in the system. As is known that many bugs cross testing phase without being identified and come into light during live run (i.e. post implementation phase), sometimes after months of successful completion of a project.
Do we require such software – productive (producing bugs) like a poultry farm? Or we need good programmers to make good software and not turn it into a poultry farm.
I invite your valuable Feedback/Comments please.
Equivalence Partitioning Testing Method is a method used in Black Box Testing (see: https://itknowledgeexchange.techtarget.com/quality-assurance/what-is-black-box-testing/) for the purpose of finding out minimum set of data for testing for valid and invalid inputs.
The purpose of this testing is to:
1. Ensure that the software acts according to the type of data is entered, i.e. for a valid entry it should accept the data and for an invalid entry it should return the appropriate message/alert.
2. To ascertain that minimum data is used for similar results (the result of a test case could be valid or invalid)
For a group of values, if the software behaves in same manner, the group or set of value is called ‘equivalence class’ or ‘equivalence partition’. So primarily for an equivalence class the software will behave in the same manner and shall produce the identical result. Different equivalence classes will make the software behave differently and shall produce different result.
The tester has to be careful in carving out the different equivalence classes so that one value is taken from each class to ensure complete testing. The testing in this way is completed with least efforts/ test cases.
Let us take an example to illustrate:
In a particular Airline based on number of flights taken in a year, a membership is offered with different privileges as below:
If the number of flights taken is between 5 and 10, the membership offered is ‘Silver’
If the number of flights taken is between 11 and 15, the membership offered is ‘Gold’
If the number of flights taken is between 16 and 20, the membership offered is ‘Platinum’
If the number of flights taken is between 21 and 40, the membership offered is ‘Diamond’
Based on above all inputs between 5 and 40 are valid, less than 5 and more than 40 are invalid. And also any non-numeric value is also invalid.
So there will be 7 equivalence classes viz: 5-10, 11-15, 16-20, 21-40, 40 and non numeric and thus one value from each equivalence partition will give you the complete test coverage.
Black Box testing of a software is based on two primary things –
1. Customer Requirements Document
2. Functionality of the software
Purpose of the Black Box Testing is to ensure, that means, that –
1. The software meets are customer requirements as specified in the Customer Requirements Document (business needs)
2. And the software meets user requirements (user needs)
By user requirements we don’t mean that it has to meet every user’s requirements but more appropriately the appropriate usage requirements.
Black Box Testing is the most widely used testing and is used to used to check if the product meets per se customer’s all requirements. The tester is not worried about how the software has been built, what queries have been written, whether the coding is technically optimal or not, but focuses more as if the customer is working on the system and feels like customer when he tests the software.
A basic requirement of Black Box Testing is that nothing should be presumed about the functionality of the software while testing the software, rather the tester is supposed to check each and every functionality of a unit, module and the product as specified in the customer requirement document. So the fundamental of this testing is the Customer Requirement Document, based on which the appropriate test plan, test cases and test scenarios are built and ultimately test results are drawn as a Test Report (or Bugs Report).
For building various test cases in Black Box Testing certain techniques are used like:
Boundary Value Analysis (or Boundary Value Testing), see my earlier blog on this:
Equivalence Partitioning: This is a technique of testing with minimum of test classes or data input to identify between valid and invalid input.
Prior to starting actual testing, a tester must verify and validate:
1. Customer Requirements
2. Functional Specifications of the software
3. Test Plan
4. Test Cases
5. Test Data
6. Test case execution
After above validation, the test cases are executed, results are recorded and reported to development team.
Way back in 1960s softwares were being produced but there were no established standards or software engineering practices. Poor design and Regular hardware upgradation left many types of software useless and thus a strong need arose in the industry to provide effective and reliable software developers who could pace up with regular hardware advancement to produce faster and highly reliable softwares.
All this, plus the increasing dependency on softwares increased not only demand of good developers, but also a requirement of software testing took birth, may be, not initially in a very strong and structured manner but it did.
Software engineering practices that were taking place at that time were not formulated, or structured in an efficient manner. Some of the bad practices were no measurements or metrics were in shape to measure up –
Development lifecycle or product lifecycle
Lack of planning, monitoring
No control on development schedules
No control on customer requirements changes
No reviews of documents
Failures at customer end during or post implementation due to product incapabilities, schedule delays, revenue losses
Therefore a need to test the software came into existence to:
enhance all these processes,
increase the reliability of product,
deliver a quality product,
deliver in time
Let us take an example that a code is to be written for entering employee details in employee master table. There are certain conditions for validating while entering a new employee for a specific post that his minimum experience at the time of joining has to be 5 years but not more than 10 years. The coder has written the code and designed the screen to input data satisfying these conditions. Now this code comes to tester for functional testing to verify if it meets the desired requirements.
How tester is going to confirm it?
A tester has to write test cases by understanding the following:
2. Valid entries
3. Invalid entries
4. Inside edges of the boundary
5. Outside edges of the boundary
6. Boundary points
Let us see what each one is and why it is important as the test cases are going to be built on this basis:
1. Boundary: The boundary as per requirement here is 5 to 10 including both values.
2. Valid Entries: The valid entries shall be all values between 5 and 10.
3. Invalid entries: All values other than in point no. 2 are invalid
4. Inside edges of the boundary: Inside edges of the boundary in this case are 6 and 9
5. Outside edges of the boundary: Outside edges will be 4 and 11
6. Boundary points: Boundary points shall be 5, 6, 7, 8, 9 and 10.
So the test input data shall be: 4, 5, 6 to 9, 10, 11
Definition of BVA or BVT: Boundary Value Analysis is the process of selecting test cases (or test data) by understanding boundaries that differentiate between valid and invalid conditions. Tests are run to check the inside and outside edges of these boundaries, in addition to the actual boundary points.
During Software Development certain business rules and disciplines need to be embedded in the software based on the customer and business requirements as specified by customer during the initial business study. These are required so that the software meets customer expectations and needs. A Document is prepared based on this needs which is known as SRS (Software Requirements Specifications).
To prepare SRS a group of professional from the organization who is responsible for this software development, visit the customer site to have a first hand look at their processes, documents, functions and requirements. SRS is prepared as detailed and explanatory as possible because this document is handed over to the development team who will be responsible for developing a good software matching customer needs and meeting customer requirements.
SRS in a way is the foundation stone for good software that has to take birth based on it. It is very important to meet the right people at customer location who are deeply involved in these processes in their respective fields. A detailed study of each process, supported by relevant documents, forms, reports need to be collected. Process owner must devote ample time to the team member who has come for business study so that he in turns documents it well as part of SRS.
Finally each Team member compiles, edits, verifies and prepare a document that is called SRS.
In Software Testing we need to devise an approach that features a gradual progression from the simplest criteria of testing, to more sophisticated criterion via many planned and structured steps, each of which brings incremental benefits to the Project as a whole. That way, as a tester masters one skill or area of testing, there will always be something worthwhile to try next.
Important Aspects are:
Trust: Testing and development teams have to build a trust-trust relationship in each other and in the product under development.
Constructive Communication: The initial communication of testing team with the Development team has to be quite open, transparent, constructive and encouraging. This is the phase that decides about the course of action for testing and any alternatives. The future of software, its accuracy and the successful implementation at customer site afterwards will be the key factors emerging out of it. Don’t make this communication too formal.
Documentation: Ensure that testers have sufficient documents in hand before starting the analysis and testing of the product. Appropriate analysis of the documents is very important to remove and ambiguity/confusion/loop-hole in understanding the customer requirements. Test cases and scenarios will emerge out of these requirements.
Open Mindedness: A tester has to go with open mind in all discussions only then he will be able to understand developers, development process, customer requirements and then he will plan on the testing requirements accordingly.
Variables: Keep all variables in the loop that are important to the testing and successful product development. Different variable could be developers working on different modules, documents, testers etc.
Learn from Past: Keep learning from past experience and try not getting caught in any mistakes that have been made earlier. A common saying is – “Never repeat the same mistake again”. It is a mistake to go for testing of a product in a crude or unstructured manner. It is a way of improvising your testing model over a period of time with more and more experience.
Indentify Shortfalls in advance: Be clear about what is in scope of testing and what is not. Clearly define what is not going to be covered in testing and what could be the impact of them on the product. Foresee and uncertainties/delays. It is important to understand the benefits you are going to get well in advance rather than getting no benefits at the end of this whole exercise.
Keep Interacting: A tester has to keep interacting with the developer throughout the testing process. Ultimately the goal of both sides is to have a better product at the end crossing all barriers of conflicts, bugs, faults or defects.
Your valuable comments are most welcome.
Assume that after functional testing of a software following is the summary of count of bugs:
Total Bugs : Tn
Severe Bugs : Sn
Critical Bugs : Cn
Desirable : Dn
Here Sn + Cn + Dn = Tn
And let us assume that the estimated expenses of removing of those bugs (based on developer’s direct and indirect cost) are:
Average Cost of removing a Severe Bug : $S
Average cost of removing a Critical Bug : $C
Average cost of removing a Desirable Bug: $D
Then we can say that the Average cost of removing a bug (Severe/ Critical/ Desirable) : ($S + $C + $D)/3 = let us say $B
Here Total cost ($T) of removing all bugs is: $T = ($S * Sn) + ($C * Cn) + ($D * Dn)
This will be for one project and similar data van be obtained for various other software projects under development in the organization.
The graphical or tabular comparison of Tn * $B to $T for all these projects is the linear approach of Cost Estimation of Bug Fixing or Defect Removal.
At times Testers need to make the door open (friendly or forcefully), shut by the Project Manager. If door is not opened at appropriate time it may hamper/delay the progress (or ill-effect even the successful implementation) of a project. And those times could be:
Initial Documents: When the Initial documents are incomplete or ambiguous, not explaining completely about the user requirements or the business rules to be embedded in the software. This is the most crucial situation and if not handled immediately – will build up a wrong product – built by right people and tested by right people.
Test Cases: At the time of preparation of test cases and test scenarios, a tester may consult Development Team members to confirm if all aspects are covered and nothing is missing.
Testing: During testing it is always better if the respective developer sits along with the testing team, so that the bugs are confirmed hand-to-hand. This shall not require simulating the bugs at a later stage (and this will definitely save time).
Testing Report: After submission of testing report, do confirm with the concerned developers/ Project Head if any clarification is required.
Testing Feedback: Development Team is supposed to send its feedback on test report in a stipulated time frame, remind them if it does not happen.
Feedback: Check the feedback thoroughly and if there is any point raised by testers which is missing feedback from developers or if any feedback is not clear, do check it with the developer’s team.
Verification: Insist on development team sitting with testers while verification of closure of bugs. Record all closures and any new bug generated or old bug reopened during verification.
Creator vs Destroyer: As a law of nature – creator can not be destroyer. Similarly in software, a programmer can not be critic of his own code.
Constructive Criticism: A good programmer will always love and admire his code, but will never have that ‘third eye’ (that a tester has) to locate bugs in the code. A programmer is fond of listening to ‘praises’ for a well written code than listening about the shortcomings in the program written by him.
Vision of a tester: A programmer will always think of the pass conditions in most of the cases and will bother less about the negative scenarios. A simple example could be that the programmer has to write a code to accept a password that has to have minimum one special character, two numerals and length not less than 8 characters. Once this condition is fulfilled, the programmer will be more than satisfied – but what about n number of different test cases that a tester creates just to check this simple requirement.
Fear Factor: Even sometimes when a programmer knows about a flaw in his coding, seeking a major change and coming into his notice at a later stage, he will like to oversee it and also try it to be to be overseen by others. He may fear that highlighting a shortcoming in a code written by him at this critical juncture may loosen up his superiors trust in him, which is usually dear all everyone at any cost.
Thinking Pattern: Walking on the Same Road everyday makes one perfect on that road but what about other road patterns, walking backwards etc. Continuous Coding, Coding and Coding make him a perfect coder but only a perfect coder.
Documentation: Usually a coder writes code perfectly but with very little or no documentation. At times, he himself forgets what logic has he written for a specific requirement and then he has to refer to his own code (and waste some time) to find it out.
Self Testing: The confidence (or over confidence) stops a coder to introspect or examine his own coding by doing smoke testing or jotting down some test cases and scenarios to run them and get a pass through for his own satisfaction.
Tight Schedules: A programmer is always having his task basket full and has delivery schedule tight. That also becomes one of the reasons for passing on some bugs (in a hurry).