Software Acceptance or User Acceptance is an exercise that happens once the software has been cleared by QC. So far software has been tested internally only (by developers and testers). But customer also would like to test it and accept it prior to finally start using it. The phase is also important in case if any user requirement has been skipped, misunderstood or wrongly built in the software. Clearance by development team and testing team does not ensure that the product is meeting all requirements specified by the customer correctly and appropriately.
User would like to go through each functionality, process and rule with real data to ensure that the software is behaving as per the needs specified. Users during acceptance testing is supposed to run through each screen built and by trying data of both types – right and wrong to ensure proper handling of exceptions in the software.
Once the key users find software functioning fine, customer (management) accepts the software and then it is finally deployed or implemented.
User acceptance can take place at either of the locations – customer or vendor.
Software Testing is the process to ensure a bug-free, fully functional software meeting all customer requirements. Testing starts right from the development stage. The developer himself is responsible for unit testing, smoke testing and code testing while he is building the code. Once different units, built by different developers, take the shape of a functional product, a need arises to test the product as a whole, each of its unit, each module and sub-module. This part is handled by a separate team known as QC, QA pr Testing Team.
Software Testing is very important to ensure:
2. All business processes have been built appropriately
3. All business rules are working fine
4. Functionally the product is ok
5. Each unit, sub-module, module and finally the product as a whole is meeting all software requirements
6. Exceptions are handled appropriately
7. User manual and software match
8. Software requirements, User manual and software match
Based on software requirements, business process, business rules, software design and customer specifications, the software coding begins. Project manager distributes the coding among the development team members with clear time frame instructions. Coding will begin in two manners – phased or sequential and parallel.
Phased or sequential coding is one in which the next coding part is not possible unless the first part is complete.
Parallel coding is coding that can be started simultaneously for different independent units between which the dependency is not too high.
Software Coding is the process of conversion of software design (concept) into reality.
Software design is the stage that comes next after freezing software requirements. The requirement specifications are given to the team of developers headed usually by a project manager. The project manager or the technical lead decides on the software architecture, OS, front end, back end, database etc. System design or ERD (entity relationship diagram) is prepared once the initial database design is clear with the table structures to be built in the database.
Software design is nothing but the conceptual design of the database, tables and table structures, relationship of various fields of tables with one another. Once conceptualized, it is discussed among the team, make any changes if required and then freeze it. Once the concept is finalized, the database is designed by creating tables and building relationships. The next phase that comes after is the coding.
Security concepts will vary from software to software except few generic requirements that will remain standard for most of the softwares. The major varying requirements will be dependant on following factors:
b. System specific: The other major factor deciding on security concepts to be built in the software will be system design”. What architecture is chosen, what technology, what database, what front end etc.
2. Risk Factor – The gravity of risk involved and what level of security is required will decide on the security features to be built in the software
3. End User – What level or type of user is going to use the software
4. Money Matters – If there are any money related transactions in the software will require a different set of security concepts.
5. Statutory requirements – The outward or inward connection with other legal/non-legal agencies will formulate the specific security specifications. Also will depend on any statutory requirements to be met by the software.
Software requirements are the requirements that are given to the developer based on which they build new software. Software requirements could be functional or technical in nature. Broadly software requirements can be divided in two parts as – business process and business rules. Software requirements lifecycle can be defined in following steps:
1. Initial discussions – in this phase the top officials of the two parties (customer and vendor) discuss broadly the business requirements that are required to be built in the software.
2. Detailed discussions – this phase comprises of a team from vendor side that goes to meet the respective process owners of customer to understand the details of each process and the rules to build in.
3. Documentation – the requirements (process and rules) in the above process need to be documented in a clear manner because this is going to be the basis of software.
4. Forms and Reports – any forms and reports that are in business practice and have to be built in the software will have to be collected.
5. Sign Off – Never forget to get the complete document signed off by customer key users and top management
Here are some quick guidelines for a tester. As and when a new product (software) come to a tester for testing, (s)he should keep following things in mind to ensure the full justification to the purpose of testing. The purpose is to ensure a complete and correct testing of the product.
The guidelines are as below:
1. Assume yourself in customer’s shoe: Treat yourself as customer to run/use the product as they are going to use.
2. Walk with open eyes: When you walkthrough the product, keep your eyes open to observe the minutest variation/deviation in the behavior of each unit of the product.
3. Educate yourself before start walking: Before you start testing, ensure that you have thoroughly gone through the customer/business requirements documents.
4. Don’t compromise with the Docs: Ensure that the documents are complete in all aspects, as any ambiguity in documented business process/rule will have marred the purpose of testing and would lead to wrong results.
5. Don’t cross the red signal: If documents are incomplete or ambiguous, never start the testing.
6. Ensure all passengers are on board: The documents you are studying must be vetted by the customer and project manager before it reaches you.
7. Don’t sleep during flight: Keep your Pen and Paper or excel file ready as soon as you start testing. Keep noting each and every test case results.
8. Don’t take snacks at Food time: Place appropriate screen shots wherever required for your each test case results.
9. Ensure you have a right pilot in place: The project manager/ project team should be available to you while you are testing so as to ensure you all are travelling through the same path.
10. Appreciate the Air Hostess: Appreciate at places where a good job has been done by the developers by putting your remarks.
11. Mistakes are unintentional: Don’t blame the developers for the mistakes (bugs) in the product, purpose is not to punish or fire a developer.
1.Focus on Quality Policy, Strategy, Test Plan, Test Cases.
2.Maintain heartfelt sympathies with developers who are developing Bugs (and their impact!).
3.Raise your anxiety higher for any software coming to you for testing.
4.Your first and foremost priority is to prevent the damage a bug spreads across the product.
5.Investigate the damage (impact of bug, through test cases).
6.Ensure safety of software by ensuring that you are able to catch all bugs.
7.Keep visiting developer’s den during testing to highlight the severe bugs found so far.
8.Work like craftsmen with high precision to attain micron-order accuracy in testing.
9.Be absorbed in your work and not even notice anything else happening around.
10.Be proud of your testing skills.
11.Have uncompromising approach towards ensuring bug free software.
12.Don’t worry about the rising count of bugs in the software (that ensures your job more secured).
13.Use your experience and documents business requirement, business processes and rules in stabilizing software.
14.Mount a fundamental response from developers.
15.Take a leading role.
16.Issue a crisp and clear report of bugs encountered with elaborative screenshots.
17.Ensure through coordination that developers nullify or atleast minimize the impact of bugs reported in your bugs list.
18.Carry out tasks at hand including measures to cope with developers for removing bugs.
19.Respond steadily and quickly to any changes in the documents, business requirements, process, rule or software design.
20.Deadlines remain poised on knife-edge, focus on materializing each task. Remember the old famous saying – “You can’t eat an elephant in one go”.
A bug right from its insertion into the software till its removal crosses through different stages. These different stages of a bug formulate bug lifecycle. Each stage has an action and meaning connected to it. Various stages of a bug are:
Unconceived: A bug that has to come into a future software, not even knowing who is going to write it, and in which software.
Unborn: A bug not yet taken its shape, still lying in the mind of a developer.
Unconcealed: A bug residing in software but unrecognized by a tester.
Open: Any misbehavior in the software identified and recognized by tester is recorded with “Open” status.
False: Not actually a bug but marked as bug.
Disguised: A bug existing but identified with a different behavior.
Accepted: The bug accepted (as bug by project manager and its development team for a fix.
One time: A bug encountered by testers, but not encountered/simulated again. Such bugs are required to be fixed unless encountered again.
Not Accepted: Project Manager or development team may refuse to accept a bug in case it is not critical and seeking too much time to fix it, it is meeting customer requirements, tester has not understood the business requirement well and identified a wrong bug, or in case the same bug has already been reported.
Not to be fixed: The reasons would be same as mentioned in “Not Accepted”.
Pending: A bug accepted by the developer but may not be fixed immediately. It could be due to non severity, other priority or any other reason.
On Hold: Some bugs need to be discussed with customer, or management. Or the reasons may be as mentioned above in “Pending”.
Fixed: As soon as a bug is fixed by the developer, it is assigned as status as “Fixed”. A programmer must always doubly ensure before marking a bug as fixed.
Closed: The bugs marked as fixed once verified by tester and are found fixed are marked with the status “Closed”. Tester must doubly ensure before marking any bug as closed. Tester must also ensure that the bug fixed has not generated any other bugs.
Re-Open: A fixed bug found not fixed by the tester is to be marked as “Re-open”, which goes back to the developer for fixing.
There are three ways of functioning for any software organizations. First ways is to develop software and deliver it into the market. The survival for these sorts of organizations is too difficult and short. Second category of organization has a well placed “Quality” department in place. The quality department is responsible for ensuring a bug free product going out to the customer. The third category of organizations have a well identified and well structured team in place to ensure that every next production of software is going to have far less bugs as compared to the earlier produce. Most software organizations fall in category number two. Very few fall in category one, as organizations falling in this category will either not be able to survive for long, or will live on a very little customer base and profit. This is the third category that we are talking about – that falls in the bracket of “Bug Control Management”.
This organization lying in the third category will be continuously working towards improvising the processes of product development, ensuring lesser and lesser bugs in each of the next release of the product or new product. The progress of this team will purely be objective subscripting the improvement in every next product release. It does not mean that quality department will have no role now. Rather quality department will have a new meaning and more valuable role to perform.
But point to ponder here is – is it just a void promise, a flawed process or process insanity. Or is it just the darker aspect, brighter aspect is being felt by the organizations who have or are in the process of adopting BCM.
Bug Management and Bug Control Management are two separate aspects in software development. Bug Control Management seeks a higher maturity level in terms of organization, developers, project managers and all others involved. On the other hand, Bug Management starts in organizations with low maturity level. Organizations that attain the Bug Management level and keep it for years, stop maturing and keep a particular level of maturity sustained within them. There are organizations that move (over a period of time) from Bug Management to Bug Control Management. These organizations keep improvising and seeking improvements in their processes and procedures.
Usually a software organization life starts with software development. Gradually with the increase of business, increase of experience of developers, and increase in customer expectations an increased level of reliability in the software from all directions is expected. To cope up with this pressure, the organization creates a department called as Quality Assurance. This department takes care of testing of software, finding out bugs and getting it verified after the bugs are fixed by the development team.
Bug Control Management is a management initiative to involve all concerned to form a steering committee. The responsibility of this committee is to ensure that lesser and lesser bugs are generated in the software while development.
In simple terms BLC or Bug lifecycle can be defined as the different stages of bug. BLC or Bug lifecycle will consist of all these stages starting from the birth of a bug till its removal from the application. The various stages of bug lifecycle or BLC can be listed as below:
2. Bug identification
3. Bug behavior study
4. Impact study
5. Bug classification
6. Bug admittance
7. Bug removal assignment
8. Bug removal
The birth of bugs is unintentional, uncontrolled and hidden. A bug is prone to take birth while a code is being written. A bug is not a bug until it is identified or recognized.
BLC or Bug Life Cycle starts with an unintentional software bug or unwanted behavior and ends when the assigned developer fixing the bug.
This is in continuation of my previous blog where I listed the 10 most important must-dos for customer to have clear software requirements in first go. Ten most important must-dos for vendor to have clear software requirements according to me would be:
2. Ensure that management selects the right person(s) for the purpose.
3. Ensure that management at back office is continuously involved in the whole process as per plan.
4. Ensure that customer management is clear in their goals to be achieved by this software.
5. Ensure that customer has identified the right people who are actually the process owners and not the subordinates of the process owners.
6. Do not hesitate in analyzing the each sub process station.
7. Ensure that analyst gets process owner’s ample time to document each and every process requirement well.
8. Don’t trust on your mind. Document along with the understanding of the process or soon after it. Keep getting it vetted by the process owners.
9. Keep updating customer management after the finish of each process.
10. Don’t compromise with time and documentation.
Note: Please do not hesitate in giving your valuable add-ons. I might have missed out certain important points (if! Any!!).
Having clear software requirements is very important as it is going to be the foundation of the software to-be-built. Having clear software requirements in first go is equally important especially in case of overseas client. Few things are very well known to all:
2. Documents collected and prepared are the vital asset of software requirements.
3. Selection of right person for the customer/software requirements study is very crucial (and sometimes it hurts at a very late stage)
4. Travel and time is a cost that is affecting the overall Project Cost, Margins and Profits – irrespective of whether customer or vendor is bearing it in the beginning.
Keeping all this in mind, the 10 must-do to have clear software requirements in first go, which I am listing below. There could be many more (which I would like to be added in the comments section), but these come to my mind as most important:
2. Ensure that management is continuously involved in whole process.
3. Ensure that the vendor representative(s) coming for the purpose is the right person(s). They should not only be the master of their domain but should have fairly good number of years of business exposure too.
4. Ensure that the process owners are the right persons to explain the process.
5. Ensure that the process owners have their process charts ready with them, the document is going to help really.
6. Ensure that the processes are sequenced in proper fashion.
7. Ensure that the time should not be the constraint for process owners. Devoting 6-8 hours for 2 days is always better than devoting 2 hours per day for 10 days. After all more time spent by the vendor representative is equal to more cost to the customer (overall project cost).
8. Ensure that the analysts (vendor representatives) keep updating their documents during the understanding of process or at the finish of a process. Let not a lot remain in the mind and be documented later.
9. Ensure that both the managements (vendor and customer) finally sign the document finally prepared (both have equal ownership on whatever has been documented).
10. Do not get bogged down by the size or volume of the document. Let it is as elaborative and explanatory as possible. Just be sure that it is crisp and to-the-point. Attach hand written papers, reports, formats, legacy manual with specific markings of important features required to be built in the new software.
Note: Please do not hesitate in giving your valuable add-ons. I might have missed out certain important points (if!). A similar set of guidelines for the vendor end, I would be posting in my next blog.
Broadly there are two main loopholes in case of unclear software requirements – vendor and customer. Unclear Software Requirements could lead to disaster not only for the vendor but for customer also as I stated in my previous blog. To vendor it may cost money, time, reputation, business and at worst the closure of business. To customer it costs time, money, employees trust in its management, management confidence in selecting a vendor and at worst a decision to not to streamline their business in terms of a new software.
The main reasons of unclear software requirements could be:-
The vendor representative’s tenure is too short at customer site to study its business and requirements
The vendor representative is not curious enough to dig down each process at customer site
The vendor representative being poor at documentation
The vendor representative thinks collection of relevant forms and reports is a waste
The vendor representative keeps more in his mind than documenting (the business rules, processes and requirements)
Business clarity is delivered by the person other than actual process owner
Process owners too occupied
Process owners depute their subordinates and perceive they will explain the correct picture
Management before signing the final documents do not vet each process and business rule
Poor management’s involvement
Wrong perception that vendor will build a strong software that will meet all their requirements on its own
Unclear Software Requirements could lead to a disaster. It may lead to a dead end or deadlock from where there is no way back, with zero possibility to move ahead. The situation may cause the company a huge cost in terms of understanding the requirements again, rebuilding the software, re-testing, re-configuring and re-implementation. The cost involved sometimes is small enough that the company is able to absorb it or adjust it, else sometimes it is so huge that it effects not only the current project(s), but has a recursive disastrous effect. Sometimes companies are never come out of this crisis and gradually have to bear huge losses.
Usually it is lose-lose situation for both the vendor and the customer in this scenario. The loss at both the ends could be in terms of money spent on customer visits (travel and stay cost, borne by vendor or customer, but it is spent and has gone waste), time consumed by vendor in development of the product based on these unclear or incomplete requirements, the reputation of vendor (which is somehow irreversible if the name goes bad, and has recursive effects), confidence of staff at both the ends (customer and vendor), management’s trust (in each other).
The reasons for this situation could be many but the ultimate result is always the same – DISASTER.
In normal scenario when an order is finalized between a vendor and a customer, for building new software by the vendor, the payment terms are set in such a manner that some percentage of the project cost is paid soon after the completion of customer requirements study, generally on finalization and submission of customer requirements document (CRD) by the vendor to the customer. The completion of this phase shows that the vendor has clearly understood the customer and business requirements and is capable enough to build all those requirements in the software product that he is going to build for the customer.
Customer’s ultimate goal is to acquire or purchase the final product that fully meets its functional and business requirements. Then why customer should pay out of his pocket for something at a stage that is of no use to the customer. The requirement study is the need of vendor and not the customer. Customer finally needs a product.
What if the product built is incapable of meeting customer’s requirements. Then why not customer pays to vendor just once at the time of product delivery, User Acceptance and implementation completion?
Do we buy a complete car or pay in parts – for steering, for body, for brakes, for seats, for belts… and so on…
Just a thought…
Senior Management including CIOs, QA Heads, CEOs, Development Heads, Project Heads, Customers, Vendors and/or sub-contractors related to software Organizations across the globe has repeatedly displayed their concern over the Bugs Management Process from time to time.
These concerns could be grouped as below:
2. The test results do not lead to more effective coding: The coding is always prone to create bugs (programmers are always as overconfident as ever).
3. The identification and quantification of a Bug is highly dubious: All the efforts by testing team in identifying and categorizing a bug is not always able to meet the seriousness it is meant for. Moreover sometimes a bug becomes an issue of debate over its category (critical/ severe/ desirable etc.).
4. Quality managers and testers often appear to have a limited knowledge of the code and business concepts and related issues.
5. Excessive focus and reliance on quantitative bug analysis is dangerous.
6. Bugs with a low probability occur more often than would be expected.
7. Bug management leads directly to bug avoidance.
8. Bug prioritization by Test managers is usually a simple ranking showing little or no understanding of the business process or customer requirements.
Most of the time the goal of testing is to diagnose the cause of bugs and produce better solutions. Instead the focus should be more on avoidance of bug generation at the code level so that least changes happen in the code once generated.
In my previous blog we tried to understand what can be termed as a Bug. Here we will (in continuation to that) try to find out the reasons of causes of generation or occurrence of a Bug.
A Bug shall occur in consequence to:
An effort to embed business requirements in the software
A complex code structure in place of a simple code structure
Complex customer requirements
Repeated code change
Unclear understanding of business requirements
Unclear or incomplete customer specifications
Unclear or incomplete documentation
Multiple programmers working on the same set of coding over a period of time
Incompetent programmers working on a code
Improper coding tool selection
And sometimes a bug is passed as it is (uncaught or unhandled) in the final product released to the customer due to improper testing which can have very serious repercussions.
I will address to this issue in one of my next blogs on how to ensure that the customer gets a total bug free product.
Any defect, shortcoming or error in software (built to perform a specific function or set of functions) is known as a Bug. A bug is usually an unexpected event encountered by a tester (or sometimes a programmer himself) while testing a unit, module or a complete product. This unexpected event could be due to a coding error, a code defect, code fault, flaw in coding, imperfect coding, erroneous coding, or improper coding.
The purpose of a code is to perform a pre-defined or pre-conceived function or set of functions. Once the desired function is not performed or is performed with some errors, the code is known to contain bug(s) which need to be fixed. The ultimate goal of a code is to perform as it is intended to perform under all circumstances.
The software built is meant to perform perfectly under all scenarios/ customer requirements/ business rules built in the software to satisfy all functional and business requirements of the customer.
When a product gets completed, it is released for testing to QC department along with the relevant documents. The QC department based on the scope of testing, availability of testers and the time at which the product is estimated to be released to customer, prepares its test plan. After studying the business and customer requirements, test cases and test scenarios are built by testers, based on which bugs or defects are reported. Once the bugs report (or defects report) is released to development team by the testing team, it is the development team that comes into the action. They based on the category, validity of a bug divide among themselves the bugs to be fixed and inform the testing team the estimated time required to close or address all the bugs/ defects. Once all the defects are fixed, the product goes back to testing team, for verification of closure of bugs.
The question arises here is that what is now the scope of testers for a re-released product. Does it suffice the purpose if testers just validate the closure of bugs? No, it is never going to be a fool-proofed product. What about the complete re-testing of the product for the two main purposes:
2. The new bugs arisen during the fixing the earlier bugs reported.
This is very crucial phase and the testing in this phase needs to be more exhaustive and extensive than the earlier round of testing.
As soon as a bug of defect report is released by Quality department to Development Team, the first task of the development team is to call for a meeting inviting all those involved in the development of the product, the project head, the quality head and the testers who have performed testing. Prior to this meeting, it is good if the development team just have a go through of the list submitted to them. The purpose of this meeting is – to clarify if there is any understanding gap regarding a bug reported, to assign the tasks to developers (who is going to fix the bugs), the time frame. And at the same time mark the bugs that need simulation, clarification, need not be fixed, duplicate bugs reported, false bugs, incorrect bugs (the design is meeting the customer requirements but falsely reported as a bug by testers).
Spending time in this meeting with most of the people required is going to be very fruitful. This meeting places all involved on the same mindframe.
Immediately after the meeting (or during this meeting, if possible), the development team should fill in the estimate time required for fixing each bug (time required is to be filled against each bug). Then arrive at a conclusion when the product tentatively is going to be handed over to the test team again for verification of closure of bugs, or finding out any new bugs.
Let us understand what is the purpose of bugs report or defects report? Is it serving its purpose? How much?
The purpose of a bugs report or defects report is to make the intended recipients of this report to understand each and every bug reported in its perspective view. A bugs report has to be quite elaborative in explaining the developers if any business or customer requirement has been failed to meet or has been built wrongly.
Logically defects or bugs report is a report card of testing process. This is the only way to find out how extensively and perfectly testing has been performed. How well have the business rules and customer requirements been understood by testers is reflected by the test cases performed. It also reflects how well Business requirements or customer requirements been documented.
The bugs or defects report not only reflects the total health of the system built but also shows how perfectly has been the testing performed, how well the requirements have been documented and how well the documented requirements been built into the system.
The quality of the product fails its purpose if the testing is not performed well. On the other hand a good testing of a bad system will not be of much use in ensuring a perfect system meeting customer’s requirements.
The bugs should be written so elaborative that they are quite clear to the development team to understand and act on it. These bug reports can be referred to any time in the future too. The more is the clarity or simulation required by the development team from testing team, the less is the purpose of bugs report being served. All it indicates is that the report needs improvement.
Based on the above facts, the bugs or defects report’s content has to be very effective in conveying the intent of the testers performing the testing.
The components of an effective bugs report would be:
Reference Number: Give a unique number for referring to it any time in the future. Treat it like a ticket number generated against a complaint.
Reference Documents: Mention all the documents being referred to, for building your test cases, scenarios and test plan.
Scope of Testing
Initiated by: Write the name of the person who has initiated the test process
Initiated on (date)
Test Setup prepared by: Name of the person
Test setup prepared on (date)
Handed over by: Name of the person who handed over the product and documents to test team
Handed over on (date)
Schedule of testing (from what date to what date)
Test team composition: Persons involved in testing and their roles
Test cases/ Scenarios
Testing started on
Testing completed on
Summary of bugs: Total bugs, bugs count category wise (like show stopper, severe, critical, desirable etc.)
List of bugs with their reference to test cases/ scenarios (against one test case there could be no. of bugs): the list would have the columns – S.No., Module, Sub Module, Description, Screenshot reference no., test case reference no., Category, Status (to be filled later by development team once they start acting on bugs), Remarks.
A bugs report or defects report is a list of bugs found out by testers while testing a software product in testing phase under a testing environment. The test environment is created at development site similar to the actual environment in which the software is supposed to work or run in live situation at customer site. Environment built for testing is supposed to be an independent entity meant only for testing team to perform testing without any development changes happening on it.
Bugs or defects report is the most vital tool for developers to understand where the product that has been built by them is lacking its functionality or performance. Testers have to be very careful in studying business and customer requirements while building their test cases and test plan. The quality of test cases will clearly enhance the core performance and functionality of the product built.
What is a test environment? – I have already explained in my earlier blog (https://itknowledgeexchange.techtarget.com/quality-assurance/what-is-a-testing-environment-for-software-testing/). Also I stated in my previous blog – “why build a separate test environment” (https://itknowledgeexchange.techtarget.com/quality-assurance/why-build-a-test-environment-for-software-testing/) from which two major parties being benefitted are customer and quality testing team of the vendor. Here let us try to understand five essentials to be adhered to while building a test environment for software testing. Let us also understand that these are not the only essentials, there could be many more, I am taking the five top priority factors of them.
1. Customer’s Environments: Understand clearly the environment in which the customer is going to run this software. This has to be checked not only for server but also for the user’s machines. The environments factors could be the hardware, OS, Database, Front end tools, browsers etc. Take care of all the versions of OS, browser the customer machines that are going to run this application.
2. Test Server: Build your test environment as much as possible a replica of customer environment. This is to be applicable to Server and client machines as well.
3. Separate Test Server: Build the test environment on a separate server free from development and dedicated exclusively for testing purposes.
4. Understand business requirements well: The testers and test lead should be very much clear about the customer requirements based on which the test cases are to be built. More understanding, more coverage. Much clearer understanding, wider coverage.
5. Documentation: Aesthetically document each and every test that the testers perform for a unit, module or integration testing of the product.
In continuation to my previous blog – “what is a test environment?”, let us understand here why a test environment is required for testing a software product? I see two major purposes for this, one purpose catering to customer end, and the second purpose catering to the vendor end quality testing team.
The vendor or supplier who is preparing or supplying a software product to its customer must be very clear that in what environment his customer is planning to run this product. The environment could be the hard side – i.e. the hardware configuration comprising of hardware server and users PC components viz – hard disk size, its speed, RAM – size and speed, Processor, bus speed etc. in the soft side it will comprise of the Server/ Users PCs Operating System and its version, OS Patches requirements, Browser – which all browsers are supported and which versions, IIS server specifications at server end, Database Server – What database and which version or release. Any other software components required at server or users computers. The purpose is to run this software as if it is being used by the client at his site. In a way it is the simulation of client site.
The second purpose to have the test environment separate to development environment is to restrict developer’s intervention in test environment during the test phase. This is to be done so that the test results are clear, real and non-ambiguous.
Any other reason coming to the mind while reading this blog that comes to my readers – is most welcome in the comments section.
A testing environment is a setup of software and hardware on which the testing team is going to perform the testing of the newly built software product. This setup consists of the physical setup which includes hardware, and logical setup that includes Server Operating system, client operating system, database server, front end running environment, browser (if web application), IIS (version on server side) or any other software components required to run this software product. This testing setup is to be built on both the ends – i.e. the server and client.
I remember a case where an application was built strictly as per customer requirements by a software vendor without understanding or taking it seriously that what database version the client is intends to use for this software. The vendor developed the application on Microsoft SQL Server x version and the client purchased the new server with the next version of MS SQL. When the product was completed at vendor side, he did a fantastic testing and made the software functionally very strong meeting customer’s all business needs. And when the product was launched at the customer location on the next version of OS and database, the product misbehaved (or did not perform) in few of the instances for which the technical team had to be sent to the customer site to fix the gaps and make it run hassle free. Now this not only affected the product cost but delayed the project deadlines in a major way. This small ignorance at both client and vendor end made a recursive effect at the vendors profits and next few projects.
The production or development team when completes a product development at their end, have to produce this test environment for testing team prior to loading their newly developed product on that environment for testing.
A Project Manager is mostly sceptical to the testers that they unnecessarily try to increase the bugs list just to keep quality department on top. He also feels that it is intentional and many bugs can be avoided to be mentioned which are not very important. Many Project managers in general also feel that quality department is not necessary to exist in an organization. The quality can be maintained by hiring good developers, is what they feel. Then why a separate battalion of testers is required in most of the software organizations across the globe.
Many factors, I would say are responsible for this. Customer getting more cautious towards quality and delivery of product, timelines, competition, increase in internal and external expectations, benchmarks, awareness, prospects, continuous improvement, survival of the fittest, revenue growth and timely returns are few of the top most of them.
There is another side to this coin – A project manager’s skepticism initially may be due to a fear that his team’s development weaknesses will come into light. One more weak area could be the incomplete or inaccurate documentation or study of customer requirements. But with positive signals from Testing Team, Project manager has to gradually understand and admit that all the efforts being done by testing team is not to highlight the weaknesses of his team or product, but to strengthen his team and product.
The switch over or additional responsibility may arise out of different scenarios. Small software organizations initially have no quality assurance department, but with their gradual growth and to meet project commitments, the quality assurance emerges as a separate department.
At this juncture the organization may not be ready to appoint somebody from outside as quality lead, due to various reasons. The reasons could be that the new person will take his own time to settle down in the organization and a couple of months to absorb organization culture. Other reasons could be since this is an experimental start of a new department for which organization may itself be unclear about the department size and its growth.
A new person may refuse to join the organization as a quality lead with too less people in the team. The requirement initially could be too small that one or two persons are required to start the new department. And most important factor is that the internal person working in say production, development or implementation will understand the gravity of emergence of the newly formed quality assurance department and hence will be able to move the department in the serious direction management expects it to.
During my 20 plus years of experience in IT, from time to time I received questions from career beginners (or seekers) having spent 1 to 4 years in one stream of career with a query related to their career growth. This query came from different individuals having spent some time successfully as a developer, or tester, or project leader, or quality leader etc. Streams like development, testing, implementation, support and so on keep prompting or attracting the professionals to peep on the other side of the career that appear to them greener than their own. You never know the comfort and pain of that green appearing side unless you put your feet in their shoe and feel the pinch yourself.
Similar questions I received from others were asking, if not about the career growth, then about the possibility of diversion in the career of one of these persons and the scope of that change. Certain other questions were related to a person wearing a special Hat in the way of switch over or in the form of additional Hat. That person may be a quality lead told by the organization to switch over his role to development manager, or vice versa. Or sometimes someone queries if it is possible for a person to wear different Hats at different times during the day performing multiple roles at the same tenure of his career.
So it does not imply that the hat change or additional hat is always wishful, many times it may be imposed one too. I would say that there is no harm in accepting the challenge and getting into it although it will definitely ask for extra efforts and involvement.
On the other side it is very true that a strong developer may not turn out to be a strong tester and vice versa. Same is true for project managers and quality managers.
What are my reader’s views on the same?
Have you experienced a change in career from one stream to another, if yes, what was the reason.
It may not have happened to you but to someone around you, but still share your points?
It is not only the coding or development that accounts for a successful project. The journey starts right from the release of order from customer after which the customer requirements need to be studied accurately and extensively. This is going to be the foundation of the project. One small slippage or one wrong business requirement in customer requirement document may lead to a big cause of failure of a project. Now there are two ways of going about it. Either the organization has a product in place which is to be delivered with minor or major customization depending on how much customer requirements are meeting with the product already in place. In that case, the next step of mapping it with the existing product and simultaneously finding out the gaps is very crucial. As the clarity of customer requirements and business rules is important to build a right product, equally important is the clarity of gaps. Once gaps are clear, the role of developers comes into picture. Project Leader has to prepare a plan to finish off the job through his developers and under his guidance. The plan should be aesthetically adhered to, to complete the development in time.
Next comes the role of quality control department that has to ensure that the product finalized and finished is meeting customer requirements in all respects. Testing starts at various times in various projects and various organizations depending on the seriousness, methodology and awareness of the organization. Many organizations now involve testing team right from the beginning of a project at the time of its inception stage when not a single document has been prepared. Belief and fact is that involving testing team at this level always enhances the development and helps in meeting the deadlines better.
Documentation is a very important factor – be it user requirements, gap analysis, technical architecture of the product, database design document, ERD, users manual or any other relevant document.
Last but not the least comes the implementation team’s role. Each member plays a vital role especially when it is an overseas project and the success is very crucial. Going to a new environment, adjusting to a different culture, away from family, daily tight targets, late hours working are all the factors that have to be taken care of.
This is the last in my “Project Management” series. To enjoy this blog you will have to go through Project Management I (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-i/), Project Management II (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-ii/), Project Management III (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-iii/) and Project Management IV (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-iv/).
Nothing is constant in this world except “CHANGE”. One this was clear by now that we have to change – immediately and without any further delay. To enhance our Project Management Process, we had already identified our weak areas, we knew what is to be addressed, and when it had to be done also was very clear – it had to be NOW. How? We had to think and analyze ourselves.
We had to transform ourselves in following respect:
Project Requirements: Not only the customer and business requirements but all project requirements need to be specific, clear cut, vetted by customer, documented – including any communication taking place thereafter – in form of email, telephone or otherwise.
Quality: Involvement: Testing team right from the beginning of the project and make them a part of development team, let them participate in all development discussions. Let them be very clear to identify the risks involved if the product is developed as per the project specifications and design finalized. Testing team has to be an integral part of the project now onwards, testing not at the final stage, but at each and every stage of development, by clearly identifying the scope of testing (unit, module, product, functional etc.) and highlighting the defect or bug at the moment it is encountered without waiting for submission of a test report (which in any case will happen but not after each defect identification!).
Sizing and Right Candidature: Go for the right size and right people while building a team for a project.
Backlog: Be clear about the requirements still pending to be catered to, built into the product. Count it as backlog, cut backlog into smaller pieces, and assign each element of backlog to a team member with the time plan (hours or max day).
Monitor: Regular monitoring of this backlog planned versus the completed is required atleast once a day. Involve only the concerned in the backlog as with the backlog resizing, the team also have been divided into smaller parts.
Involve Customer: Keep customer updated on each and every backlog completed. Let the customer, if they wish, to run through the backlog completed. Let the customer, if they wish, to check the balance backlog.
Release the Product: Release the product as per your commitment date. Any pending jobs (if at all!) may become the part of next release after their completion.
Hope you enjoyed the series.
In my earlier blogs on Project Management – Project Management – I (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-i/), Project Management – II (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-ii/) and Project Management – III (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-iii/), I tried to spot on certain defects in our project management process. All these defects or shortcomings were initially not visible due to small teams, small size of projects and different management perspective. But a requirement to relook into ‘project management’ arose when management had to change their perspective due to change in customer perspective.
This asked for introspection, analysis and thought process in which we identified certain factors as stated on my earlier blog “Project Management – III”.
All these factors were causing:
Missed Schedule – Most of the projects were either getting overdelayed by dying to their natural death prior to completion or implementation.
Missed Functionality – Lack of customer requirements, lack of documentation, isolation were few of the factors that caused the build of an improper functionality or totally missing it from the product. The customer living on another island was in a different imagination world dreaming about a different functionality and product as compared to the one being built (or missed) by the development team.
Missed Budget: Obviously – any delay in schedule, or building of a functionality that was skipped during initial built will ask for a fresh expenditure. Is customer ready to pay this additional amount?
Brittle Product: The product is too fragile or brittle that it is loosely threaded in such a manner that it will lose its flow or functionality during actual usage.
Quality Aspect: The product that is seeking changes during the later stages is always prone to weak quality due to time and budget constraints and various other pressures from customer side.
One thing was clear by now that we need to change, as there is no alternative. We need to change to strive, to exist, and to deliver. We need to change to get out of our static system. We need to change to meet customer requirements. We need to TRANSFORM – OURSELVES, OUR PROCESSES, OUR THINKING.
Change – What and How – wait and hop on to my next blog in the series – “Project Management – V”
To carry on further from my earlier blogs on Project Management I (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-i/)and Project Management II (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-ii/), this is the third in series. Project Management in my opinion, is not a static Management, it is a dynamic state. It keeps on giving a new learning in every project, and seeks changes in the project management process. There are instances where the earlier project management methodology of freezing customer requirement with a document preparation, a sign off from customer, stating that the document is sufficient to cater to our requirements and the software be built based on these requirements mentioned in the document. Now the requirements may change during development, or even implementation, during any stage of project management. You can not say ‘no’ to those requirements asking for change in software, as you know very well that ultimately a ‘good software’ is the one that caters to the customer requirements and customer business than to those mentioned in the ‘business study document’.
What we lacked in our methodology of project management mentioned in my earlier blog “Project Management – II” was – a clear cut ownership of the project, involvement of all concerned throughout the project life cycle. Documentation was another weak area – documentation of not only the requirements, but of system design, database design, communication (internal and external) associated with the project. And above all, the random and isolated involvement of different stakeholders, like QA will enter into the project only when the product is complete and ready for testing.
Now, of above, imagine the time spent on QA to make them understand the complete product right from scratch, that too in absence of complete documents.
As discussed in my earlier blog on project management (Project Management – I, https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-i/), different organizations adopt different procedures, processes and policies for their Project Management, and then keep improvising them until they reach at the level of ‘best’. Best practices in ‘Project Management’ are not a single set of practices that are used by various organizations – but rather I would say any ‘good’ practice when used to its best of effort, sincerity and energy becomes a best practice. Any best practice in Project Management asks for involvement, from all levels, starting from top management to the junior most level. Here, I would discuss some practices in software project management, which we adopted earlier, then brainstormed on those practices that where we were as compared to the best practices being adopted and followed by various other global organizations, analyzed where we need to change and how we need to change with clarity of visible benefits out of those changes adopted.
Initially it was a purely waterfall scenario – with the sequence as – Requirements and System Analysis, System Design, Testing, Implementation and then support finally. This was going well for years, without any much of issues. But later when we grew and analyzed we found that it goes well with small team sizes, small projects, fixed requirements, in scenarios where reporting of apparent success to management was sufficient.
But we found a resistance in out own system – team sizes were not small now, projects were of all sizes – small, medium and large, requirements were never definitive and we have to be open for any requirement change asked by customer during a project, and above all management is not at all interested in any offline report showing progress of the project – rather they are interested in ‘progress’ if it goes well without any ‘reporting’. That means the management wanted to see the progress – visible and in unanimity with the customer (internal as quality, and external as the Product Owner).
Project Management is nothing but adoption of a set of processes to ensure the smooth running of a project development passing through various stages and its successful and timely completion. It is the crux of any project as I too believe that it is not the people who make the projects successful, it is the processes and procedures adopted by the management responsible for the project. Project Management depends on how matured a company’s processes and procedures are. Broadly in terms of Project Management, organizations can be divided into three categories: First category will be of organizations having no or a very little system in place for Project Management. The organizations falling in second category would be those who have processes and procedures in place but not adequate for 100% successful Project Management and hence those organizations keep on striving for improvements by continuously analyzing, trying, good practices for enhancements, optimization, and successful project management. The third category of organization would be those who are matured, balanced, and a showcase for other organizations. When I say balanced, I mean they are neither too rigid nor too flexible in their practices. These organizations have a set of best practices in their sleeves being used for years, time and tested again and again. They would decide upon the best suitable practices depending on a specific project. When I say matured, I mean their success rate is higher than their peer organizations, they are the best in Project Management. Their employee turnover ratio is quite less, their employee satisfaction level, energy level and passion level is higher. They have not only adopted the best practices in their system, but in their organization culture, and in the blood of each employee.
For first and second category of organizations, the journey is long to reach to the final level, but equally higher is the scope to achieve it. Provided they have an urge, zeal and fire to achieve it, they stay on their levels for years. It is the management that has to drive it and for management it is sooner the better looking at the competition at global level and the expectations level of the customer. That does not mean that for the third category of organizations it is the end of the journey, rather it is difficult to maintain, sustain and keep improvising as everyone knows it is easier to reach at the top once than reaching every time, and maintaining it.
This is in continuation to my earlier blog “Project Management – A Sad Story” (https://itknowledgeexchange.techtarget.com/quality-assurance/project-management-%e2%80%93-a-sad-story/). Keeping all the character names same as described there, I will update my readers on what happened to this particular project headed by Roberts. Although Roberts is new to these projects but is trying his best to get into each issue and address it appropriately keeping ‘common sense’ part intact while making a decision. Roberts knows and often tells others that project management is not only about the management of a project, it is about managing people working under you, it is also about customer. The story takes a couple of twists and turns and moves altogether differently.
Julia, the project leader, accepts her defeat, and resigns from the organization. Linda, Lisa, Paula, Edger and Alex are doubtful about the success or completion of this project. The issues that arose at customer site were majorly two as stated by the customer – one is that Julia is not competitive enough to understand business requirements and embed them into the software, and second is that Team has no purpose in staying at customer site till the product is complete. So they sent the team back to their country. Julia’s resignation is accepted as management also feels she is one of the reasons of this failure. Sandy, the QA head, is given an additional charge as project leader of this project. Sandy has a good record or project management and has been 100% successful in all the project he handled so fat prior to becoming the QA Head. One would wonder when he was so successful in Projects, why he became QA Head. Probably he knew that this Quality of ‘quality; of a product in him is one of the key factor for all his project management success. Sandy accepts this challenge but with an apprehension that before Julia stops coming to the organization, she atleast explains him the complete picture at customer site about project, requirements, and customer. Sandy has to not only take care of all customer needs to be built in the product, but also to ensure the quality aspect of it. The project is delayed by 20 odd days. Development team is working in the customer requirements. Sandy is overall now is the linking part between offshore customer and his development team (and also for his testing team).
Only apprehension Sandy has now is that if the fresh requirements brought by Julia were complete and exact or still if he has to face the similar situation again!
Let us wait and watch!
By the way – what is your opinion on this – what would happen to this project – SUCCESS or FAILURE!
STLC (Software Testing Life Cycle) is an integral component of SDLC (Software Development Life Cycle). Gone are the times when any software was made on the basis of its requirements and the moment it used to get completed by the development team, it got released to the customer. But now, Testing has become a distinct phenomenon during and after the development of software. No software is released to the customer without a comprehensive testing by QC or Testing team in the organization. The Scope and Methodology may vary from product to product, customer to customer, and organization to organization. There are certain aspects of Software Testing Life Cycle. To name top few among them, I would like to list twelve essential steps of Software Testing Life Cycle.
The Steps are to be followed from the start of Testing of software to the end of the testing as follows:
1- Before the dynamic testing, there is a static testing. Static testing includes review of documents required for the software development. This includes following activities:
(a) All the documents related to customer requirements and business rules that are required
for software design and development should be handed over to QA. These documents
should be complete and dually signed by client and representative of the company
(usually Project Manager).
(b) QA reviews these documents. The reviewing of documents includes comprehensive and
thorough study of the documents. If any discrepancy is found then it should be noted
and raised in the review meeting with the Development team.
(c) After this there should be a formal meeting between the QA and development team
regarding these documents, the agenda of this meeting mainly includes what is missing in
the document, QA queries to be answered by Development/Project Team and/or
clarification required for any confusions.
2- After the Software development or build of a module, QA starts dynamic testing. If during the development the requirement has been changed on customer demand or due to any other reason, then that should be documented and a copy of this revised document is given to the QA and also discussed as explained in point 1 above.
3- Development and Testing environment should be made clear to the QA by the Development team. It include the following activities:
(a)- Server to hit for Testing
(b)- Installation of latest build on the test server.
(c)- Modules/Screens to test.
(d)- Test duration as decided by test manager and project manager mutually based on scope
of work and team strength.
(e)– Demo of the software on test server by development team to the QC members.
4- After this Test cases and test scenarios are prepared and then the Test execution by QC.
5- A comprehensive Report of Bugs is prepared by the Testers and a review/verification by QC/QA/Testing Head takes place. Before handing over this report to Development Team there is a thorough review of Bugs List by Test Manager and in case of any clarification required on a bug submitted, the Testing Head discusses the bugs with the assigned tester.
6- Release of bug report by QC Team to Development Team.
7- Discussion/simulation of bugs by QC with development team if development team requires and time required for fixing the bugs should be made clear by Dev team at this stage.
8- Feedback from Development team on reported bugs with the stipulated time frame required to fix all bugs.
9- Any changes in the software being made in respect to fix these bugs should be made clear to the QA team by the Development team.
10- Testing team then Retests or verifies the bugs fixed by the development team.
11- Submitting the retesting bug report to the Test manager and after this the step 5 to step 10 are followed until the product has reached a stage where it can be released to customer.
12- Criteria for ending the testing should be defined by management or Test Manager Like when all major bugs are reported and fixed. Major bugs mean the bugs that affect the Business of the Client.
Testing is a process of facilitating development team/ project team in improving the quality of software before it is released to the customer for use. Some key essential steps are always there that need to be followed by the Testers during Software Testing to streamline the process. The most important checkpoints for testers during software testing, in my opinion, would be:
(1)- Complete document of customer and business requirements specifying all the requirements for the development of the product.
(2)- Latest completed build and URL of the application to hit for Testing.
(3)- Software requirements for installing the application on PCs for testing or for database connectivity.
(4)-Training/Demo of the project by development team to Testing Team so as to understand the flow/functionality of the software.
(5)- Scope of testing should be made clear by the development team/Head.
(6)- If the build comes for retesting, then it should be accompanied by the revised document which includes the updated changes incorporated in the software.
(7)- Clarity regarding which member of development team should be contacted in case of any clarification required during the testing phase regarding the functionality of the module or if testers encounter a showstopper in the Software.
(8)- After release of the bugs list to development team, how much time they will require for fixing the bugs.
On his blog by the famous Leader on Project Management Bas de Baar, he asked a question to all his blog readers – “What is the best advice you can give to project managers supervising offshore development projects?” Various readers posted their opinion as per their perception, experience and learning. His focus was more on Offshore development projects, whereas I would like to point to any offshore project that has to be developed implemented on site. So in context to offshore development and implementation of a project, a project manager has to essentially develop some soft skills other than his/her standard project management tools and methodologies. According to me for an offshore project management following would be the essential key requirements as I am sure every one of you will agree that an offshore project is definitely different to an onshore project. The basic things to be taken care of in context of that country, management, users shall be:
Language: Understand what their prime language is in day-to-day use. I went to a country where language was a big barrier and I was heading the project. Before going there, I learned few of the commonly used words, downloaded a dictionary to translate from English to their language. Although very few of my users were fluent in English, and I used them extensively as a medium to translate what I used to say in my training sessions. But there were instances when I am sitting with a couple of users who don’t know English or who know very little of English, then this offline/online dictionary was very helpful. Also I learned before hand that they speak English very slowly and can understand only if spoken in low speed and a different accent (like “show” will be pronounced as “cho”). Ascent was another criterion that I had to gradually master upon. This will also depend on deciding upon the pace of the project. I had to change my plan accordingly to cover up the backlog at times.
Culture: They will love if you talk about their country and culture, and that is the best way to make them your best friends. Then it was easier to understand them faster. Learn about their likings and disliking, what they don’t like is very important, or what annoys them easily, so avoid doing that or using those words. Don’t refuse to attend any of their customs when invited. After all it is the people that make any project success. And most of the people outside will be interesting to know your language or culture, do not hesitate in sharing your knowledge/skills and spending some time on this, although it is not the part of your project but it will develop into a very strong tool to progress your project in right direction with right pace.
Be as them: Don’t act as a foreigner. Act as one of them. Show the management that you are the part of their organization and it is not only their but your project also. Keep top management involved in all your decisions and actions. Seek their help where required.
Celebrate: Celebrate any small success that will motivate them and speed up the implementation.
Do let me know anything that I might have missed to mention.
Roberts: Project Manager
Laura: Project Manager
Julia: Project Leader
Sandy: Quality Head
Toby, Andy, and Victor: Software Testers
Linda, Lisa, Paula, Edger and Alex: Module Leaders
Storyline: Project Manager, Roberts has newly been shifted to manage various projects (running or in pipeline) as this post got vacant after the resignation of Laura. Roberts prior to this was head commercials and this is a new avenue for him, he now handles many projects at a time and is okay with it. Julia his Project Lead for the project in question is leading a project for the first time. She has been involved in project implementation earlier but as a module leader and not the project leader. Sandy is the Quality Head and is responsible for the QC of any software that goes to any customer. Toby, Andy and Victor are reporting to Sandy. Linda, Lisa, Paula, Eder and Alex are working on different modules of the project lead by Julia.
Laura was a cherished leader a year back, as she was able to grab quite a good number of overseas projects. But somehow she failed to manage the projects as each project got overscheduled due to one reason or the other. Ultimately, Laura had to admit her defeat, and she resigned from the company. But does it solve the problem? No, the company by now is in deep troubled waters and has a big question mark on its reputation for the existing and future projects. Julia and the team were already designated by Laura and were sent to the client site for implementation. Sandy and Laura were not in good terms as Laura always wanted Sandy to bypass the defects in the product and wanted him to give a go-ahead for implementation without worrying about the completion of the product (or ignoring unfixed severe bugs reported by Sandy’s team).
Roberts did not disturb or made any changes in this particular project team lead by Julia. Julia and team are back from customer site without completion of the project as some issues arose which could not be handled by Julia. Few of the outcomes of the team and team leader that are back from the project (still incomplete) are:
1. Toby, Andy and Victor were not happy with the state of the software when it was taken for the implementation at customer site
2. Sandy had clearly told management in advance that there are chances of project failure
3. Linda, Lisa, Paula, Edger and Alex are not happy with their leader Julia as they feel that Julia who is also heading a module of this project is always over occupied in her module and has no time to address project or other module issues
4. Julia was not able to convince customer management about the product
5. Many business rules that were built in the product shockingly absolved as the wrong requirements understood by business study team.
6. Julia is not happy with her team members – Linda, Lisa, Paula, Edger and Alex as she feels they are not listening to her and doing whatever they want to do without consulting her
7. Julia and her team were all sailing their individual boats in different directions at customer site.
8. Julia feels she is the team lead just on papers otherwise it is being lead by someone else.
9. Roberts feels that the initial system study done at the time of Laura was not satisfactory and accurate.
10. Roberts also feels that even now when the team goes back for implementation of the same project, there will certainly be many issues.
11. Roberts is not sure about the timeline or schedule of the project, that whether this time the team lead by Julia will be able to finish the project in scheduled timeframe, but is not worried as he knows that the management will accept his failure because he is not the one who started it. He feels secured even if this project fails as he is sure that the management will not him fully responsible for this, or even if it happens he can pass on all the blame to Laura who is not with the organization now.
Ultimately how do we ensure the successful completion of this project in stipulated time is a big question (although it is not disturbing anyone as everyone has already a bundle of reasons in his pocket)?
Does someone have the answer?
Testing does not ensure risk-free software. There is nothing called hundred percent testing. Be it manual or automated testing but all testing is in a way manual (even in automated testing the test scripts are altered manually) and hence it is for sure that the testing will never be able to cover all possible test cases in a practical situation if compared to an ideal situation. Every time my team comes to me with a final report of bugs of any software tested, I am able to point out many cases that they say “yes, it should have been done! But we missed it or skipped it!”. Sometimes some cases or scenarios are intentionally dropped by the tester based on just his imagination that this case has no significant role. But if looked from another angle, the same intentionally dropped case appears to be a very important test case that is must to conduct to understand the result of an important behavior of the software. Like security, testing also does not ensure zero percent risk after it has been done. Only thing it ensures is minimizing the risks and vulnerabilities to the best possible level. Software Testing is the process to reduce or minimize the bugs or defects hidden in the software.
For further tips on how to achieve this you can go through my previously posted blog as “Twenty ways to ensure complete coverage of software testing”: https://itknowledgeexchange.techtarget.com/quality-assurance/
To ensure complete coverage of software testing, testing team must be careful about certain activities that are part of the process. If the software testing is not complete as per the business and customer requirements, it could have a severe adverse effect during or post implementation of the software at the customer site. The more is the coverage, the less are the chances of any bugs passing to the implementation phase. So, to ensure the complete coverage of software product and to find the maximum possible bugs or defects, testers have to ensure following steps:
1. Ensure that the documents defining the business and customer requirements are complete and correct. How to do that can be taken up in a separate topic.
2. Ensure that the development team has clearly understood the documents.
3. Ensure that testers themselves have thoroughly read and understood the documents.
4. Prepare a clear cut scope of testing based on product documents.
5. The strategy and Test Planning is as per system requirements.
6. Decide test methodology and test tools (if any), and test schedule.
7. Prepare Test Cases based on business rules and customer requirements.
8. Ensure that the test cases are extensive and sensible to cover the complete requirements testing.
9. Ensure that during testing no changes in the test environment (coding etc.) is done by development team.
10. Ensure that development team representatives (1 or all) are present during the complete testing.
11. Create Test Scenarios based on test cases.
12. Observe the result of each test case and record it accordingly.
13. Prepare a comprehensive and detailed test report explaining each of the test case, scenario and its result elaborately.
14. Ensure that all bugs reported should make sense (no duplication/overlapping of scenarios and no repetition of bugs reported)
15. Ensure that the complete testing finishes as per schedule.
16. The final report submitted should clearly state the areas not covered under testing, reason for the same and its impact on the product.
17. Simulate any bugs that are not clear to development team.
18. Ensure that you have a tentative plan from the development team when they are fixing all bugs and submitting it back to testing team.
19. Verify all bugs fixed and ensure that the development team is sitting with testers during verification.
20. Prepare the final report and submit back to Development team, giving the status of each bug fixed, as verified as fixed or not fixed. Report any new bugs arrived in the software while fixing these bugs.
3 Years back there was no QA department in the organization. The development manager, John, was the king. Nobody was there to check at what stage his development projects were, all used to go by his words. One day he was asked – hey, what is the status of your software that you are developing for Canada? Oh Sir, he reported his boss, it is all ready. When do you want to send it to Canada with my implementation team? Ok, if it is ready, let us book tickets for coming weekend so that your people land there, settle down in the hotel, enjoy weekend, and start implementation on Monday – his boss told him. Ok Sir, he replied.
John called his development team responsible for Canada project, takes stocks of situation, gets to know that still many screens, sub-modules, units are under development and at no cost finish by weekend. Oh, but I have told boss that everything is ready – he tells his team. Don’t worry sir, his team tells him, we will finish it there. Ok, he replied.
So the team goes to Canada, the target of finishing the project is not met, users are not happy with the product (it was not a complete product), management got angry, stay is increased, but even with the extended stay, the project does not take proper shape and hence finally the team is sent back.
What now! An emergency meeting is called. Reasons for project failures are found out. It comes out that John had projected a rosy picture to the management without showing the existing thorns in it.
And the QA department took birth with a resolution that no software will move out to any country without a final clearance from QA.
3 Years have passed, John’s life got miserable in the beginning. Each of his products had to undergo 5, 6, 7… cycles of testing and re-testing by QA. Initially QA reported why the product has come for testing when it is not complete? Why it is declared as complete? Why during testing, the development team keep doing something or the other in the same product instead of giving time to QA etc. etc.
Ultimately maturity arrived. The products only are shipped when they are sensibly completed and verified/tested.
The Development Manager finally says to QA Head – You exposed me, no problem, your team tortured me (my testing and re-testing), no problem, ultimately I am the happiest man now because now when I declare about a product that it is complete, and my boss stares at me, I add immediately, and it has been cleared by QA after x number of testings finally. I am sure it will not fail now as usual.
John’s life has become tough but he is the happiest man on this earth, you know why, because now he is sure that none of his projects will overrun or fail.
Software Developers are not doctors and that is why are not in the habit of doing ‘first time right’ coding. All newly developed software contains bugs, bugs and bugs. Developers have this mindset that no software can be written without a bug in it. They are convinced that bugs are integrated part of coding and involuntarily that are bound to place them here and there in their code while writing a software. If doctors start working like this that in every operation they leave some important activity ‘undone’ and have a privilege of getting each operation audited by another doctor (specially meant for auditing a patient immediately after operation), the auditing doctor will find out the shortcomings in the operation, report it to the doctor who operated upon the patient, the patient gets operated again, again auditing, again operation, so on and so forth. Think of what will happen to the patient in this case. And I think the patient will prefer to die on his own than in this manner.
The same thing happens to the health of software when some bugs are embedded in it, found out by testers, reported to developers. The developers fix those bugs, in turn produce some other bugs, again it comes to testers. Another bug report, another fixing, another different set of bugs production, so on and so forth. What happens to the health of the coding in this case?
If software could shout and cry!
A point to think upon and act upon!
To me sometimes it appears, may appear funny to some, and logical to others, that there is a strong resemblance in a Poultry Farm and Software. As different Hen in a poultry farm keep producing eggs day in and day out similarly different modules in the software keep producing bugs.
Do we agree then that as hatching eggs produce more hens, similarly fixing of a bug produces more bugs?
Eggs which are not hatched but consumed may then be related to bugs which are there in the system but have not been identified and remain floating in the system. As is known that many bugs cross testing phase without being identified and come into light during live run (i.e. post implementation phase), sometimes after months of successful completion of a project.
Do we require such software – productive (producing bugs) like a poultry farm? Or we need good programmers to make good software and not turn it into a poultry farm.
I invite your valuable Feedback/Comments please.
Equivalence Partitioning Testing Method is a method used in Black Box Testing (see: https://itknowledgeexchange.techtarget.com/quality-assurance/what-is-black-box-testing/) for the purpose of finding out minimum set of data for testing for valid and invalid inputs.
The purpose of this testing is to:
1. Ensure that the software acts according to the type of data is entered, i.e. for a valid entry it should accept the data and for an invalid entry it should return the appropriate message/alert.
2. To ascertain that minimum data is used for similar results (the result of a test case could be valid or invalid)
For a group of values, if the software behaves in same manner, the group or set of value is called ‘equivalence class’ or ‘equivalence partition’. So primarily for an equivalence class the software will behave in the same manner and shall produce the identical result. Different equivalence classes will make the software behave differently and shall produce different result.
The tester has to be careful in carving out the different equivalence classes so that one value is taken from each class to ensure complete testing. The testing in this way is completed with least efforts/ test cases.
Let us take an example to illustrate:
In a particular Airline based on number of flights taken in a year, a membership is offered with different privileges as below:
If the number of flights taken is between 5 and 10, the membership offered is ‘Silver’
If the number of flights taken is between 11 and 15, the membership offered is ‘Gold’
If the number of flights taken is between 16 and 20, the membership offered is ‘Platinum’
If the number of flights taken is between 21 and 40, the membership offered is ‘Diamond’
Based on above all inputs between 5 and 40 are valid, less than 5 and more than 40 are invalid. And also any non-numeric value is also invalid.
So there will be 7 equivalence classes viz: 5-10, 11-15, 16-20, 21-40, 40 and non numeric and thus one value from each equivalence partition will give you the complete test coverage.
Black Box testing of a software is based on two primary things –
1. Customer Requirements Document
2. Functionality of the software
Purpose of the Black Box Testing is to ensure, that means, that –
1. The software meets are customer requirements as specified in the Customer Requirements Document (business needs)
2. And the software meets user requirements (user needs)
By user requirements we don’t mean that it has to meet every user’s requirements but more appropriately the appropriate usage requirements.
Black Box Testing is the most widely used testing and is used to used to check if the product meets per se customer’s all requirements. The tester is not worried about how the software has been built, what queries have been written, whether the coding is technically optimal or not, but focuses more as if the customer is working on the system and feels like customer when he tests the software.
A basic requirement of Black Box Testing is that nothing should be presumed about the functionality of the software while testing the software, rather the tester is supposed to check each and every functionality of a unit, module and the product as specified in the customer requirement document. So the fundamental of this testing is the Customer Requirement Document, based on which the appropriate test plan, test cases and test scenarios are built and ultimately test results are drawn as a Test Report (or Bugs Report).
For building various test cases in Black Box Testing certain techniques are used like:
Boundary Value Analysis (or Boundary Value Testing), see my earlier blog on this:
Equivalence Partitioning: This is a technique of testing with minimum of test classes or data input to identify between valid and invalid input.
Prior to starting actual testing, a tester must verify and validate:
1. Customer Requirements
2. Functional Specifications of the software
3. Test Plan
4. Test Cases
5. Test Data
6. Test case execution
After above validation, the test cases are executed, results are recorded and reported to development team.
Way back in 1960s softwares were being produced but there were no established standards or software engineering practices. Poor design and Regular hardware upgradation left many types of software useless and thus a strong need arose in the industry to provide effective and reliable software developers who could pace up with regular hardware advancement to produce faster and highly reliable softwares.
All this, plus the increasing dependency on softwares increased not only demand of good developers, but also a requirement of software testing took birth, may be, not initially in a very strong and structured manner but it did.
Software engineering practices that were taking place at that time were not formulated, or structured in an efficient manner. Some of the bad practices were no measurements or metrics were in shape to measure up –
Development lifecycle or product lifecycle
Lack of planning, monitoring
No control on development schedules
No control on customer requirements changes
No reviews of documents
Failures at customer end during or post implementation due to product incapabilities, schedule delays, revenue losses
Therefore a need to test the software came into existence to:
enhance all these processes,
increase the reliability of product,
deliver a quality product,
deliver in time