No. I don’t think so. If you have to identify the bottlenecks in your newly built software application, you are bound to adhere to this approach. Use a progressive bottleneck identification approach for performance testing of the application. The testing approach should be to apply holistic load on the application server. There could be multiple aspects for users working simultaneously on the application in real scenario:
Multiple users would be accessing different workflows of modules of the application.
Multiple users would be accessing the same module of the application
Multiple users running different reports
Multiple users running same report
Multiple users running different batch processing
Multiple users running same batch processing
Multiple users logging in
Multiple users running multiple sessions
Now to find out the peak performance point of the application beyond which the application does not behave normal, an incremental or progressive load approach need to be applied while performance testing. Testing would be carried out to identify the impact of various performance parameters on the application. Application would be tested under a concurrent user load and the transactions response time for critical transactions is reported back with their response time. As the load of users, or sessions is increased on the application, the behavior of the response time is studied to ascertain the optimum users or sessions permissible in the application under a pre-defined set of hardware environment.
A project starts with initiation phase and ends up with project closure report. Then afterwards there is post project feedback (after a considerable amount of time given to the customer to get conversant with the product) and warranty support followed by support contracts over a period of time. Let us begin our story with the project initiation and proposal approval from customer. The real ball-game starts from here.
Now planning starts, teams are formed, milestones are set and teams move into execution phase.
Execution phase comprises of development, customization, configuration, testing, training, reporting, sign-offs, monitoring upto implementation, parallel run, live run etc.
And then comes the project completion phase with Project sign off.
During all the phases of project initiation, planning, execution… are the milestones broken into tasks and tasks further into sub-tasks unto the smallest measurable, achievable ‘hour’ unit. Or it is measured in days, weeks or months where it loses its continuity and rhythm. If everyday something is declared done, finished, achieved that is visible to the project team, stakeholders including customer, it carries out a satisfaction, rhythm, confidence and regular progress.
A software project has to undergo various stages before reaching the final stage of customer sign off. At each stage of the project there are certain set of documents that are maintained by the project team for internal or external purposes.
These documents are prepared by various team members – by business analysts, by coders, by project manager, by testers and by other managers.
The Quality and maturity of documents straightaway tell about the health of the project, the team, the management, the product and the progress. It tells clearly about the intentions behind the documentation – that if it is merely a formality or it is really meant for helping the project progress.
And it is not difficult to ascertain the intentions after going through the documents maintained or being maintained during the project.
A coder or programmer when told that he is not writing his code, by way of presenting him with a list of bugs, he is being told to CHANGE.
A tester when is told by his superior that he lacks business depth for testing the product, he is being told to CHANGE.
A project manger when told that he has to improve his way of managing a project, he is being told that the current methodology does not suffice the purpose. It requires a CHANGE.
A management when consistently fails to deliver its commitments to its employees and customers, means it needs to introspect and CHANGE.
When a person, department, team or country stops delivering, or starts giving indications of failure in one aspect of other, it asks for CHANGE.
Change is powerful.
Change is inevitable.
Change does not challenge your knowledge.
Change does not indicate that you are incapable. Rather when one is being told to change, someone has confidence in his capability to CHANGE and PERFORM better.
Change does not mean that you are wrong, it means that there is a scope of improvement.
Change is often opposed as a natural tendency because it is taken as an indication that you are not performing well and what you are doing is not doing well.
A new project, a new product development – as a QC head how do you estimate your testing effort?
Well, some quick steps for this:
1. Customer requirements: Customer specifications or requirements captured at the time of initial study period would be a quick reference guide for estimating testing effort. One word of caution is that incomplete or non-documented requirements may lead to wrong estimations for both – development and testing.
2. Business Specifications: Like customer requirements, there are some business rules and specifications that are well defined by the customer representatives that need to be built in the software. Building test cases and performing tests will depend on such requirements and hence the estimation.
3. Testing Scope: What is the overall scope of testing? Performance, Load, Functional, or all will help you in estimating your testing effort, team size, and test plan
4. Development Plan: Development will definitely go in phases. Plan testing accordingly and estimate testing effort in that manner to accommodate parallel or phased testing so that all load of testing does not fall at the end of development phase.
5. Test team size: Testers availability will be major criteria in estimating your test effort.
We all know developers and testers both have a tough job all the time. Developers have a key role in developing the software as per customer requirements embedding customer’s business needs into it. Similarly testers have to put all their efforts in ensuring that the software is matching customer specifications flawlessly and is bugs free. In a nutshell both developers and testers have a common goal of ensuring a superior product delivery at customer end. If that is so why there is a never-ending tussle between testers and developers. Why developers feel testers are unnecessarily trying to poke their nose into their affair. Why testers feel that out all the bugs found out by them, most of the bugs would have been already handled by developers if they had done their job more seriously.
All this leads to certain questions about testers and developers which they only can reply to:
1. Testers are not supreme and so are developers. If developers can build so many bugs while writing the code, testers are also bound to leave certain loopholes in their testing. This is universal and never ending story.
2. Most of the testers around the world who test software do not understand very well the purpose behind the testing. They keep oscillating between their role as policing, controlling quality and excessive reporting.
3. Both developers and testers carry a single goal of ensuring good quality of software at the end of the day but still keep blaming each other for the shortfalls.
4. If developers are kept for writing code, it is well understood that they are being paid for writing good code and not bad code. Then why bugs at all? If a developer has been hired for coding, is it wrong at organization level to expect a 100% bug free coding from developers. If they are permitted to write code with bugs, why not every other function in the organization is allowed to perform their daily tasks with errors. Can’t we have perfect coders?
5. If testers find out the bugs, instead of being thankful to them, why developers start finding out reasons of cornering them. Developers are hurt when testers find out bugs in their code, and instead of going into a thanks mode for testers they start going into another mode where they themselves start losing their respect. In turn they start finding out weaknesses in tester’s capabilities, testing criteria or bug reporting process.
6. It is a well proven fact that while fixing reported bugs developers are bound to generate new bugs. Does it not make them circling them around the same product?
7. Testers sometimes have an understanding that if they report less bugs will mean a question mark on their job, which forces them to report many a times non-quality bugs thereby increasing gap between developer and tester.
8. Developers, once they know that the product has to undergo testing, write code so foolishly that they generate lot of unexpected bugs.
9. If testers are hired for finding out bugs, is it not their lack of depth of knowledge that leads to bug explosions at the later stages? Are testers involved in coding, or business study or implementation?
10. If developer’s after reading so many books on development write codes with bugs, I don’t think a good tester criteria should be if he has read a book on testing or not.
Project Management is not simple. It requires lot of skills (and learning and experience) to become a good project manager. A good project manager is the one who plans and executes well – all the stages of a project. Finally if project finishes in time with a SMILE ON CUSTOMER FACE, the project can be treated as successful. It is the option of a project manager to label himself as “lousy” project manager or “successful project manager” depending on how he/she manages the show throughout.
To become a “lousy” project manager, following steps can be followed which are quite simple in nature and easily adoptable. Vice versa of these steps can help you in becoming a “successful” project manager, but following that will require some smart efforts and a “fire in the belly”.
The steps are:
10. Don’t own the project: You can easily pass on the buck to anyone engaged in the project. Keep yourself focused on the minutest of the failures and find out the right guy to be blamed for it. With smart reasoning you can easily convince the management and other stakeholders about the failure, its reasons and justification for delays in the project.
9. Assure insufficient customer requirement gathering: To achieve your target, ensure that the customer requirements are not gathered in the proper fashion. Ask irrelevant questions to customer and that to the irrelevant persons for gathering customer requirements. This will surely give you a good reason to convince customer later that it is because of their fault in specifying proper requirements that the product built in not meeting their requirements and you will have another stack of ‘months’ in your kitty to linger on the project.
8. Build a thick wall: Between your management and customer, build a thick wall so that all communication never gets transparent. This will help you in moulding the things in your way as and when required. Similarly build a thick wall between various sub-teams in your project like testers and developers, developers and implementers etc.
7. Build inappropriate teams: For managing your different phases of project, inject inadequate number of persons in your various teams responsible for different phases of the project. Also ensure that that most of the team members are “dumb” enough in knowledge and smartness so that you can easily place the “failure” hat on anyone’s head at any rough moment of the project.
6. Ambiguous documentation: Ensure that the documentation is not at all crisp clear at any phase of the project. Let it be as ambiguous as possible, but in a smarter way, so that nobody is able to figure out the objectionable part.
5. Stay unfocused: Find out some other critical issues not related to project but that can easily make you reasonably justified for not able to devote proper time on the management of project.
4. No Reporting: Let each member enjoy. Make no provision of timely reporting about the progress of project.
3. Dearth of technical knowledge: Don’t develop your technical knowledge required for your project. Let your technical people befooling you in their own way. Ignorance is always bliss.
2. Be Political: Keep on taking your customer on a different track by telling them wrong stories about your management and product in process. Also keep informing your management about customer’s lack of knowledge, involvement, providing of sufficient key users etc. that will become your strong ammunition for shielding you.
1. Repeat everyday: All points above have to be read repeatedly everyday so that you end in becoming a “perfect loser” at the end of the day.
Next month is a marriage in your close relation. You plan to buy an expensive suit length and get it stitched by the best tailor in the city. You buy the best cloth, go to the best tailor, he takes your measurement and gives you a trial date suitable to you. You go on that date, find minor or no change in the stitched suit, tell him the alterations required and get your fully perfect suit after 2 days, a week before the function date.
Your customer decides to go for a business application, decides on you to build it and implement it, gives you an order, you take the measurement (understand business rules and customer requirements), you give them the tentative date for trial (UAT)… but UAT goes down flip flop. You are not able to deliver the product on promised date.
Your tailor delivered the suit, with your complete satisfaction, on the promised date.
Where is the difference? Something went perfectly between measurement and trial date for your suit that your tailor had to deliver to you but not for your product that you had to deliver to your customer.
This is called product sizing and team sizing i.e. project sizing.
Project Failure: First failure will downgrade the level of next project to be given to the project manager. Not only this, but it will also trigger hidden cameras in the organization that start monitoring each and every step of project manager. These hidden cameras could be top management or some selected down the line people working with project manager.
Customer Feedback: Organizations are taking customer feedback very seriously. A remark by a customer regarding a project manager – “I want this project manager to manager all my future projects ordered to you. Infact if you don’t depute this person as project manager, you don’t accept our orders” triggered such a magic that it immediately presented the project manager with an out-of-turn hefty increment. On the other hand a customer CEO sent a confidential email message to the CEO of projects organization stating – “We don’t want the current project manager to be seen in out campus with immediate effect” put a very big question mark on project manager’s capabilities.
Project Team Feedback: Sometimes even if Project fails project manager is able to survive if project team and customer give a positive feedback. Then the reasons are different which are beyond the control of project manager. But if project fails and feedback is also not very positive, all axes will fall on project manager’s neck.
Lack of Technical Depth: A shallow or artificial demonstration of technical depth will not help for long. Although project manager has not to do technical activities on his own but his depth of knowledge will definitely help in driving the project in right direction and meeting targets.
Lack of business knowledge: Sometimes smart team members manage many a weaknesses of their project manager if he has demonstrated his strength in certain areas. But the lack of business knowledge is something that will certainly affect the product built, customer confidence in product and project manager’s knowledge. And if project manager is not clear about the business concepts of the customer, his presence in project, product and customer meetings will not be as effective as it is supposed to be.
Pareto Principle or Pareto Rule is quite fascinating in managing personal and professional life, time management, task management, self motivation etc. Crux is if you focus few vital issues in life you manage major part of your life better. The same applies in profession, organization, department function, and activity too. Only thing is it is to be applied objectively, and smartly.
Let us see how it can work in terms of software testing and quality control:
80% of software quality is maintained by 20% of programmers
80% of bugs in an application are written by 20% of developers
80% of bugs are fixed in 20% of time
20% of a business application accounts for 80% of bugs
The 80-20 Rule or most commonly known as “Pareto Principle” was first formulated by the famous Italian economist Vilfredo Pareto in 1895. The principle was named after him and still holds good almost in all the aspects of life. Vilfredo Pareto found that 80% of Italy’s wealth was with 20% of people. The crux of the principle is that most things in life are not distributed evenly. That may imply that in an organization 80% of decisions are taken by 20% of people. In a manufacturing unit 80% of production is done by 20% of workers. In an class of 100 students 80% of intellect lies with 20% of students. And so on…
In a software project management, based on 80/20 rule, it may be concluded that:
20% of code manages 80% of business in a business application
80% of developers write 20% of code in a team of developers responsible for writing a business application
20% of total productivity of a software developer produces 80% of results
20% of core business requirements, if focused and built well in an application, account for 80% of customer satisfaction
20% of bugs fixed make 80% of software bug-free
And so on…
HP announced end of life for WinRunner in February 2008. Many companies using WinRunner for years started wondering what to do. IT people using WinRunner got panicked and started convincing their organizations to switch over to another product, mostly QTP. Another product means another investment in terms of money, time, and expertise. HP itself started promoting QTP in lieu of WinRunner. From 1st August 2009 HP has stopped supporting WinRunner.
There are techies available having expertise on WinRunner. Some companies are still using it, others supporting these companies in using it. There are quite a few reasons for not immediately throwing this investment in the dustbin, instead keep using it with the following reasons:
5. You have in-house expertise on the tool: If you are using it in your organization for years, it is sure that you have acquired the expertise in-house on this product. Keep using it, it still produces wonderful results.
4. It is a matured product: It is already a well matured and stable product and can run for years without any external assistance required. As long as your products developed are covered by this product, use it.
3. You have experts in the market: Even if you have this product but not in-house expertise, there are plenty in market. Either absorb or outsource for using it.
2. You can move back to OST or manual testing: In worse to worse situation you can switchover to Open Source Tools for testing or can decide to go for manual testing. Open source tools are increasingly available and are performing quite well.
1. You have third party products with expertise in migrating from WinRunner to QTP: At a later stage when it gradually becomes outdated for your products, you can always use your repository of test scripts to migrate to another product, say QTP, instead of writing the scripts manually.
If answer to the first question is No, start moving into the direction of converting it to Yes. Answer to second question also has to be yes. Let us see what is meant by Imagine, Dream and Innovate and how to adopt them at workplace.
Imagination- The ability to create, and paint a mental picture of a new concept or situation can be defined as Imagination. In your case when you have to write business process in your code, it is primarily important for you to imagine business in right sense to embed it in your code.
Dreaming- Coming up with scenarios or goals that with a bit of work can be achieved. Your imagination of scenarios has to be set as a goal of coding it rightly and then ‘doing’ it.
Innovation- Making the impossible become possible by finding another way around a situation or problem. At times you have to find an alternative to optimize your code or choosing a right alternative out of many for a scenario.
A software division in an IT company is considered to be a profit center whereas the Testing division is considered as cost center. A set of developers develop software, get it tested by a set of testers, sell it in the market and earn profits. The credits and benefits on success of the software never come near to testers.
Let us look at a model where testing is offered as a service that costs. Developers claim their software to be ‘fit’ for market after they develop it and do some testing at their end. If testers now test the software and find out meaningful bugs that could have created a ‘sorry’ situation at customer end, then for each of this ‘meaningful’ bug, some amount should be debited from development division’s earnings statement and should get credited to testing division’s earning statement. And the real cost of product is development cost + bug finding cost + bug fixing cost
On LinkedIn an IT projects guy posted a question about plus and minus of outsourcing software testing for his software project. After getting 12 replies from various experts he posted his intention behind this question. The intention was to outsource development and testing to two different vendors (‘slicing’ in his terms) so that they maintain a self-check on their performance and resultant.
My next post immediately after his last post regarding his intention was that if he had clarified this right in the beginning at the time of posting his question, he would have got better replies rather than experts posting their views on pros and cons on outsourcing ‘testing’ or not.
Well my last post said there the same. And I added – “if you are slicing – then make 10, 20 or more slices and distribute it among all your vendors. This will also call for more self-checks”
What is your take on this?
If a person who develops software is software developer, why not the same person developing bugs in the software be called bugs developer. How many developers ethically perform the unit testing after completing development of a unit? It could be – None, a few of them, some of them, most of them or all of them. Some of them might be under the impression that they perform unit testing after completing a unit but the way they do it might not be really helpful in finding bugs. It might be just to satisfy them what they do and call it as unit testing.
If developers really perform unit testing, find out the bugs and fix them, in actual, then when testing is being performed by the testers why they have to perform unit testing again? Why can’t the testers skip it and save lot of money and time of the organization.
I have two sets of developers. Both bunches contain quite considerable number of developers. Let us call it first set of developers and second set of developers. Both sets have their own unique way of functioning and performing.
First set of developers work randomly with no documentation, no fixed plan in place. The requirements come invariably from different directions and as soon as a new requirement comes, the developer changes the priority of his tasks based on the influence of the requirement generator. In this manner after sometime the developer loses his track of what elements he has addressed or fixed and which are pending as there is no systematic way of recording requirements and their completion track.
Second set of developers works absolutely in different manner. They record all the requirements in one place, prioritize them, maintain software version control, mark each requirement as completed only after their satisfaction and testing and reassess their requirements at each day end. This way they never lose control and are absolutely clear on what is balance and estimated time required.
Surprisingly first set of developers spend more time at work place but product less result as compared to second set of developers.
Let us compare this with two similar cars with their fuel tank full. First car is driven randomly on road for whole day without any specific purpose and exhausts its fuel at the end of the day. Second car driver is sensible in planning his route before starting his journey, reaching back early in the evening, finishing more tasks with still some considerable fuel left in his tank.
If you, as a project manager, are fond of thunderstorms, volcano eruptions, etc. it is ok howsoever you drive a project. Otherwise look below at six indicators mentioned below. Even if one of the reason prevails in your project’s lifecycle, manage it, get rid of it, immediately, before a small wound becomes a big septic.
Irregular releases – when plan or commitment varies from actual delivery inconsistently. Some plans finish in time, some with small variation and some with huge variation. It means there may be a volcanic surprise.
Drop in quality or inconsistent quality of product built: Testing is happening, bugs are being reported but if the product coverage is incomplete while testing, or volume of low quality bugs reported is higher as compared to some severe bugs skipped in reporting, there is a mishap going to happen for sure.
Wrong cycle time estimations: if your estimations are going haywired, you plan milestones and achieving them itself if becomes a project, and go beyond control for achievement – there is a mess, in a big way.
Unplanned expenditure like overheads, delays: all these estimates going wrong, will call for extra time for team members and extrapolation of deadlines – a clear indication of a thunderstorm coming on the way.
Predictability: if your team members lose their commitment level, your predictability will become a big question mark for your management and for the customer.
Reliability: your team members’ reliability will invariably start decreasing the moment all this starts happening – and so will be yours.
What is cycle time?
Cycle time in terms of a software project is the time taken from its initiation to handover. Another measure in terms of commercials could be order to execution to recovery… A project lifecycle in other terms
Do you measure it?
You might be banking on your experience without any data in place!
What is your definition of project cycle time?
In your opinion it might be what I said above. It might be nothing for you! Or might be some other definition!!
Is it merely based on gut feeling and past experience?
If you are not recording the development of a project from one stage to another, your cycle time measurement sheet could be a blank sheet.
What is objective measurement?
As we all know each project has several phases. Each phase is divided in milestones. One measure could be the time taken by each milestone from it planning to completion. There would be certain parallel activities. On the other hand some activities could be incremental in nature. Measure accordingly.
Do you segregate your similar type of projects?
You might be running different types of projects. Some could be similar to each other and may fall in one category. Another set of projects could make another category.
Do you know average game is successful for similar types?
An average cycle time worked out for similar set of projects falling in one category will work best for you for your future estimations and meeting those estimations. Else average may not work well and estimations may go haywired.
There is no end to an application. It always asks for a new feature, alter in functionality, addition/ change of business rule etc. With any change in the existing application running in a live environment, the change needs to be tested for all aspects of quality before putting it live. The question comes what should be the scope of testing in this case. Should tester test only for the change part or the complete application?
A change in application small or big is always going to mark an impact on the whole application. Even if not on the whole application, to some extent at various places in the application. Sometimes it could be beyond the knowledge of developer.
Therefore, in my opinion, it is wise to test to whole application even if it going to take more time and efforts to minimize the risk of impact of ‘change’ in the application.
Lot of efforts can be saved in terms of time and money if we reach to a stage of ‘first time right’ in application development. It has been proven largely that no good application can be built and released without extensive testing. Testing is not developers’ ball game – this is also a well proven fact. Reasons are many as far as it is concerned that why developers can’t build a bug-free application, or why can’t they test on their own. We are not going to discuss those reasons here. Focus here would be on what developers should keep in mind while building an application that it requires minimum efforts and time in all testing stages. As we all know the cost of bug fixing goes manifold, depending on how much distance (in terms of time and project stages) it has covered after development for bug identification. The bugs identified at a later stage, say during UAT cost more significantly as compared to the bug identified by the developer himself immediately after writing a code. Few qualities if a developer acquires and keeps in his mind while writing codes would not only benefit him but the organization he is working for and their customer also for whom the product is being built.
1. Commitment to application performance should be kept in mind while writing a code.
2. Clarity of business requirements and rules/ validations that are being translated into the application with real aptitude of business and not a developer. Don’t imagine and build. If there is some lack in clarity – discuss, record and build.
3. Treat yourself as the business process owner and end user – and build the application accordingly as if you have to use it. Don’t think yourself as a bartender, think as if you are preparing the drink for yourself.
4. Collaborate and build – rather than building in isolation- collaborate with other developers working on the application, the end user, and the testers.
5. Optimize your code – don’t just write it. There are n numbers of optimizers almost in all technologies. Use them and build a strong application in terms of functionality and performance. Be quality focused. Don’t do efforts that call for more efforts later.
6. Be focused. Don’t work on various applications development at the same time unless it is too mission critical.
7. Gain customer experience after launch of your application. It will certainly help you in your future builds. Build a customer satisfaction metrics.
8. Don’t take short cuts in fixing bugs – whatever stage they are identified. That way you will build more bugs while fixing identified bugs.
9. Work like a champion. There is a difference between playing a shot and playing an accurate shot.
10. Be loyal to yourself, your organization and your work.
UAT or user acceptance testing comes as the last exercise in software testing lifecycle. It is probably the first phase or beginning of customer preparing to takeover the charge of the product. Actually this is a sort of test drive by a perspective buyer who has studied well about a car, has made up his mind to buy it but wants to satisfy himself by actually sitting in the car and driving it. Even if you have made up your mind to buy a particular model and after sitting in the car or after having the test drive, something does not suit you, the decision can take 180 degrees turn.
The same usually does not happen in UAT because the unlike car the software has been built as per customer specifications. UAT usually includes interfacing (if any), look and feel, ease of usage, functional requirements, integration test etc.
Like a car test drive, here also use runs the complete software to assess if the software is meeting their requirements completely or not. It is the last place where the user gets to determine whether or not the software meets his or her requirements. But one thing is very clear – whatever defects occur during UAT, their fixing cost goes manifold as compared to the same defects occurring during the initial building of software.
Main difference between the earlier testing done at development place by testers is that here the business process and validations built in the software will be checked by a business process owner with real data.
Probably in this busy world, if the end user, by any chance is available during the development and testing phase to do appearance, functional, process and validations testing, it would save a big amount of time and money.
1. Evolve, develop and freeze standards.
2. Keep a breathing space by not developing too rigid standards.
3. Live with open mind. Always be open for change in standards, if it is for improvement, and if it makes sense.
4. Let everyone involved in the projects have the same drink at the cocktail. Distribute the standards to all. Make it available on demand.
5. Understand the usefulness of standards. Don’t follow them just for the sake of it.
6. Let it not be too bulky in terms of documentation work involved in following standards. Standards should not supersede projects and hamper their progress.
7. Let everyone use a common terminology defined in the standards so that everyone speaks same language. Not that everyone using their own project terminology.
8. Standardize but don’t compromise.
9. Goal of standards should be achieving success in projects in a better way.
10. Follow standards.
1. Purpose: The main purpose of the application for which it is built need to be defined.
2. Load Test goals: Identify what you want to achieve by load testing, what are your primary goals to go for load testing. Define all your goals. One example could be – login time for 100 concurrent users should be less than 2 seconds.
3. Architecture: The major concerns in the overall architecture about achieving these goals
4. Simulations: Think of all possible scenarios in which the end user will be working when application is put live. This could include some batch processing, some different client types etc.
5. Past experience: don’t forget your past experience with load testing solutions. The results, the shortcomings, bottlenecks – all could lead to a new learning.
6. Topology: Define all infrastructure components and the network which links them.
1. Framework: What is the framework being used in the application? For example it may be .Net or J2EE or any other.
2. Servers: What all server services you are using to run/ use this application? What standard server applications are used for Presentation, Application and Database level? Some examples could be MS SQL Server, Oracle, Apache, Websphere, IIS etc.
3. Number of machines: Are these servers running on same machine or different machines. If different machines how many machines and what servers specfically on which machine.
4. Details of these machines: The details of the machines running all these server applications or server services. Details would include Operating System, Hardware configuation – like Hard disks, RAM, processors etc.
5. Access details: Last but not the least important point is how the users will be accessing the application – via LAN, through dial-up, through internet, through some specific port and third party tool etc.
Which Server: Where is the load testing intended to be performed? Is it the test server, production server or staging server where load testing is to be performed? If it is being performed on Production Server, it is ok. Otherwise if it is to be performed on test or staging server, be careful that it is as near to production server in terms of configuration and setup as possible. It may give wrong projections if there is a wide gap in environment which is to be used in real versus the environment on which the test is performed.
How many Tiers: It is very important to understand how many tier application is, on which the load testing is to be performed. The n-tier count should include the client also.
What Browsers: Be very clear and specific on defining what all browsers are meant to be used for running this application (if this is a web application). The browser and its version is very important to conduct the load testing as each browser behavior, architecture and performance varies. Even between the different versions of the same browser.
What Protocols: Whether this is a client server application, web based application or n-tier application – identifying what protocols you are using to run the application is very important.
A love for music – if you can move to music, you can dance: Same applies to testing also. If you have love for testing, and you can move to its music, you can be a good tester.
A sense of rhythm: If you understand the product well and move along well, in a right rhythm, you win the game.
Willingness to give it a go: A true willingness is a must for a tester to be successful.
A bit of natural ability doesn’t go astray: Don’t forget that anybody can not be a good tester. There is a natural ability that gives you an extra edge than others to be a good tester. Only you need is to catch this ability in you and keep polishing it.
Work hard enough: Hours of hard work and dedication is a must to be a good tester.
The responsibility of a tester is to ensure the peace of mind of the end users who are going to use the software product. Another target should be to safeguard customer’s investment in the product. In order to discharge this responsibility, the tester should focus all his skills on understanding customer requirements and measures to map it with product built by writing appropriately comprehensive test cases, performing test and preparing a detailed self-explanatory test report. Complex business requirements will require extraordinary level of understanding. Keep in mind the budget and time constraint for performing testing without compromising with the coverage or quality.
The overall impact of testing should be clearly visible on the product built. The bugs reported should bring a sense of belonging with the developers and customer users. Clarity of bugs in testing report should not only help developer to fix defects properly but guide him not to repeat the same type of issues in future releases. This may happen slowly but gradually should help in increasing developer’s performance and quality of code writing. After all the development team should start taking testers as their best friends and well wishers instead of someone who is trying to find out the weakness in code writing.
People are not wrong, processes are. People in an organization do what they are told to do. Organizations who hit on people at the time of failure are not doing the right thing. It is the process that is to be blamed not the person with Q-tag. As long as quality is considered to be the child of people with Q-tag in the organization, and the culture is so around, the non Q-tag people will keep themselves enjoying building a product without quality. Why not blame the developers for developing so many bugs along with the requirements building. Why not blame the analysts who could not understand or translate the requirements correctly. Why not blame the management for not putting the right people and process at right place. This blame game itself is merely a scapegoat in any organization until the focus shifts from people to process enhancement.
Analyze the failure as a team involving all the stakeholders. Management, business analysts, project managers, project team, development team, testing team, implementation team have to sit together and brainstorm. Failure is to learn and not fail again. Mainly failure can occur due to following three factors:
Teams do not work and deliver in desired fashion – required innovation, awareness, training, cohesiveness, team building, and collaboration
Wrong people in the team – HR and management has to play a major role and look into the recruitment process.
Failed to build right product – needs training, demonstration, pairing, indentifying right people for right job.
In any software development and implementation company there is always a need of quality assurance and quality control people who own the responsibility of setting the right methodology and framework for development and implementation (QA), bugs identification and product approval (QC). Usually everyone in the organization has an inherent feeling that the quality is the responsibility of only these few persons belonging to this Q-department of the organization. Business analysts understand the customer and business requirement, hand it over to development team for building the product. Development team develops the product, and hands it over to QC team. QC team tests the product, finds out the bugs, and submits the report to development team. Development team fixes the bugs and returns the product to QC persons for re-testing and verification. After few exchanges between development and QC team, the product is declared as defect free and is released or launched for implementation.
If top management, development team, business analysts, implementation team and all other stakeholders think that quality is just the responsibility of only the persons belonging to quality department, they all are wrong. If Q-tag is limited to only a limited persons belonging to Quality department among all stakeholders, a product can never be built with great control on quality aspect.
Q-tag has to be on each of the stakeholder in a software project. When each and every person wears a Q-tag – the analysis, building, testing and implementation will be more justified. Otherwise there will always be a big question at the time of failure of a product build – that who is responsible?
As in a software project, the complete project lifecycle comprises of different phases. Similarly the performance testing lifecycle also comprises of various phases. Performance testing is usually, as the name suggests, is done to evaluate or examine the performance of the software product with the help of some tools and certain parameters. The performance testing may include components based on the requirements that include – load, volume, stress etc.
Five prominent phases of performance testing can be:
a) Test Plan
b) Test strategy
a) Load Modeling
c) Benchmarking criteria
a) Benchmark execution
b) Other executions
c) Reports generation
d) Reports Analysis
a) Reports reconciliation
b) Reports compilation
c) Final Test Report
d) Bottlenecks identification
a) Execution of recommendations
In my June 15 2009 post – “Do’s (+) and Don’ts (X) in Project Management” (https://itknowledgeexchange.techtarget.com/quality-assurance/do%E2%80%99s-plus-and-don%E2%80%99ts-x-in-project-management/), I got a query and am posting it here as it is – As mentioned by you in one of your post “Single constant in business is Change”. But often software vendors are too bound by the requirement documents that they fail to gauge the change in the underlying business need that they are trying to cater to. I understand that there are time and cost considerations, but many a times the attitude is not conducive. Could you throw some light on this in your writings…?
First of all let us be clear – we can not just record requirements and seal it. And then stay isolated from customer in building the software. One fine morning communicate to the customer that the product is ready and you are reaching customer site for product launch. If you feel that during the conceptualization, build and test phase customer has no role to play – you are wrong. If the customer thinks just by giving the requirements they will get a good product meeting all their expectations – customer is wrong. Both have to be in tune during the product development. If vendor fails to gauge the change in underlying business need that they are trying to cater to and think if they do so, it will increase their cost and time – again the vendor is wrong. By not doing so – they call for more time and cost. By not doing so – they surely call for more discrepancies, ambiguities, delays and failures. A small investment of time and cost during the build phase to involve customer and take his consent at every step will definitely lead to later on investments, time delays etc.
If attitude is not conducive, it has to be. Customer has to realize that. And customer has to realize the importance of getting involved in each phase of the project rather than assuming that it is vendor’s call and everything will go fine.
The requirements keep changing, as the business. Only thing that varies is the pace of change in requirements. Overlooking changes happening in the business process at customer end after freezing requirements will result only in undesirable product. Both customer and vendor have to understand this and keep overseeing the changes rather than overlooking.
If changes are not recorded and incorporated in time, customer is always going to blame the vendor and may be vendor gets exchanged with another vendor for the same project.
At the birth (inception) of a new software project the project manager is puzzled and confused just trying to gather and understand customer requirements. He starts like a wanderer in the dark islands of customer for collecting various requirements and understanding their business norms. The moment he is able to collect this information, he aggregates to get the stock of ‘total requirements’. Understanding this makes him getting into ‘catching the rhythm’. Now comes his planning phase where he has large ‘in depth’ discussion with his development teams. By identifying different milestones for various project stages, he prepares a project plan. At this stage he is supposed to discuss this plan with customer and get ‘in sync’ with him. Once approved from customer, he breaks this plan into different stages plans to hand over respective plans to their ‘stage owners’. Development plan goes to development manager. Quality plan goes to QC head. Implementation plan goes to implementation head (himself mostly). Accordingly each stage milestones are identified which might be more than overall project milestones shared with the customer.
Then starts the actual war phase – the development phase. Meetings, discussions, brainstorming, logs, monitoring are all the weapons of this war phase. A product gets birth during this phase which gets vetted by quality control team. Documentations are also integral part of this phase.
Various test phases occur during and post development or build phase. Smoke testing, Unit testing, module testing, performance testing, security testing, load testing, functional testing… to name a few.
Now the baby has started walking. So baby is dressed well to take it to the grand function taking place at customer site. The function is ‘Implementation’. This is a long function, comprising of baby show, meetings, discussions, demonstrations, recordings, UAT, training etc. Once the baby is able to walk neatly in front of all the guests at the function, all praises fall on baby. The ceremony closing takes place. Project manager adds one more bullet of ‘confidence’ in his gun.
And starts over again for a new project.
A software tester evaluates software based on certain parameters. These parameters are set as per product, customer and organization requirements. Testing could be just of functional features or include load, performance and security. For any parameters a tester has to work as quality analyst to understand requirements, features and accordingly build test cases and perform test. This is the quality control part. On quality assurance front the quality team has to build standards for requirement freezing, planning, development, implementation and post implementation phases of a project.
A software tester at various stages of a project gets on to the job of a Quality Analyst by performing following tasks:
Analysis of customer requirements: The first and foremost analysis required is that of the customer requirements to ascertain if it is complete, detailed and free from any confusions, ambiguities or equivocalness. Any flaw in requirements will certainly lead to a big disaster at a later stage. Unclear requirements are not difficult to build, but are difficult to manage. Every requirement should be in black and white. Each line should be very clearly documented such there should be nothing hidden between the lines.
Analysis of Product Functionality: Requirements documented and product built has to go hand in hand. It should not happen that requirements and product speak differently even a single line. Usually while testing functionality of a product, tester forgets to refer to requirements documented, or asks developer about the functionality. The developer will certainly explain him the functionality he has built not what exactly has been mentioned in the requirements document. If this happens, it will certainly cause a big blast at implementation or acceptance stage.
Analysis of Product related documents: There are many documents prepared during the project. Some are meant for internal use, some are prepared for customer. All these documents need to be inspected thoroughly and neatly.
Analysis of test results: Test cases are built to perform tests resulting in bugs report or test results report. A thorough scan is must to ensure complete coverage and thorough testing. The report should be detailed in all respects in terms of clarity and coverage.
Analysis of Testing Process: The testing process once establishes need to be revisited again and again to improve further at every go. Once established does not mean it is ultimate and best. Improvement has always a scope howsoever best your process or product is.
You, as a project manager are most crucial factor of a project being the key driver. Your commitment matters most as it can do any wonders. Your real commitment can put fire in your team member’s belly to run the project full swing. But if your commitment is merely Pretence, it can wipe off the energy and zeal of others in the team. It can trigger on the pace by acting as a catalyst or adversely trigger off the pace by decelerating it.
There have to be certain checkpoints in your thought process that may ascertain if your commitment is in right direction or not. Let us see what those points as mentioned below are (against each I am mentioning if the commitment is PRETENCE or REAL).
1. Believe – “All stages of a project are crucial and important as far as the overall success of a project is concerned.” – Your commitment is REAL.
2. Believe – “Each milestone matters and need to be converted to win-win for all for a smooth sailing towards this bigger goal.” – REAL
3. Believe – “All stakeholders have a stake and a part to play in the project. Overall every task and success has to cross ‘mine-yours’ barrier to reach at ‘our task’ and ‘our success’ stage.” – REAL
4. Are – “Intentionally creating conflicts.” – PRETENCE
5. Are – “Avoiding ‘important to be resolved’ conflicts.” – PRETENCE
6. Are – “Unnecessarily trying to prove your point (imposing your point) (putting yourself over the project). – PRETENCE
7. Have – “Lack of knowledge and understanding.” – PRETENCE
8. Are – “Giving more importance to emotions and feelings over logics.” – PRETENCE
9. Have – “High level of actions but poor planning.” – PRETENCE
10. High level of planning but with poor actions. – PRETENCE
11. Framework is not important – people, tools and management are. – PRETENCE
12. High performing teams are different from highly organized teams. – PRETENCE
13. Best practices do not require improvements. – PRETENCE
14. 100% test coverage means correctness. – PRETENCE
15. Automated testing is a silver bullet – PRETENCE
Projects do happen with the beliefs and actions mentioned above. Whether they end up in good or bad state, I am not sure. PMs do exist with these beliefs and actions. But they have to change themselves over a period of time, I am sure.
We learnt in earlier two posts about the strategic decision of a management to outsource a complete project or part(s) of a project depending on certain factors, and the factors respectively. In this post let us see at the various components of a project that are most widely outsourced or otherwise we can say these are the components of a project which can be outsourced. It is very less often that a project awarded to a company is totally outsourced.
We are talking about outsourcing an activity of a software project. The most important components of a software project can be listed as:
Requirement analysis, gathering and freezing
Design, development and testing
Implementation, training and hand-holding
Post implementation support
These can further be broken into various sub-components under each component thereby creating a tree like structure to have a bird’s eye view of any project. Planning and execution for each phase or component comes next with control everywhere.
Outsourcing mainly is the resultant of constraints in an organization.
Outsourcing is neither bad nor good. It is a strategic decision based on certain parameters taken by an organization to offload some projects or part of a project to a third party. Offloading certainly comes with many boons and many banes. Offloading should be managed very carefully as the project delivery ownership still remains with you even if you are offloading complete project. Insertion of seriousness in the outsourced vendor regarding the project, requirements and its timelines is very important. There are many parts of a project that can be outsourced and as said above, the decision is purely management’s to decide what part of project is to be outsourced and to be awarded to whom.
Definitely in offloading the vendor-customer chain becomes longer depending on number of components of a project outsourced and the number of vendors involved. Even for a single component of a project there can be more than one vendor. And for a number of components of a project there can be a single vendor.
Offloading or not will depend on many factors that can be listed below as:
1. High number of projects
2. Shortage of skilled manpower
3. Shortage of right skilled people
4. High engagement of required persons
5. Lack of resources (in case of testing tools or otherwise)
As already has been discussed, User Manuals play a crucial role in any software project and are the solid bonding between product, users, customer management and vendor project team. The stronger this bond is the more comfortable and happy each stakeholder will be. Each player has to play a crucial role during the game of building of User Manuals. Both Project Management teams have to work hand in hand for that.
The quality of User Manuals has also been discussed in my earlier blog (Six Pack User Manual – how to build).
Let us discuss below the role to be played by the respective project managers in this regard:
Vendor Project Manager has to ensure that the user manuals are built along with the product development and are vetted/ approved by their QC team via a walkthrough. If need be they should as for draft manuals in build phase. If there is a foreign language issue, the Project Manager should also ensure that the user manuals are required in their local country language.
Customer Project Manager has to check that user manuals are complete in all respects, and checked thoroughly before handing it over to the respective key users
Six major key points that should be kept in mind while preparing User Manuals can be as listed below:
1. Simple – Never expect your product users to be as knowledgeable about your product as you are. Remember when you first time tried working on this product, how ignorant or untrained you were. The expertise that you have on your product has come over a period of time on repeated usage (in terms of implementation and training) of this product. So keep your User Manuals as simple as possible. Don’t use technical jargons, or tough language clauses. And even if you have to use them, break them in fractions or explain them in detail.
2. Extensive – Prepare User Manuals as detailed as possible. Use less text and more visuals/ diagrams/ flow charts/ process charts/ drawings/ illustrations/ screenshots/ pointers, highlights, captions, bold/ caps and italics to make is more user friendly, more understanding and less cumbersome.
3. Crisp and Fluent – On the other hand the User Manuals that you are preparing for your product users have to be crisp, up-to-the-mark, relevant, in-tune with the product, devoid of useless stories.
4. Quality – Imagine an airplane handed over to a person who doesn’t know ABC of even kite flying keep aside an airplane. And with the help of a user manual if he is able to take off and landing, the whole credit goes the User Manual. Build your user manuals like this.
5. FAQ – This is an important section of your User Manuals simply explaining two things. First is what – if situation that means what will happen if you do a particular activity. And second is if – what that means if you want to perform a particular task, what you are supposed to do. Understand the difference between what – if and if- what and build your FAQ section accordingly.
6. Right Person – Don’t imagine. Not at least that a good actor will be a good story writer or script writer. These are different jobs. Let developer do the development and chose the right person for understanding the product correctly and design User Manual.
The product owners or stakeholders might be many in a software project, but the real frontrunners who drive, run and use the software product post implementation are the key users and other users. It is their feedback that matters most. They must be the most comfortable lot on usability, functionality, reliability, stability, durability, resultant outputs in terms of reports and analytics, feel and look of the product. They are the one who are going to matter most in the success or failure of a product during and post implementation. The best tool to give them comfort, satisfaction, confidence and support is the User Manuals that they refer to most of the time during post implementation live run.
Even if the product is excellent and trainings are most rigorous ones, in absence of the project implementation team, every now and then the key users will be seeking help from User Manuals. User Manuals are the supporting agents for them at all times. That is why User Manuals have to be perfect in all aspects.
Infact a user manual prepared for product users should be such treated as a replacement of implementation and training team right from the moment the project is signed off and users start using the product in real business scenario.
+ Single constant in business is Change
X Single constant in business is resistance to change
+ Change means shifting to a better comfort zone
X Change means shifting away from comfort zone
+ Recruit well in advance before the project start time
X Recruit at the time of start of project
+ Give to new recruits the ample time and guidance to charge them well
X No time for relaxation should be given to employee
+ Relaxation re-energizes, recharges the employee
X Relaxation destroys the enthusiasm and fire of an employee
+ Retention of good employees is worth spending in case of no projects
X No Projects No Retention
+ Importance, Encouragement, Attention, Satisfaction, Opportunity to grow all these
are important for employee and it pays back to an organization
X Organization pays, Employees work
+ It is better to monitor new employee to check its readiness for appropriate job
X Recruit and assign directly to project
+ Shortcut is the longest route to achieve success
X Omit steps, adopt shortcuts and attain results
10. When you start getting increased number of bugs as compared to earlier releases
9. When your developers stop thinking about quality in code
8. When bugs pass through QC unnoticed
7. When you stop acknowledging quality efforts and start giving more importance to speed and volume of writing a code
6. When quality and development teams are not in sync
5. When you start feeling QC is a waste of money, efforts and time
4. When quality aspect is given least time in overall project
3. When management starts overlooking then overseeing
2. When customer does not get grossly involved in performing UAT
1. When above mentioned in point numbers 2 to 10 starts spreading in other projects
In the previous post we learnt what all could a software project could lead to in presence of higher scope of software development at customer site during implementation phase. Let us see what all factors are responsible of insufficient requirement gathering during business study phase of a project.
1. Incompetent team (vendor): Any project calls for a project team. A good team at both ends is important. Both teams have a substantial role during a project. If key users selected at customer site for delivering process knowledge to requirement analysts (vendor), there are higher chances of wrong information driven. The key users chosen should rightly be the actual process owners not essentially be at the higher level in the organization.
2. Incompetent team (customer): A right selection of project manager and analysts is equally crucial. The team has a short duration stipulated for gathering requirement information and understanding business processes. And mind it, this is not an individual race, it is a relay race, and weakest link will decide the overall result of the project.
3. Non commitment (customer): Well the process owners are the key users, but if are not serious or have not received the right message from their management may not be seriously committed to the vendor team. A non committed team will not be able to impart right knowledge, processes, practices and information required critically for a project.
4. Non commitment (vendor): A team of good committed project manager and analysts is equally crucial for gathering information. The previous experience on business domain and success factor definitely counts in.
5. Lack of time (customer): Key users identified if are engaged in other important projects running in the organization will be always short of time for providing right information to vendor project team.
6. Inappropriate discussions: In a limited timeframe of business study, there will be limited discussions. So all discussions should be crisp, purposeful, well driven and result oriented.
7. Improper documentation: Well explained requirements if not documented properly may lead to a wrong, incomplete product. Right people with right business and process knowledge have to acquire a good documentation skill also to lead the project to right direction and grand success.
8. Top Management involvement (customer): If customer management thinks that providing key users completes their job in the project, they are wrong. Their involvement in all important meetings is as important as the business is.
9. Project Organization: A right architecture of project team is very important. The project sponsor, project directors, project managers, key users, developers, implementers, technical leads from customer and vendor respectively should be the right mix of people.
10. Change in requirements: Any change in business or process during or after the requirement study phase has to be communicated to vendor team well in time, so that by following the change management procedure, the changes are taken care of properly.
Ideally, in a software project, for an offshore customer, the requirement gathering phase should be given an extra care to understand to a maximum extent so that the product developed and tested when ready for implementation at customer site requires no development. Practically, it is very difficult to achieve. The more it is left open to be handled at customer site, the higher is the chance of customer and vendor getting affected.
So that means the higher scope of development at customer site will change the team requirement and affect the overall project. Let us see what major factors will get affected and where could those lead the project to:
5. Money Factor: The most important factor is money. Any scope of development at customer site will need extra developers in the team besides the implementation functional leads. This will lead to an extra burden in terms of cost of technical staff not only in terms of developers but also testers.
4. Time Factor: Delay in project is inevitable in these circumstances when
requirements will be clearer at customer site at the time of implementation and
thus will change the overall implementation plan to accommodate development
and testing in between.
3. Quality Factor: Poor quality will be a major concern. Everyone knows you can not
take a bundle of testers at implementation site. That means limited testers will be
in under a tremendous pressure to release the product at the earliest and hence
may not justify with their job. This may lead to lot of holes in the pot –
intentionally left or unintentionally skipped.
2. Exuberance Factor: Enthusiasm, tempo, momentum, pace will all start
diminishing and in turn start creating frustration and dissatisfaction at both teams
1. Product Factor: Product Failure, if not, then delays and unpolished product is
always visible on the platter under these circumstances.
All these factors, being inter-related will have recursive effect on each other thereby increasing each by manifold and everything may turn into a complete mess in place of a successful project.
The Post implementation review is usually filled collectively by customer project team and signed by the project sponsor, who is usually one of the senior members of management. This form is to be submitted back to vendor (after filling) to serve the purpose of – sharing experience, product evaluation in live scenario, user’s satisfaction, assessment of management itself in understanding the goals assumed or perceived versus the actual goals achieved. The components of this review and WHAT, WHY and WHEN have already been discussed in earlier posts.
Let us now understand – what are the guidelines for management (customer), project team and key users for filling in the Post Implementation Review:
This is not a sports match: The project should not be taken as a sport match where one has to be declared winner and the other loser. In a software project either both teams win or lose. So intention of the review should be taken in right spirit and with right approach.
Be Courteous: Don’t be too blunt while filling in the replies. It could hurt the sentiments of vendor. After all if something has gone wrong, it needs to be analysed and resolved rather than throwing stones at each other. Remember that someone had made a choice at your end for this product and that is why it is here. On the other hand if all has gone well, be generous in praising.
Don’t be a critic: Don’t analyse this product with a critic’s eye. Your goal while selecting this product was that you had a trust in the vendor and this product. Evaluate more on the basis of your key user’s confidence, satisfaction, results, improvements, and product strengths.
Be Transparent: It is good to be courteous, non-critical and not-blunt. But don’t shy away in exposing any weaknesses in the product, the methodology adopted, trainings imparted, or vendor’s team efficiencies. Strike the right ball in right hole at right time. After all it is learning for both the vendor and customer that is helping in product and people maturity.
Ask for more: If you are happy with the vendor, don’t hesitate in asking for more features or functionality (may be as a bonus, if you are able to pamper the vendor enough!)
Both vendor and customer have to understand that the real journey of a product starts only after a successful implementation of software at customer site. The challenges get a new meaning once the project team leaves the customer site after project sign-off. Both the managements can celebrate the successful completion of project. But it is better to hold on this celebrations for a while, till customer really learns to live with the product, starts tasting the real fruits of the product committed earlier. Let the key users start managing the show on their own, run the complete process in live scenario and get satisfied with themselves and the product. The project team that comes at customer site for implementation has a role related to post implementation review prior they go back home after successful closure of project. Similarly the management (customer) have to clearly understand their critical role once the vendor project team goes back after the completion of project. It is the key users and management (customer) now to run and manage the show. No doubt that the vendor team is always at the back to support and assist them in achieving this. Let us assume that the project is over and team is about to leave the customer site. At this juncture it is the prime responsibility of vendor and customer project managers to shake hands and understand the importance of post implementation review in following terms:
The Project manager from vendor side has to hand over the post implementation review plan to customer at the time of implementation sign off and explain clearly each and every agenda of the post implementation review. The clarity in terms of product, managements, users, usage and product is very important for assessment purposes after a stipulated period.
The project manager from customer end has to ensure to get the post implementation plan for feedback purposes at the time of implementation sign off. This will not only leave the project open for another some time in terms of providing a lease for customer team to assess and experience the product in complete isolation in their own terms.
The purpose ultimately is to share the overall health of product, the key users, and the management.
What is post implementation review? When it should be done? Why is it required? All this has been discussed in last three posts. Let us now understand what ideally would be the components of a post implementation review. As discussed earlier, some components can be answered immediately at the completion of a project with a formal closure. But for answering some other components, customer needs some time for project to run in the absence of project team, experience it, feel it, see the user’s reaction on the way things are happening in the product when it is in use.
Infact before filling the post implementation review, it is advisable to use the software at its maxima, as extensively as possible, involving all key user’s areas, using all inflow and outflow procedures. The basic guidelines required while filling a post implementation review can be take up in my next post. Let us also understand that this questionnaire has to be quite elaborative and descriptive. Let us first understand the essential components of Post Implementation Review for a software project. The questions can be built as per the need of the vendor and product. The essential components to be covered are:
1. Effectiveness of the Project Team
2. Effectiveness of Customer management in managing Product understanding and implementation
3. Effectiveness of Customer management in understanding the requirements, building the product, selecting the right team and procedure
4. Change management during the complete cycle
5. Issues management
6. Preparedness of key users and management for accepting the product
7. Communication skills and management of vendor team and management
8. Risk perception, risk management
9. Product effectiveness, usefulness, maturity and friendliness
10. Customer management eagerness for future business to the same vendor
These are the core components based on which the elaborative questionnaire can be build to assess customer satisfaction over the product, vendor and team.
When is to perform a post implementation review? A witty answer could be – obviously after implementation. Ha! Definitely a successful closure of implementation could declare a project closure with a formal project acceptance report or project sign off. So shall we have a post implementation review as soon as we have a project sign off? Nah! That would not solve the purpose. Give an appropriate time to the customer team key users to settle down as the captain of the ship and sail it smoothly. One day or one week smooth sailing will not tell you the turmoil or undercurrent storm about to come in future. Correct. Then future is too long. That means keep waiting for turmoil. But mind it, all ships sailing in the sea do not experience storm. Similarly all products after implementation and project sign off do not guarantee a serious disaster.
Product performance in actual sense requires a certain timeframe to establish and to give confidence to end users. Some part of post implementation review related to team performance (implementation) can be answered quickly, maybe immediately after the project closure. But the other part needs a considerable amount of time to understand the product from different perspectives and accordingly present a right picture in the review report.
Certainly, then, atleast a period of minimum three months is required to experience the product and then fill in the post implementation report. Ideally, I would say, wait for six months, use product in all respects, aggressively, and then the top management need to sit with their key users and project board to evaluate, assess and fill the post implementation review.
A successful implementation does not ensure the completion of project. The reality starts when the implementation team has gone back and users have started using the project in full swing with the help of training material, learning, knowledge and product. The health of users in respect of using the product is sustained, deteriorated, or improved will depend on many factors. A post implementation review is always important to understand the users understanding, pains and delights during this tenure. This will translate further into management’s pains and delights. The overall delight weightage has to be higher than the pain. In a blow of project implementation phase users might feel quite confident regarding the product features, functionality and ease. When the whole thing falls upon them, it usually drive them in confusion, wrong actions or stoppages. Or a smooth drive.
A post implementation survey will help in a real measurement on two fronts. One front will be users’ understanding, ease and comfort (or vice versa). Second front will be product’s stability, performance and functionality. It also will assess the after effects of a successful project implementation.
The conclusions could be misleading although and will require a deep analysis. A user’s lack of understanding may spell into products inefficiency or the opposite of it.
A Post implementation review is conducted after a substantial period from implementation sign-off. This review is to ascertain customer management’s and users’ experience on product in absence of product implementation team. The product has been implemented successfully and the team is gone. Ofcourse sufficient learning, knowledge, exposure and material have been imparted to users by the implementation team.
Post implementation review by the customer management with key users will spell out – user’s depth in using the product and product’s reliability and stability. The format of post implementation review document should be designed by the vendor management for the purpose of understanding the status of project after a certain period of usage post successful implementation. Filled by customer management in agreement with key users’ experience, the report is sent to vendor for their assessment.