Try finding the cost of hardware components like RAM, harddisk, floppy drive, cd-drive etc. a decade back and you will be shocked to notice their price at that time as compared to today’s price. And just go back to that era and check the warranties we used to get on these components – 3 months, 6 months, or maximum a year. Time changed fast with the change in technology. Hardware grew technically stronger and more rugged. Gradually came the era of cost and warranty war to consumers from different hardware vendors. Prices reduced, technologies improved and warranties increased. Warranty of a harddisk or RAM increased from 1 year to tow years or so. Finally, today if we buy a hardware component or a PC/ Laptop, most of the components inside carry lifetime warranty. Though each component has a pre-defined lifespan from its manufacturer or assembler which varies from component to component from 3 years to 5 (or higher).
The software development cost a decade back was huge. Warranty used to be hardly three months or six months. Technologies have changed; prices have rather increased but warranty on software remains more or less the same.
Why can’t we treat software as a hardware component – with improved quality, increased stability, higher performance assured, more trustworthy and a pre-defined life-span? Why we still fear to encounter bugs post release of the product? Why we still give so less warranty to our customer? Is it that we still don’t trust the quality or technology which we embed in our software built for customer?
Hope to see soon an era of software having lifetime warranty.
Yes, you are not mistaken. We are not talking about the ‘will’ here; we are talking about the ‘wheel’. This is not ‘wheel of fortune’ giving you some pleasant and unpleasant surprises. This is ‘Quality’ wheel. If you keep this wheel rotating in your projects, the project health remains intact throughout, and wherever there is an fallacy, the running quality wheel lets you know instantly what is lacking and what needs to be done to bring the project back on track.
Everyone involved in the project is responsible to keep this wheel rotating. It is not only Quality Department that has to be held responsible for any Quality leakage in the product. Quality leakage means any point in the complete project lifecycle where quality is compromised or bypassed. A Quality leak may not raise an alarm immediately. It has to be sensed. Sometimes it acts as a Volcano in the making. The earlier it is sensed, the better it is for the project health. To sense quality leak appropriately, you need to have ‘quality sense’.
Thus we can conclude that if the quality wheel keeps rotating in a project during all its stages, the way formulated leads only in one direction and that is the direction to Success.
It is said that if you are going to a new place and you have done your research about that place before starting your journey, you are definitely going to call off lot of troubles. It does not mean that when you actually start your journey and reach that place or till the time you stay there and come back, you will have no troubles. But definitely you already have perceived the risks and are mentally prepared for their countermeasures. Now how better you perceive the risks depends on how well you understand.
The same thing happens in any Software Project too. The more a project manager does his homework and goes into depth of the details prior the beginning of the project, the happier he will be during the project lifecycle journey. Now here a project manager needs to be a little more systematic. He needs to build strong pillars on which his strong walled project is to be built.
The two strong pillars in this regard will be:
QA or quality Assurance: QA is different to QC, we all know this. Quality assurance is more to do with the quality of the project and adherence to predefined set of norms and practices. Usually we term them as standards, procedures, standard procedure, norms, standard practices, or methodologies. It is at organization or project type level. it is broader in sense.
QC or Quality Control: QC is project specific. The main purpose of QC lies in assuring a ‘good’ product going to customer that makes him ‘happy’ to release the payment ‘in time’ and generate more orders from customer in future. QC is more concerned with all types of testing to be done on the product to meet customer’s current and future requirements.
QA and QC both are very important part of project management.
Development Team has finished with a new version of the software product. The product is ready for testing before releasing it to production team provide QC approves it after testing that no severe bugs are there. Test environment is prepared for QC team to perform the complete functional, regression, performance and load testing. Extensive testing is done by QC team and a complete list of bugs is submitted to Development Team. Since the bugs include certain serious bugs, the product can’t be released or be declared ‘ready to launch’ before fixing and verifying these bugs.
The development Team work on these bugs reported and since the list of bugs is quite long, once a substantial number of bugs have been fixed, it is decided to send the product to QC team for interim verification and testing. After the product is handed over to QC for the said purpose, the QC team demands the list of bugs with their current status from development team. Development team updates the list and sends it back to QC.
Here is the CATCH.
The development team released the product first. Then they started updating the bugs list whereas meanwhile other members of the team fixed some more bugs. THE PRODUCT GIVEN TO QC FOR TESTING IS DIFFERENT FROM THE PRODUCT DEVELOPMENT TEAM IS HAVING. The status of the bugs match with the product development has. QC team in this case can’t actually verify the bugs fixed by development team.
These interim testing are infact waste more time as stipulated if not handled very cautiously.
Any software development and implementation project comprises of risks. The visible risks are easy to handle or manage. Invisible risks are more vulnerable. Invisible risks are like volcanoes that can erupt without any warning and can cause more harm. More harm because we never know how severe will be the intensity of the eruption.
The same happens in software also. Risks can come from any corner during the complete project lifecycle. Developers, testers, documents, process, methodology, customer, requirement – anyone or anything can generate a risk. During a project the project manager’s critical role is to be ready for a risk and manage it without hampering the project development. Like the variance in planning of tasks and their actual occurrence decides the timely or delayed completion of project. Higher the variance, more chances are for a project getting delayed.
A similar variance is the difference between perceived risks and actual risks. All perceived risks may not happen. All actual risks may not be perceived well in advance.
You are in software development and in today’s world you can’t escape from most of your customer demanding either replacing their legacy software in use with new web application or the development of a new web application. Every customer wants to keep maximum leverage for its employees in terms of mobility, flexibility, easy usability etc. and that is why most applications in demand are web based.
Various stakeholders of the project get directly or indirectly activities assigned to them so as to make the project run and finish. The major role in web application development is that of development team. They should be very clear about the customer requirements – what browsers they want to use, what browsers they desire, what version of the browsers, future expectations etc. If these web based requirements are not crystal clear, it is going to create troubles not only for the developers and testers but for customer also. You can’t just dream and design, you have to have specific requirements in hand to develop an application.
Similarly Testers role is also quite prominent in validating customer’s browsers related requirements and ascertaining that all the browsers mentioned by customer (essential and desired) have to be checked for running the application completely.
This small issue can create a major backlog at a later stage. So it has to be handled right in the beginning before the start of development.
Once you are in a situation where you understand your business processes (What-is), you need to know to manage them. Managing is not let-going, it is more than that. Managing is a regular process of understanding and improving. So let us discuss the major key factors that drive Business Process Management to make you understand if you are managing your business processes well or not:
1. Improved Process Cost: In past six months have you done any improvement in your existing process to make it more cost effective? If not, record it that there has been no improvement in this specific process past six months. It also proves that your Business Process Management is not effective.
2. Decrease in CoQ or Cost of Quality: If you are incurring the same cost on your process quality and there has been no reduction, it needs to be re-looked into the way it is being examined.
3. Improved PTT: Process throughput time should improve (decrease) gradually with the enhancement in the process.
4. Training Time: Don’t bother too much if your training time or training cost is going higher if you are meeting all other factors mentioned here 25% plus. If not, it would be a worry point.
5. Reduction in Internal Complaints: If process drivers and users have realized a substantial reduction in complaints pertaining to the process, you are definitely moving in right direction.
6. Reduction in Customer Complaints: The worst situation could be that customer has stopped complaining you about your product or system because you have stopped handling it. Jokes apart, if customer complaints have reduced and complaint resolution time has improved (reduced), it is a healthy sign.
7. Your ‘Surety’ about ‘Tomorrow’: If you are more confident ‘today’ about your ‘tomorrow’ than you were ‘yesterday’, you are steering your business process management well.
But mind it; this is an ongoing process itself. A small improvement and a big rest will make you worse than you were. You need to have continuous improvement.
Are you ready for Business Process Management? As the title suggests, it is not an easy task or rather is an uphill task. You will find many roadblocks on the way to stop, deviate, misguide or befool you so that you are never able to reach your destination and wherever you reach – you will be convinced – as your destination. These roadblocks will be in the form of your peers, management, superiors, juniors and everyone else. Very few process owners will seriously cooperate you in this journey and the less you are aware about their business process, the more troublesome will be the journey for you.
When you start Business Process Management in any organization, your first task is to understand ‘where-is’ situation for all your processes. You will have to meet all business process owners – again and again to get more into it. The key factors of “Where-is” would be:
Process Cost: It is very important to estimate each process’ cost once you start this exercise. If you don’t know the cost of a process, you can’t manage that process, and you can’t understand what is lacking in that process and you can’t chalk out an improvement plan.
Cost of Quality: How much are you spending for the quality portion of a process? Is it visible and measurable? Or you might have to dig out further to arrive at this part. The cost incurred on finding/fixing bugs or errors is one of the major parameter.
Process Throughput Time: If you don’t measure it, start measuring it. If you don’t understand how to measure it, or what is it? There are ample ways to understand it.
Training Time: You might be incurring cost on trainings, have a record of all the trainings incurred ready.
Internal Complaints: Your process might comprise of say Hardware Support to internal customers within the organization. Analyze the data to assign various parameters that will help us in concluding certain facts at a later stage.
Customer Complaints (External): If the same process or system relates to external customers also, collect the data pertaining to customer complaints and resolution time.
Your ‘Surety’ about ‘Tomorrow’: If your processes are strong, your confidence will speak about it, if not, you will not be very much sure what is going to happen tomorrow. Check it – how confident are you?
And mind you, this is just the beginning. A lot more is yet to come…
Requirements Change management if managed haphazardly may become a disaster for both customer and the product, so it has to be managed very wisely and tactically. And the role of a project manager in this is very crucial.
In such a case the role of Project Manager can be sequentially summarized as below:
10. He has to understand well the total requirements of the customer
9. He has to map these requirements in the software already catering to other customers
8. This way he will be able to find out and check what is the impact of these requirements on the software?
7. He also has to check out if some practices already built in the software are better than what the customer is asking for?
6. He has to revert back to customer to discuss (and rather educate him) the benefits of adopting the practices already built in over the changes asked for
5. Customer might agree to some, and might still ask for the changes at certain places
4. Now the project manager has to sit with his product manager to convince him to incorporate these changes
3. Understand the impact of these changes and get back to customer if there is any adverse effect
2. Finally get those changes incorporated in the software
1. But don’t assume that the story is over…
Let us talk about existing software required to be implemented at a new geographical location. Definitely because of a different location there will be certain new requirements plus some changes here and there in the existing built to meet customer specifications. This need to be handled very minutely and tactfully in such a manner that on one hand it meets all customer requirements and on the other hand puts least burden on the team and software in terms of catering to those specifications or changes asked by the customer.
How it needs to be managed, monitored and done is an art that requires certain level of high skills in the project manager who has to act as a solid bridge between the customer and the product manager. If we consider Project Manager, Customer and Product Manager as three different islands – it is the Project Manager who has to synchronize and gel well together all the three different islands in the journey of building or moulding the software to meet all the requirements of the customer. The project manager in this case is the center point with Customer and Product Manager on his two sides.
New requirements or change in existing requirements is an inevitable process in any software project. As a project manager you encounter it during every phase of a project. Some requirements emerge internally by your own team and some come from the customer.
Internal requirements result from clarifications in customer’s existing requirements and enhancement of project team in business knowledge. Your team evolves better ways to manage customer’s existing and forthcoming requirements in a better way thereby seeking a change in existing code. It could be a major change asking for involvement of considerable number of developers for certain mandays. Or it could be a minor change involving just a couple of hours.
On the other hand, another scenario of change in requirements could be when your team of developers is working on building the customer requirements in the new or existing software and you receive a change note from customer requiring a major or minor change in the software.
What if you have chosen to develop a product for which you don’t have a customer right now? If you perceive that by the time you complete development phase and the product will be ready to launch if will not be obsolete as per technology or concept, go ahead but take care of following cautions to be a winner in the game:
10. Technology: Ensure that you are starting with the latest technology, as even the latest technology will be a little older by the time you complete the product.
9. Concept: Ensure that you do not start building a product that has several variants already in the market. Beat the drum to give the world a new beat.
8. Keep the air in your bag: Let the concept not leak out until you are ready to launch the product. Launch it with a bang. Advertise, blog, press conference, and whatever you feel appropriate for the launch. But ensure that your team confide in you in this exercise till you are ready to shout.
7. Convince, build trust: Convince yourself that you have chosen a right path even if it is risky. Demonstrate your management about your idea and the way you want to design/ launch it. Build trust among your team in giving a real shape to your dream.
6. Risk Management: It is very essential to jot down the risks involved, and ways to mitigate them depending on their severity.
5. Incentive: Let your team know what incentive they are going to get once the product and project is successful.
4. Project Sponsor: Mostly in such type of projects your management will sponsor the project, so all risk lies inside the house. Your stake is quite high in such projects. Equally important is the success of such projects.
3. Project Methodology: Adopt the right methodology and adhere to its requirements.
2. Project Plan: Ensure that such projects cannot tolerate much deviation in terms of time or money. Since in such projects all risk is yours, don’t let it increase at any cost.
1. Definition of successful project: Building a beautiful product in this case is of not any use if there is no buyer at the time of launch. Your total investment in the project can return only if you are able to find out a buyer.
If we have to compromise with the quality of project at various stages there are many ways to do that. Most stupid way will be to compromise with the quality of the software which in any case is going to create lot of hue and cry in the organization either prior to it goes to customer during internal testing, or when it goes to customer for implementation. The undercover holes covered howsoever smartly will create seepage sooner or later.
Most common mistake that is made during the complete lifecycle of a project is not formally giving documentation (required at various stages) in project plan by assuming that documentation is not that prime. It is presumed that either the documentation will be done at the end or it is taken too casually and told to be done by everyone without assigning a proper ownership.
Both – Quality of Software and Quality of Documentation play a lead role in project management. Compromising with any of the two leads to increased cost, loss of customer satisfaction, delay in implementation or revenue loss.
One of the project managers of an ERP implementation company got himself into a tight corner. He found himself in a tough situation where an already ‘mutually sealed’ project scope asked for one or two new requirements (or changes in the existing functionality) from the client everyday while implementation. The broadly agreed upon requirements within the earmarked project modules came out with some changes here and there, some new add-ons. Customer is not ready to accept ‘no’ to any of the requirement since they have a mindset that they have ordered for a big project and are investing a large amount of money in it. The customer keeps on pushing for all their ‘now invented’ requirements to be mapped in the existing ‘project scope’. This seems to be a never ending story. The project manager is tightlipped to start a new module since the running one is not ‘done’ so far. And he also can’t say NO and mark any requirement as ‘out of scope’ since he does not want to annoy the customer and wants the project to be a success.
What should be done in this sort of scenario?
The project manager privately updated me about the situation and asked for my help to get him out of this situation. I told him if he carries out in the way he is – he will never be able to finish his project.
I advised him to have an emergency meeting with his client and share his pain with them. Make them clear that you are not saying No to their requirements but there is a need of a boundary line drawn with mutual understanding. Cater to so far documented requirements as phase I. Finish it off. Get it signed off. Whatever new requirement or changes come from the customer – document it, analyze it. Any requirement that is asking for more than 4 hours of efforts, put it in phase II of the project. As soon as you finish off the phase I, finish off Phase II, sign off. And so on.
18. All meetings related to the project must be fruitful for its continuous progress and timely actions.
17. Duration of the meetings should be optimum to cover all major concerns and immediate actions required.
16. Meetings should bring all participants close to break the barrier between them.
15. Don’t hesitate to have one-to-one talk where important.
14. Have lively discussions.
13. Have concrete progress.
12. Explain after taking time the points that require proper knowledge to be brought to all the members.
11. Propose your views and action points.
10. Stress on your viewpoints where you feel the importance need to be expressed to the members.
9. Commit full co-operation
8. Understand every member’s viewpoints
7. Participate in complete
6. Focus on each member and their suggestions
5. Strip away the mental wall separating the members
4. Strengthen the mutual cooperative relationship towards the common goal
3. Give sincere response to the issues
2. Restore trust
1. Make it overall a meaningful meeting
As stated in my previous post, a Business Analyst is a quite powerful role that establishes the base of a project. It is the first visible pillar for a project which involves communication, leadership, writing, technical and functional skills together. A business analyst has to have a great depth of knowledge of the business on one hand, a sharp understanding power, strong writing skills, great communicator, and a good influencer. Let us see what are the ‘must-have’ skills without which a business analyst can not survive? And why are those skills so important to be a business analyst? Without any relevance to the order in which they are mentioned (as all are equally important) these skills are:
5. Business Knowledge: A good amount of experience/ exposure/ knowledge of customer business are very important for a business analyst
4. Listen and Understand: A business analyst has to be a good listener and with a sharp understanding power without which all the discussions with customer will be fruitless.
3. Technical Knowledge: There will be quite a few technical discussions at customer site. The BA has to be quite conversant with the technologies and methodologies present at his organization.
2. Communication: A business analyst has to be a strong communicator. During the customer meetings, if he does not communicate well about his organization’s capabilities to build up the trust and confidence, probably customer may not gel well with his ideas.
1. Writing skills: Very important skill required for documentation and for conveying the right messages across the board.
Business Analyst is a quite powerful role forming the base of a project. It is the first visible pillar for a project which involves communication, leadership, writing, technical and functional skills together. A business analyst has to have a great depth of knowledge of the business on one hand, a sharp understanding power, strong writing skills, great communicator, and a good influencer.
For a software company having various new development projects a business analyst has to understand the existing business processes, methodology, rules etc. of the customer, document it (which itself is a specialized task) and hand it over to development team to embed the customer requirements into the software to be built.
For a software (or IT) sales company a business analyst has to sit behind the sales/ business development teams – understanding their current process of acquiring new business or sustaining current business and bring out a better approach, methodology, process to enhance business in terms of new business and holding current business.
For a manufacturing company a business analyst has to understand the process, re-engineer it to enhance the production, product yield, thereby increasing customer satisfaction and reducing defects or rejections.
A business analyst ‘s various caps thus include – business process analyst, business strategy analyst, business methodology analyst thereby becoming a backbone to business process managers, sales teams, management, development teams , product teams, quality teams etc.
Be it large organization or small performance management is the key concern for any size of organization. Every organization has a goal to achieve their goals bound to be for a stipulated period, gain profits, enhance, and set higher targets. Growth is important for every organization.
The same applies to project also. Irrespective of its size, client, and customer requirements, each project has to be a success if in turn the organization has to achieve success. For that purpose performance management becomes a great tool. There has to be a performance management methodology clearly defined for that organization so as to assess the performance of each individual of the organization involved in the project directly or indirectly. A performance methodology should very clearly define the process of measuring performance. The process should be transparent, acceptable, measurable, and simple. Any cumbersome process is good for document purposes for difficult to adhere to.
The methodology should be universal for all the employees at all levels as it is very clearly evident that a project is like a relay race. The overall performance is highly affected by the weaker links. Even if there is a weak member in a very strong team, it is high risk for the overall project. As each member has to perform their individual task, in today’s scenario of recession, lean and low costs, each member’s performance keeps overall organization’s performance at stake.
You have a programmer who is writing codes for years that comes to you for testing. The programmer might be coding for a number of projects simultaneously or sequentially. Similar would be the case with you. You would be testing a number of projects simultaneously or one after the other. By now your subconscious mind knows the pattern of bug writing while coding by each of the programmer. But since these patterns are not recorded or analyzed you can not predict the probable number of bugs in the next set of a code from a programmer.
You can do that with the help of your bug reports so far you have submitted back to the programmer or development team. Just pull out your last ten bugs reports produced from the code written by the same programmer. Check for a specific number of lines of code how many bugs were found. Based on this one report’s bugs were from how many lines of code. Do the same exercise for all other reports. Take an average for all ten reports.
Based on the number of lines of code, this way you can easily predict the number of bugs that will be there in the fresh set of code submitted by the programmer for testing.
Involve all stakeholders throughout: This does not mean that all people involved in the project have to keep them available full-time during the project but it means that the knowledge about the project, project progress, shortcomings, bottlenecks etc. should be continuously shared across the board universally. All members should have the same set of information available with them at any moment of time. Involvement will definitely vary from person to person and phase to phase.
Collaborative, Participative, and Interactive: The information flow should not be one way. It should comprise of praises, shouts, appreciations, arguments etc. Let each brain contribute in making it a success.
Be Demonstrative: Lead the project. Demonstrate by actions what you want from other members. Use fewer words and more actions, especially in case of crisis or a problem.
Identify the lazy goat: If there is any lazy goat that is bound to spoil the show, indentify it at an earlier stage so as to avoid higher risk at a later stage.
Keep a set of your customer shoes with you: Borrow a set of your customer shoes, and keep it with you during the project. Put off your shoes frequently, wear your customer shoes and then have a look at the project pace, progress and status. It will definitely give you a different perspective view.
1. A project manager is always right
2. Quality is seriously maintained by developers while developing software
3. A separate set of people (quality control or testers) is not required to take care of the software produced or developed
4. Testers don’t do any substantial value addition in product development
5. Testers lack business knowledge
6. Testers lack technical knowledge
7. Testers lack essential skills to test software
8. Testers don’t understand product requirements well
9. All developments and implementations are prone to extensions because of change in customer requirements or other customer constraints
10. All delays in development and implementation are due to customer
11. Testers have a very little role to play in product development
12. Testers need less than 1% of project time for testing and reporting bugs
13. Testers should be able to test the product well even if substantial knowledge related to product has not been shared with them
14. Testers are the biggest misleading agents
15. Developers and programmers are not testers and thus are not supposed to test what they develop
16. Testers are de-motivators to programmers and developers
17. Testers are project delayers
18. Customers blame development and implementation teams just to hide their own shortcomings
19. Customers don’t know to explain their processes and business rules well
20. Customers are always interested in delaying the project
One customer type focuses on current requirement, rightly built, with more flexibility towards the business requirements built-in in the database rather than in the code. They believe that if the software meets their current requirements well, the future requirements will be built in at the need of the hour. The reason for this is that nobody at the moment is sure about the future requirements and when the change will be required. The change could be required one day after the current implementation or after five years. The changes required could be insignificant, small or not so much effort consuming.
Another type of customer is who is more worried about tomorrow without focusing more on the current built. The future requirements which are not too clear at the moment, are guessed by the customer and forced to be built in the product without really understanding if these requirements will ever be used, or if the requirements built are correct as per future needs as nobody can define them correctly at the moment.
Probably if requirement handling is managed more at database level than in the code, it gives more flexibility to the product.
I have seen different type of drivers on road: some drive very fast violating all rules and regulations to reach the destination. Can this attitude work in software development and delivery? I don’t think so, if the project manager is more worried about reaching the implementation stage without bothering about the customer requirements, probably he is calling for a big bunch of troubles.
Another set of drivers are overcautious type. They will take lot of time in building customer requirements and will be uncompromising towards quality of product to such an extent that every deadline will be crossed without meeting it. Can such project managers be liked by customers? Or by the management?
Our next category of drivers is ‘stick to the route’ kind. They will never change the route whatsoever is the hurdle is and whatsoever is the impact on the delivery. Can customer accept a project manager who is damn fussy about the requirements?
Some drivers believe in ‘change with the wind’ style. They start for a destination, get a call on the way from the customer to divert to another destination, and the driver agrees happily. Probably this is the quality that customer wants in the project manager these days.
Phase I: There is nothing called ‘off the shelf” product. It is all ‘made-to-fit’ technique. You specify your requirements and that too at a broader level. For micro level leave a red-mark at areas that need to be looked into at the time of building of software. The more red-marks will mean less precise plan and deadlines. The sooner we are able to convert these red-marks (not clearly known ones) to the clearly defied requirements, the better plan we can submit.
Phase II: The requirements will go to a team of developers who will build the outline or prototype of what is required by the customer, will get is approved from them and then will build the business requirements in it. Once they are through with it, the product will go to phase III.
Phase III: The product is handed over to a team of testers. These testers are technically stronger than the team that built the product. So this team will take the total ownership of the product from the team responsible for phase II. This team will find out the bugs, remove the bugs, fine tune the bugs and will hand over the final product to the implementation team.
Phase IV: The implementation team will interact with phase III team for any shortfalls in the product. Any new requirements will go to phase II team. Post implementation support also will be with the same team.
Yu-ai in Japanese means fraternity means people engaged in a particular occupation. It corresponds to “you and I” in English. Any software project does not shape well without the exhaustive contributions of developers and testers engaged in that project. Both communities are singular pieces of a jig-saw puzzle that becomes complete only if they are organized, arranged and sequenced in a proper manner.
Both manage CHANGE. One fulfils the requirement, another checks it and verifies it. Developer’s efforts are once synchronized with the testers efforts result in a good product.
Coalition of both – the developers and testers making a strong bond with a common motive of producing a customer focused product. Their combined efforts are meant to overcome the dissatisfaction of customer and ineffectiveness of the product. If they do not gel together, their efforts fail to address the various issues in the software that burst out at the time of product facing the customer.
The ultimate goal is to overcome differences, respect each other and provide mutual support to each other during the product development
Load modeling is the first phase of the performance testing in which certain specific tasks are performed such as conducting performance requirement gathering workshop: This usually is conducted with the top level management to understand their perception regarding number of users, critical scenarios (may be top 10, top 5) for the application catering to the business. Top management definitely have their own understanding on the critical areas which need more focus while load and performance testing. This might vary from the focus areas defined by the end users or project members.
A load test environment need to be prepared based on the above requirements, which include populating database for appropriate values, setting up proper monitoring instances etc.
This all has to be managed by a team comprising of performance analysts, performance engineers, business users, management representatives etc. The performance engineers are supposed to prepare scripts on application with the help of some toolset.
No. I don’t think so. If you have to identify the bottlenecks in your newly built software application, you are bound to adhere to this approach. Use a progressive bottleneck identification approach for performance testing of the application. The testing approach should be to apply holistic load on the application server. There could be multiple aspects for users working simultaneously on the application in real scenario:
Multiple users would be accessing different workflows of modules of the application.
Multiple users would be accessing the same module of the application
Multiple users running different reports
Multiple users running same report
Multiple users running different batch processing
Multiple users running same batch processing
Multiple users logging in
Multiple users running multiple sessions
Now to find out the peak performance point of the application beyond which the application does not behave normal, an incremental or progressive load approach need to be applied while performance testing. Testing would be carried out to identify the impact of various performance parameters on the application. Application would be tested under a concurrent user load and the transactions response time for critical transactions is reported back with their response time. As the load of users, or sessions is increased on the application, the behavior of the response time is studied to ascertain the optimum users or sessions permissible in the application under a pre-defined set of hardware environment.
A project starts with initiation phase and ends up with project closure report. Then afterwards there is post project feedback (after a considerable amount of time given to the customer to get conversant with the product) and warranty support followed by support contracts over a period of time. Let us begin our story with the project initiation and proposal approval from customer. The real ball-game starts from here.
Now planning starts, teams are formed, milestones are set and teams move into execution phase.
Execution phase comprises of development, customization, configuration, testing, training, reporting, sign-offs, monitoring upto implementation, parallel run, live run etc.
And then comes the project completion phase with Project sign off.
During all the phases of project initiation, planning, execution… are the milestones broken into tasks and tasks further into sub-tasks unto the smallest measurable, achievable ‘hour’ unit. Or it is measured in days, weeks or months where it loses its continuity and rhythm. If everyday something is declared done, finished, achieved that is visible to the project team, stakeholders including customer, it carries out a satisfaction, rhythm, confidence and regular progress.
A software project has to undergo various stages before reaching the final stage of customer sign off. At each stage of the project there are certain set of documents that are maintained by the project team for internal or external purposes.
These documents are prepared by various team members – by business analysts, by coders, by project manager, by testers and by other managers.
The Quality and maturity of documents straightaway tell about the health of the project, the team, the management, the product and the progress. It tells clearly about the intentions behind the documentation – that if it is merely a formality or it is really meant for helping the project progress.
And it is not difficult to ascertain the intentions after going through the documents maintained or being maintained during the project.
A coder or programmer when told that he is not writing his code, by way of presenting him with a list of bugs, he is being told to CHANGE.
A tester when is told by his superior that he lacks business depth for testing the product, he is being told to CHANGE.
A project manger when told that he has to improve his way of managing a project, he is being told that the current methodology does not suffice the purpose. It requires a CHANGE.
A management when consistently fails to deliver its commitments to its employees and customers, means it needs to introspect and CHANGE.
When a person, department, team or country stops delivering, or starts giving indications of failure in one aspect of other, it asks for CHANGE.
Change is powerful.
Change is inevitable.
Change does not challenge your knowledge.
Change does not indicate that you are incapable. Rather when one is being told to change, someone has confidence in his capability to CHANGE and PERFORM better.
Change does not mean that you are wrong, it means that there is a scope of improvement.
Change is often opposed as a natural tendency because it is taken as an indication that you are not performing well and what you are doing is not doing well.
A new project, a new product development – as a QC head how do you estimate your testing effort?
Well, some quick steps for this:
1. Customer requirements: Customer specifications or requirements captured at the time of initial study period would be a quick reference guide for estimating testing effort. One word of caution is that incomplete or non-documented requirements may lead to wrong estimations for both – development and testing.
2. Business Specifications: Like customer requirements, there are some business rules and specifications that are well defined by the customer representatives that need to be built in the software. Building test cases and performing tests will depend on such requirements and hence the estimation.
3. Testing Scope: What is the overall scope of testing? Performance, Load, Functional, or all will help you in estimating your testing effort, team size, and test plan
4. Development Plan: Development will definitely go in phases. Plan testing accordingly and estimate testing effort in that manner to accommodate parallel or phased testing so that all load of testing does not fall at the end of development phase.
5. Test team size: Testers availability will be major criteria in estimating your test effort.
We all know developers and testers both have a tough job all the time. Developers have a key role in developing the software as per customer requirements embedding customer’s business needs into it. Similarly testers have to put all their efforts in ensuring that the software is matching customer specifications flawlessly and is bugs free. In a nutshell both developers and testers have a common goal of ensuring a superior product delivery at customer end. If that is so why there is a never-ending tussle between testers and developers. Why developers feel testers are unnecessarily trying to poke their nose into their affair. Why testers feel that out all the bugs found out by them, most of the bugs would have been already handled by developers if they had done their job more seriously.
All this leads to certain questions about testers and developers which they only can reply to:
1. Testers are not supreme and so are developers. If developers can build so many bugs while writing the code, testers are also bound to leave certain loopholes in their testing. This is universal and never ending story.
2. Most of the testers around the world who test software do not understand very well the purpose behind the testing. They keep oscillating between their role as policing, controlling quality and excessive reporting.
3. Both developers and testers carry a single goal of ensuring good quality of software at the end of the day but still keep blaming each other for the shortfalls.
4. If developers are kept for writing code, it is well understood that they are being paid for writing good code and not bad code. Then why bugs at all? If a developer has been hired for coding, is it wrong at organization level to expect a 100% bug free coding from developers. If they are permitted to write code with bugs, why not every other function in the organization is allowed to perform their daily tasks with errors. Can’t we have perfect coders?
5. If testers find out the bugs, instead of being thankful to them, why developers start finding out reasons of cornering them. Developers are hurt when testers find out bugs in their code, and instead of going into a thanks mode for testers they start going into another mode where they themselves start losing their respect. In turn they start finding out weaknesses in tester’s capabilities, testing criteria or bug reporting process.
6. It is a well proven fact that while fixing reported bugs developers are bound to generate new bugs. Does it not make them circling them around the same product?
7. Testers sometimes have an understanding that if they report less bugs will mean a question mark on their job, which forces them to report many a times non-quality bugs thereby increasing gap between developer and tester.
8. Developers, once they know that the product has to undergo testing, write code so foolishly that they generate lot of unexpected bugs.
9. If testers are hired for finding out bugs, is it not their lack of depth of knowledge that leads to bug explosions at the later stages? Are testers involved in coding, or business study or implementation?
10. If developer’s after reading so many books on development write codes with bugs, I don’t think a good tester criteria should be if he has read a book on testing or not.
Project Management is not simple. It requires lot of skills (and learning and experience) to become a good project manager. A good project manager is the one who plans and executes well – all the stages of a project. Finally if project finishes in time with a SMILE ON CUSTOMER FACE, the project can be treated as successful. It is the option of a project manager to label himself as “lousy” project manager or “successful project manager” depending on how he/she manages the show throughout.
To become a “lousy” project manager, following steps can be followed which are quite simple in nature and easily adoptable. Vice versa of these steps can help you in becoming a “successful” project manager, but following that will require some smart efforts and a “fire in the belly”.
The steps are:
10. Don’t own the project: You can easily pass on the buck to anyone engaged in the project. Keep yourself focused on the minutest of the failures and find out the right guy to be blamed for it. With smart reasoning you can easily convince the management and other stakeholders about the failure, its reasons and justification for delays in the project.
9. Assure insufficient customer requirement gathering: To achieve your target, ensure that the customer requirements are not gathered in the proper fashion. Ask irrelevant questions to customer and that to the irrelevant persons for gathering customer requirements. This will surely give you a good reason to convince customer later that it is because of their fault in specifying proper requirements that the product built in not meeting their requirements and you will have another stack of ‘months’ in your kitty to linger on the project.
8. Build a thick wall: Between your management and customer, build a thick wall so that all communication never gets transparent. This will help you in moulding the things in your way as and when required. Similarly build a thick wall between various sub-teams in your project like testers and developers, developers and implementers etc.
7. Build inappropriate teams: For managing your different phases of project, inject inadequate number of persons in your various teams responsible for different phases of the project. Also ensure that that most of the team members are “dumb” enough in knowledge and smartness so that you can easily place the “failure” hat on anyone’s head at any rough moment of the project.
6. Ambiguous documentation: Ensure that the documentation is not at all crisp clear at any phase of the project. Let it be as ambiguous as possible, but in a smarter way, so that nobody is able to figure out the objectionable part.
5. Stay unfocused: Find out some other critical issues not related to project but that can easily make you reasonably justified for not able to devote proper time on the management of project.
4. No Reporting: Let each member enjoy. Make no provision of timely reporting about the progress of project.
3. Dearth of technical knowledge: Don’t develop your technical knowledge required for your project. Let your technical people befooling you in their own way. Ignorance is always bliss.
2. Be Political: Keep on taking your customer on a different track by telling them wrong stories about your management and product in process. Also keep informing your management about customer’s lack of knowledge, involvement, providing of sufficient key users etc. that will become your strong ammunition for shielding you.
1. Repeat everyday: All points above have to be read repeatedly everyday so that you end in becoming a “perfect loser” at the end of the day.
Next month is a marriage in your close relation. You plan to buy an expensive suit length and get it stitched by the best tailor in the city. You buy the best cloth, go to the best tailor, he takes your measurement and gives you a trial date suitable to you. You go on that date, find minor or no change in the stitched suit, tell him the alterations required and get your fully perfect suit after 2 days, a week before the function date.
Your customer decides to go for a business application, decides on you to build it and implement it, gives you an order, you take the measurement (understand business rules and customer requirements), you give them the tentative date for trial (UAT)… but UAT goes down flip flop. You are not able to deliver the product on promised date.
Your tailor delivered the suit, with your complete satisfaction, on the promised date.
Where is the difference? Something went perfectly between measurement and trial date for your suit that your tailor had to deliver to you but not for your product that you had to deliver to your customer.
This is called product sizing and team sizing i.e. project sizing.
Project Failure: First failure will downgrade the level of next project to be given to the project manager. Not only this, but it will also trigger hidden cameras in the organization that start monitoring each and every step of project manager. These hidden cameras could be top management or some selected down the line people working with project manager.
Customer Feedback: Organizations are taking customer feedback very seriously. A remark by a customer regarding a project manager – “I want this project manager to manager all my future projects ordered to you. Infact if you don’t depute this person as project manager, you don’t accept our orders” triggered such a magic that it immediately presented the project manager with an out-of-turn hefty increment. On the other hand a customer CEO sent a confidential email message to the CEO of projects organization stating – “We don’t want the current project manager to be seen in out campus with immediate effect” put a very big question mark on project manager’s capabilities.
Project Team Feedback: Sometimes even if Project fails project manager is able to survive if project team and customer give a positive feedback. Then the reasons are different which are beyond the control of project manager. But if project fails and feedback is also not very positive, all axes will fall on project manager’s neck.
Lack of Technical Depth: A shallow or artificial demonstration of technical depth will not help for long. Although project manager has not to do technical activities on his own but his depth of knowledge will definitely help in driving the project in right direction and meeting targets.
Lack of business knowledge: Sometimes smart team members manage many a weaknesses of their project manager if he has demonstrated his strength in certain areas. But the lack of business knowledge is something that will certainly affect the product built, customer confidence in product and project manager’s knowledge. And if project manager is not clear about the business concepts of the customer, his presence in project, product and customer meetings will not be as effective as it is supposed to be.
Pareto Principle or Pareto Rule is quite fascinating in managing personal and professional life, time management, task management, self motivation etc. Crux is if you focus few vital issues in life you manage major part of your life better. The same applies in profession, organization, department function, and activity too. Only thing is it is to be applied objectively, and smartly.
Let us see how it can work in terms of software testing and quality control:
80% of software quality is maintained by 20% of programmers
80% of bugs in an application are written by 20% of developers
80% of bugs are fixed in 20% of time
20% of a business application accounts for 80% of bugs
The 80-20 Rule or most commonly known as “Pareto Principle” was first formulated by the famous Italian economist Vilfredo Pareto in 1895. The principle was named after him and still holds good almost in all the aspects of life. Vilfredo Pareto found that 80% of Italy’s wealth was with 20% of people. The crux of the principle is that most things in life are not distributed evenly. That may imply that in an organization 80% of decisions are taken by 20% of people. In a manufacturing unit 80% of production is done by 20% of workers. In an class of 100 students 80% of intellect lies with 20% of students. And so on…
In a software project management, based on 80/20 rule, it may be concluded that:
20% of code manages 80% of business in a business application
80% of developers write 20% of code in a team of developers responsible for writing a business application
20% of total productivity of a software developer produces 80% of results
20% of core business requirements, if focused and built well in an application, account for 80% of customer satisfaction
20% of bugs fixed make 80% of software bug-free
And so on…
HP announced end of life for WinRunner in February 2008. Many companies using WinRunner for years started wondering what to do. IT people using WinRunner got panicked and started convincing their organizations to switch over to another product, mostly QTP. Another product means another investment in terms of money, time, and expertise. HP itself started promoting QTP in lieu of WinRunner. From 1st August 2009 HP has stopped supporting WinRunner.
There are techies available having expertise on WinRunner. Some companies are still using it, others supporting these companies in using it. There are quite a few reasons for not immediately throwing this investment in the dustbin, instead keep using it with the following reasons:
5. You have in-house expertise on the tool: If you are using it in your organization for years, it is sure that you have acquired the expertise in-house on this product. Keep using it, it still produces wonderful results.
4. It is a matured product: It is already a well matured and stable product and can run for years without any external assistance required. As long as your products developed are covered by this product, use it.
3. You have experts in the market: Even if you have this product but not in-house expertise, there are plenty in market. Either absorb or outsource for using it.
2. You can move back to OST or manual testing: In worse to worse situation you can switchover to Open Source Tools for testing or can decide to go for manual testing. Open source tools are increasingly available and are performing quite well.
1. You have third party products with expertise in migrating from WinRunner to QTP: At a later stage when it gradually becomes outdated for your products, you can always use your repository of test scripts to migrate to another product, say QTP, instead of writing the scripts manually.
If answer to the first question is No, start moving into the direction of converting it to Yes. Answer to second question also has to be yes. Let us see what is meant by Imagine, Dream and Innovate and how to adopt them at workplace.
Imagination- The ability to create, and paint a mental picture of a new concept or situation can be defined as Imagination. In your case when you have to write business process in your code, it is primarily important for you to imagine business in right sense to embed it in your code.
Dreaming- Coming up with scenarios or goals that with a bit of work can be achieved. Your imagination of scenarios has to be set as a goal of coding it rightly and then ‘doing’ it.
Innovation- Making the impossible become possible by finding another way around a situation or problem. At times you have to find an alternative to optimize your code or choosing a right alternative out of many for a scenario.
A software division in an IT company is considered to be a profit center whereas the Testing division is considered as cost center. A set of developers develop software, get it tested by a set of testers, sell it in the market and earn profits. The credits and benefits on success of the software never come near to testers.
Let us look at a model where testing is offered as a service that costs. Developers claim their software to be ‘fit’ for market after they develop it and do some testing at their end. If testers now test the software and find out meaningful bugs that could have created a ‘sorry’ situation at customer end, then for each of this ‘meaningful’ bug, some amount should be debited from development division’s earnings statement and should get credited to testing division’s earning statement. And the real cost of product is development cost + bug finding cost + bug fixing cost
On LinkedIn an IT projects guy posted a question about plus and minus of outsourcing software testing for his software project. After getting 12 replies from various experts he posted his intention behind this question. The intention was to outsource development and testing to two different vendors (‘slicing’ in his terms) so that they maintain a self-check on their performance and resultant.
My next post immediately after his last post regarding his intention was that if he had clarified this right in the beginning at the time of posting his question, he would have got better replies rather than experts posting their views on pros and cons on outsourcing ‘testing’ or not.
Well my last post said there the same. And I added – “if you are slicing – then make 10, 20 or more slices and distribute it among all your vendors. This will also call for more self-checks”
What is your take on this?
If a person who develops software is software developer, why not the same person developing bugs in the software be called bugs developer. How many developers ethically perform the unit testing after completing development of a unit? It could be – None, a few of them, some of them, most of them or all of them. Some of them might be under the impression that they perform unit testing after completing a unit but the way they do it might not be really helpful in finding bugs. It might be just to satisfy them what they do and call it as unit testing.
If developers really perform unit testing, find out the bugs and fix them, in actual, then when testing is being performed by the testers why they have to perform unit testing again? Why can’t the testers skip it and save lot of money and time of the organization.
I have two sets of developers. Both bunches contain quite considerable number of developers. Let us call it first set of developers and second set of developers. Both sets have their own unique way of functioning and performing.
First set of developers work randomly with no documentation, no fixed plan in place. The requirements come invariably from different directions and as soon as a new requirement comes, the developer changes the priority of his tasks based on the influence of the requirement generator. In this manner after sometime the developer loses his track of what elements he has addressed or fixed and which are pending as there is no systematic way of recording requirements and their completion track.
Second set of developers works absolutely in different manner. They record all the requirements in one place, prioritize them, maintain software version control, mark each requirement as completed only after their satisfaction and testing and reassess their requirements at each day end. This way they never lose control and are absolutely clear on what is balance and estimated time required.
Surprisingly first set of developers spend more time at work place but product less result as compared to second set of developers.
Let us compare this with two similar cars with their fuel tank full. First car is driven randomly on road for whole day without any specific purpose and exhausts its fuel at the end of the day. Second car driver is sensible in planning his route before starting his journey, reaching back early in the evening, finishing more tasks with still some considerable fuel left in his tank.
If you, as a project manager, are fond of thunderstorms, volcano eruptions, etc. it is ok howsoever you drive a project. Otherwise look below at six indicators mentioned below. Even if one of the reason prevails in your project’s lifecycle, manage it, get rid of it, immediately, before a small wound becomes a big septic.
Irregular releases – when plan or commitment varies from actual delivery inconsistently. Some plans finish in time, some with small variation and some with huge variation. It means there may be a volcanic surprise.
Drop in quality or inconsistent quality of product built: Testing is happening, bugs are being reported but if the product coverage is incomplete while testing, or volume of low quality bugs reported is higher as compared to some severe bugs skipped in reporting, there is a mishap going to happen for sure.
Wrong cycle time estimations: if your estimations are going haywired, you plan milestones and achieving them itself if becomes a project, and go beyond control for achievement – there is a mess, in a big way.
Unplanned expenditure like overheads, delays: all these estimates going wrong, will call for extra time for team members and extrapolation of deadlines – a clear indication of a thunderstorm coming on the way.
Predictability: if your team members lose their commitment level, your predictability will become a big question mark for your management and for the customer.
Reliability: your team members’ reliability will invariably start decreasing the moment all this starts happening – and so will be yours.
What is cycle time?
Cycle time in terms of a software project is the time taken from its initiation to handover. Another measure in terms of commercials could be order to execution to recovery… A project lifecycle in other terms
Do you measure it?
You might be banking on your experience without any data in place!
What is your definition of project cycle time?
In your opinion it might be what I said above. It might be nothing for you! Or might be some other definition!!
Is it merely based on gut feeling and past experience?
If you are not recording the development of a project from one stage to another, your cycle time measurement sheet could be a blank sheet.
What is objective measurement?
As we all know each project has several phases. Each phase is divided in milestones. One measure could be the time taken by each milestone from it planning to completion. There would be certain parallel activities. On the other hand some activities could be incremental in nature. Measure accordingly.
Do you segregate your similar type of projects?
You might be running different types of projects. Some could be similar to each other and may fall in one category. Another set of projects could make another category.
Do you know average game is successful for similar types?
An average cycle time worked out for similar set of projects falling in one category will work best for you for your future estimations and meeting those estimations. Else average may not work well and estimations may go haywired.
There is no end to an application. It always asks for a new feature, alter in functionality, addition/ change of business rule etc. With any change in the existing application running in a live environment, the change needs to be tested for all aspects of quality before putting it live. The question comes what should be the scope of testing in this case. Should tester test only for the change part or the complete application?
A change in application small or big is always going to mark an impact on the whole application. Even if not on the whole application, to some extent at various places in the application. Sometimes it could be beyond the knowledge of developer.
Therefore, in my opinion, it is wise to test to whole application even if it going to take more time and efforts to minimize the risk of impact of ‘change’ in the application.
Lot of efforts can be saved in terms of time and money if we reach to a stage of ‘first time right’ in application development. It has been proven largely that no good application can be built and released without extensive testing. Testing is not developers’ ball game – this is also a well proven fact. Reasons are many as far as it is concerned that why developers can’t build a bug-free application, or why can’t they test on their own. We are not going to discuss those reasons here. Focus here would be on what developers should keep in mind while building an application that it requires minimum efforts and time in all testing stages. As we all know the cost of bug fixing goes manifold, depending on how much distance (in terms of time and project stages) it has covered after development for bug identification. The bugs identified at a later stage, say during UAT cost more significantly as compared to the bug identified by the developer himself immediately after writing a code. Few qualities if a developer acquires and keeps in his mind while writing codes would not only benefit him but the organization he is working for and their customer also for whom the product is being built.
1. Commitment to application performance should be kept in mind while writing a code.
2. Clarity of business requirements and rules/ validations that are being translated into the application with real aptitude of business and not a developer. Don’t imagine and build. If there is some lack in clarity – discuss, record and build.
3. Treat yourself as the business process owner and end user – and build the application accordingly as if you have to use it. Don’t think yourself as a bartender, think as if you are preparing the drink for yourself.
4. Collaborate and build – rather than building in isolation- collaborate with other developers working on the application, the end user, and the testers.
5. Optimize your code – don’t just write it. There are n numbers of optimizers almost in all technologies. Use them and build a strong application in terms of functionality and performance. Be quality focused. Don’t do efforts that call for more efforts later.
6. Be focused. Don’t work on various applications development at the same time unless it is too mission critical.
7. Gain customer experience after launch of your application. It will certainly help you in your future builds. Build a customer satisfaction metrics.
8. Don’t take short cuts in fixing bugs – whatever stage they are identified. That way you will build more bugs while fixing identified bugs.
9. Work like a champion. There is a difference between playing a shot and playing an accurate shot.
10. Be loyal to yourself, your organization and your work.
UAT or user acceptance testing comes as the last exercise in software testing lifecycle. It is probably the first phase or beginning of customer preparing to takeover the charge of the product. Actually this is a sort of test drive by a perspective buyer who has studied well about a car, has made up his mind to buy it but wants to satisfy himself by actually sitting in the car and driving it. Even if you have made up your mind to buy a particular model and after sitting in the car or after having the test drive, something does not suit you, the decision can take 180 degrees turn.
The same usually does not happen in UAT because the unlike car the software has been built as per customer specifications. UAT usually includes interfacing (if any), look and feel, ease of usage, functional requirements, integration test etc.
Like a car test drive, here also use runs the complete software to assess if the software is meeting their requirements completely or not. It is the last place where the user gets to determine whether or not the software meets his or her requirements. But one thing is very clear – whatever defects occur during UAT, their fixing cost goes manifold as compared to the same defects occurring during the initial building of software.
Main difference between the earlier testing done at development place by testers is that here the business process and validations built in the software will be checked by a business process owner with real data.
Probably in this busy world, if the end user, by any chance is available during the development and testing phase to do appearance, functional, process and validations testing, it would save a big amount of time and money.
1. Evolve, develop and freeze standards.
2. Keep a breathing space by not developing too rigid standards.
3. Live with open mind. Always be open for change in standards, if it is for improvement, and if it makes sense.
4. Let everyone involved in the projects have the same drink at the cocktail. Distribute the standards to all. Make it available on demand.
5. Understand the usefulness of standards. Don’t follow them just for the sake of it.
6. Let it not be too bulky in terms of documentation work involved in following standards. Standards should not supersede projects and hamper their progress.
7. Let everyone use a common terminology defined in the standards so that everyone speaks same language. Not that everyone using their own project terminology.
8. Standardize but don’t compromise.
9. Goal of standards should be achieving success in projects in a better way.
10. Follow standards.
1. Purpose: The main purpose of the application for which it is built need to be defined.
2. Load Test goals: Identify what you want to achieve by load testing, what are your primary goals to go for load testing. Define all your goals. One example could be – login time for 100 concurrent users should be less than 2 seconds.
3. Architecture: The major concerns in the overall architecture about achieving these goals
4. Simulations: Think of all possible scenarios in which the end user will be working when application is put live. This could include some batch processing, some different client types etc.
5. Past experience: don’t forget your past experience with load testing solutions. The results, the shortcomings, bottlenecks – all could lead to a new learning.
6. Topology: Define all infrastructure components and the network which links them.