Electric Cloud offers pre-deployment trial-and-error testability

How much faster would your development team be able to deploy an application if you were able to automate and aggregate all your build processes regardless of the tools or technologies currently being used? What if you were able to deploy an experimental piece of code to a running application, test it and fix it before any of your peers or colleagues saw it? Would your iterations be finalized faster? Would the overall application development lifecycle be improved? Electric Cloud promises to answer these questions in the affirmative with its application lifecycle management (ALM) suite of tools, Electric Commander.

Electric Cloud’s CEO Mike Maciag calls this type of functional testability “preflight builds,” using the analogy of an airplane pilot checking all systems, tires and engines before takeoff. “It’s roughly the same idea,” he said.

Maciag and Anders Wallgren, Electric Cloud’s CTO, visited TechTarget’s Needham office this week and met with reporters, including myself, from SearchSoftwareQuality and SearchSOA sites.

Electric Cloud offers tools to automate and test software build and deploy processes. Electric Commander is EC’s application lifecycle management (ALM) product, which incorporates the pre-flight build architecture directly into the software.

“We’ve come to realize that while software development best practices are running parallel to best practices discovered in manufacturing, software development is decades behind,” Maciag told us.

“Think of Electric Commander in the same manner you would about a manufacturing floor. On it you have tools, check points, testing and deployment,” said Wallgren. Electric Commander contains many of the stock features that customers have come to expect in an ALM offering, such as record/playback, performance monitoring and reporting, but EC has taken a cross-platform approach and allowed for deployment to private and (someday) private clouds.

Software will only increase in density and difficulty to manage as more tools become mainstream, and as developers adopt more individual ways of coding and use more open source tools. This is the landscape where Electric Cloud is trying to carve out its niche. By not dictating how a team must develop software or even which programming language to use, Walgren said, EC offers development flexibility while automating the entire development process.

“Just about any vendor, developer, tester or customer of software that you poll is going to express interest in Agile development,” Maciag said. “We haven’t changed Agile or even modified it, we are just allowing developer freedom so that teams can function as effectively as they can and deploy trouble-free iterations in much the same way.”

With SearchSoftwareQuality.com’s survey-proven interest in Agile development, Electric Cloud might be an appropriate product to explore. Costly repeat tests can be avoided with the tool’s ability to introduce coded additions to a running application, perform every available test and check the combined program for failure before checking the application. Electric Commander offers the benefit of knowing whether additions to an application will have negative effects, a feature that can also cut intervals down. This is a major focus of Agile development and of development in general.

“Where does inventory stock come from? It comes from bottlenecks within the delivery of a product post-manufacturing. These types of holdups are what we are hoping to eliminate,” said Wallgren.

Once amendments are approved and entered into the running application, Electric Commander users can monitor how the additions alter the attributes of the application. “In the preflight build phase in EC, we are able to identify an accelerated order for how aspects of the code can be tested in an accelerated way. We recognize that once the entire application is being compiled for testing that there is no real order that components need to be tested in. From that thinking, we arrange the testing in an order that will facilitate speed while maintaining quality,” said Wallgren.

In the video below, Anders Wallgren, Electric Cloud CTO, explains preflight builds using Electric Commander.

What is the definition of Application Lifecycle Management (ALM)?

The more software project and test managers I talk to, the more definitions of application lifecycle management (ALM) I hear. One calls it a set of integrated tools to manage the software development lifecycle. Another tags it as a methodology that ties together governance, development, and operations processes. Others apply it to any tool that helps in managing software. It appears ALM means different things to different people.

In my traditional manner of doing research, I looked up the definition in Wikipedia. What did I find?

Application lifecycle management (ALM) is the marriage of business management to software engineering… It is a dynamic field with changes in definition coming about quickly as the field matures.

Are they succeeding? I queried some of my friends in the industry and found that ALM tools were met with a fair amount of skepticism. “Moonbeams and fairy dust” was a definition thrown out. It seems that many practitioners think ALM is often marketing hype to sell tools.  I asked SSQ contributor and industry consultant , Matt Heusser to comment.

“It’s been my experience that there are real problems with keeping track of exactly what is in the current release and how to test it, especially as systems mature, or companies grow through acquisition and need to integrate disparate systems.  So the problem is really there, it just seems to me that the solutions offered a bit naive and incomplete.  This leads me to a search, not for ‘one ALM tool to rule them all, one tool to bind them,’ but instead for just enough management of our apps.” 

Matt will be expounding upon this in his upcoming SearchSoftwareQuality tip, Just Enough ALM . Expect it in the next few weeks.

Is ALM a modern solution to the complexity we face in software development? Is ALM pure marketing hype — a vendor buzzword used for any tool that is used in software development?  Let me know! What do you think?

Converting subtasks in JIRA with a Watir script

Whenever I do large data uploads into JIRA, I often find that I have a number of tickets that require a parent/child relationship. That is, some of my tickets get imported as tasks, and I want them to be subtasks. To solve this problem, I wrote a simple Watir script that will subtask out all my tickets for me after upload. There’s nothing that special about it, but it can be quite a time saver.

The only assumption I have in my script is that you’ll have a hyphen in your subtasks and no hyphens in your parent tasks. When I import data like this, I often following the pattern “<parent summary> - <child summary>” for my Summary fields when I import. For example, a parent ticket might be “Write a book on testing” and a couple of subtasks for that ticket might be “Write a book on testing - Chapter 1″ and “Write a book on testing - Chapter 2.”

The first step is to setup a query of only the tickets you want to task and subtask. Often, this is as simple as creating a query in the Issue Navigator where you select the project, Issue Type, and whatever other compoents/values uniquely identify that subset of data. I’ll then sort by Summary, so my parent tickets will always be before my child tickets — again because of my naming convention.

Below, the link to that filter is where you start your browser in the second line of code. In the third line, I simply click on the first result returned. Then the code loops through all the tickets and establishes the parent/child relationships as needed.

Networking your way to QA expertise

Ah! I love that feeling of starting fresh. New year. New job. I started last week as the site editor for SearchSoftwareQuality.com at TechTarget, and I absolutely love the job. How did I get it? I was blogging about QA and networking with a lot of industry leaders, including Matt Heusser, a regular contributor for SearchSoftwareQuality. When there was an opening for the site editor position, Matt recommended me, and a few interviews later, here I am!

I couldn’t have gotten this job without what I think of as “intelligent networking.” Networking isn’t just about meeting people. It’s about learning who’s who in your field. It’s about engaging in discussions. It’s about stepping out of your comfort zone and taking risks, realizing there’s a lot you don’t know, but not being afraid to learn. It’s about sharing what you learn and doing your part in connecting others.

From zero to hero

It was actually in 2002, when I was offered a QA management position, that I discovered the power of networking. I wasn’t a QA expert. I’d been in software development my whole career. And though as a developer, my share of testing was expected and required, I can’t say I really relished that phase of the cycle. Once I became a software development manager at Sun, realizing that developers aren’t always the best testers, I hired a couple of QA people. There wasn’t a formal QA organization in IT and so each development team was kind of stuck with whatever QA and test their team wanted to do, which typically wasn’t much. I suppose the vice president was impressed that I’d cared enough about quality to spend headcount on dedicated QA professionals, because the next thing I knew he asked me to head up a brand new QA organization. I almost said, “No way! I know nothing about QA!” Luckily, I had my brains audio setting on mute, so all my VP saw was a rather blank stare. “You can do this,” my inner-superhero voice told me. “You’re a manager. You dont actually have to do the work.”

So a reorg took place. I left my familiar role in software development to head up this brand new QA organization, even though I knew virtually nothing about QA. How in the heck was I supposed to learn? Here’s where it got fun.

I started networking with the people who were experts. Even though it was back in the old days of ‘02 –before anyone Tweeted or wrote stuff on FaceBook walls — there still were online forums and groups. I formed a new QA community at Sun, inviting all those who were practicing QA, and then put out an all-points bulletin: “New QA Manager needs HELP!”

Well, the help came. Before I knew it, I had pointers to templates, books, articles and tools. Any time I’d ask a question, there were multiple people that would come to my rescue with a variety of answers and resources. I made friends with these gurus and figured out who had expertise in what specialty. It didn’t take long before I’d collected enough information that I was the one who could come up with the answers when a newbie would pop in. I organized a conference, inviting experts to speak about their area of expertise. I learned from my staff as well – those brilliant people that were able to apply all this knowledge we were acquiring.

I soon became part of this elite crowd, referred to as “QA expert” myself! Even though I had no formal training in QA, my name soon became recognized in QA circles as a “go-to person.” I was welcomed for my willingness to learn, collaborate and connect people. And I was rewarded by having the highest- performing team I’d ever managed. This was a big lesson for me. Respect and credibility is acquired not by just what you know, but who you know and how you interact with them. None of us have all the answers! If we think we do, were less willing to go out and learn. It was because I started out knowing so little that I ended up learning so much. And because the world — especially the world of technology — is always changing, this process is never-ending. Learning is a continual process.

Take advantage of the internet

Fast forward to 2010. There are so many ways of finding like-minded people to network with that it can be absolutely overwhelming! Clearly, if you’re reading this, you’ve already found an incredible resource. I’m amazed at the vast wealth of knowledge and information that’s available to us via the TechTarget sites. The IT Knowledge Exchange is a great place to ask questions or show off your technical expertise by answering questions. And if reading, learning, participating and networking isn’t enough, you even get to earn points towards prizes! (When I’m on a Website that offers points, I’m like a gambler in Vegas. I can’t get enough.)

Take advantage of all IT Knowledge Exchange and SearchSoftwareQuality.com offers. Immerse yourself in webinars and whitepapers. Engage! Get to know the people that share your interests. Get to know the experts. When you do, you’ll find out that one day, without even realizing it, you will have become an expert yourself.

Software quality lessons learned from a cruise line

Three years ago, my wife and I went to a  pitch for a timeshare. The project was awful, and we would not recommend the company to others.  I should say that we would actively recommend people not deal with that company.

Yet, at the same time, last year we booked a cruise that cost more than we expected; but we couldn’t wait to rebook for this year!  In fact, we had an emergency and had to cancel our plans for this year. This year’s cruise also cost us more than we initially expected; but , thanks to trip insurance, we’ll get the money back.  Yet we immediately re-booked for next month, even though the short-term rebook meant prices went up again.

It would appear, at least to me, that there is something very different about these business models, and that, if you are in the business of software, it would be better to behave more like a cruise line than a timeshare. So I’d like to explore that theme a little.

The Timeshare Experience

At a local county fair, you run into some friendly people with a booth, talking about a water park up north, about three hours away. They tell you they represent a timeshare company affiliated with the waterpark and can offer you a deal of two free nights at a hotel; perhaps they offer you waterpark passes or $25 gas card on arrival. All you have to do is provide a $25 deposit and offer to sit through a 90-minute sales presentation and tour of the timeshare facilities. Now you’d likely be willing to pay full price for the waterpark, so this is a great deal. You fill out of some paperwork and sign right up.

Two months later, you go up north and have a relaxing vacation. You end up staying three nights at the hotel and paying an extra pittance for the extra night. You enjoy the waterpark, and go to the sales pitch.

The sales pitch is nice. There are coloring books for the kids, video games, hot dogs, soda and popcorn. The sales guy explains that the resort is one of dozens; yes, you’ll get a week — or four — here offered by law as deeded real estate, but you can trade those in for points to vacation anywhere. He’ll take you for a tour, and, after a couple of hours, ask you for just $30,000 to sign up and get started. Why, the company will even offer you terms of credit! You’re not just renting a vacation, you are buying a vacation experience you can hand down to your children. And if $30K is too much, you can always buy less points for less money; how about a fourth as many points for $8K?

This actually happened to us. My wife was ready to sign, but I wanted to think about it. The amazing thing was the company wouldn’t let me think about it; the deal was for today only. Walk away and my next opportunity would be in eighteen months, and we would never get the chance for bonus points ever again.

We said, “Not today.” The salesman’s face hardened. He slowly closed his book, stopped talking to us and walked us down a spartan, blank hallway to the exit to collect our gift card and sign out. Clearly, we were ungrateful jerks.

Fifteen minutes later, we “work up” and realize the dollar amounts that had been thrown around. Two hours later, we did a google search for the company name and “Scam.”

On the other hand, I had an arguably similar experience with Royal Carribean, and I would recommend them to anyone. Let me tell you why.

Planning a cruise

On initial glance, our cruise experience wasn’t all that different.  Why, it looks like you can get a seven-night Caribbean cruise for a mere $729 per person. And the cruise is all inclusive it’ll fund your whole vacation, including all your meals and drinks. Sweet.

Now the price could go up by $10 a person if the price of oil goes up. And don’t forget to tip the staff. The suggested tip is just $5.75 per day for the suite attendant, $3.50 per day for the stateroom attendant, $3.50 per day for the dining room waiter, $2.00 per day for the assistant waiter and 75 cents per day for the head waiter. Why, that’s only $124 all put together.

Of course, we’ll be landing in our different island ports, and you’ll likely want to get off and go on a tour. We have a list of approved tour operators, called “excursions“. Their tours average a few hours, take care of everything and cost around fifty dollars each. We can do the booking for you and make sure you get back to the ship on time; or you can just explore the city. Of course, you’ll need to pay for your own shopping and food in the city.

Now those interior cabins are so tiny; for just $200 more you could have a balcony room.

Now those free drinks are milk, water andlemonade. You’ll pay a little more for juice and a couple dollars for a soda, but you can always buy an all you can drink soda pass for $50. And when we said meals were all-inclusive, we meant the dining room or the buffet; if you want to eat at the 50’s diner, Johnny Rockets, why, there is a $4.95 cover plus extra for malts and milkshakes. Likewise, our specialty restaurants have a fee, and you’ll pay for your alcoholic beverages.

Gambling? Sure. We’ve got slots, 21, and roulette.  Of course you’ll have to pay for that; we can’t give that away can we?  And you’ll be on your own dime to get to the ship - you’ll likely have to purchase airfare, a shuttle, and likely you’ll want a hotel night before or after.  And you’ll want to get a T-shirt or hat to remember the trip, right?

So let me add that up: Our total actual expenses for the trip are around $1,250 per person — make it $1,450 if we went for that balcony room — $1,600 if you include airfare, hotel and shuttle. And if you go to the Island of St. Maarten and shop for Jewelry? Forget it.

That’s right, the trip ended up costing us twice as much as the low number first hinted at.

But we rave about Royal Carribean and dislike…those timeshare guys. What’s that about?

Putting it all together

The cruise line is selling a complete experience, not a piece of property. They have several suggested additional charges, but are completely up-front, and, for the most part, the charges are optional.  Yes, they have things like airfare; but, again, you’ll know about that from the beginning. The optional charges are small, segmented and each individually adds value. Yes, they add up.

Contrast that to the pitch for the timeshare, where you are offered a better-than-average deal to start, and the company creates a sense of obligation, lets you check out the facilities. Then, the sales person drops a massive, immediate threat; “Pay us tens of thousands of dollars, or you’re cut.”  And if you are cut, why, you’re an ungrateful little jerk.

In other words, the cruise line puts you in control and invites you to spend more incrementally, while the timeshare manipulates you into writing a large check.

Now think for just a moment about software business models. Some companies offer a 30-day trial; others 90. Some offer limited software that can not save files — but perhaps can open or read.  Socialtext and Quickbooks both offer “freemium” Web-based that can be free forever,as long as your use does not exceed a certain amount.

When users hit that amount, do they get an invitation to buy a little more, a little at a time, perhaps $5 per month, in a Software As a Service Model? Or do we slap them with a $499 fee to buy the software right now or else you can’t even get to the files you’ve been working on?

Which one of these sounds like a better model to you?

Now, is this a quality issue? You betcha. I suggest we forget all those fancy definitions about quality as the sum of a set of attributes and Ignore ‘quality systems’ like ISO 9000 and TQM. Think about what people actually mean when they say Quality. As Jerry Weinberg put it, quality is “value to some person.” It’s the reason someone might value Royal Caribbean over a timeshare condo in Florida, and part of the reason you might come back, over and over, again and again.

Next time, I’ll talk about other reasons we keep coming back: Excellent service, inclusion, depth and meaningful experience and brand loyalty.

For now, though one word of advice:  Check to see if your sales model is holding people back from referring to your product as a “quality product.”

That might just be a very, very, big deal after all.

Status reports can help provide software testing focus

Recently, I found myself struggling on a program to help plan and manage the testing effort for a couple of different concurrent projects. For both projects, testing is a critical path, and resources are tight. For both projects, there’s a mix of planning and execution activities happening within the next couple of weeks. I was just overwhelmed with everything that needed to be done and balancing that against my need to come up to speed because I was new.

When I start to feel like I’m slipping, I ask for help. I went to one of the project managers and asked for some feedback. After talking for about 30 minutes, wherein several good ideas were offered, she finally said: “I think it would also be good for you to submit a status report each week.”

You know the status report. It’s a project classic; percent complete by task, what you accomplished this week, what you hope to accomplish next week and risks to your part of the project.

While this would normally send of my this-is-busy-work-that-doesn’t-add-value alarm, for some reason I felt like she was right. It would help. A big part of what I had found myself struggling with was keeping track of what needed to be done, when it needed to be done, who was working on it and how I would communicate that in a structured way to the rest of the project managers.

As I started to pull together the report, I noticed that it started to change the way I was thinking about the project. It helped me clarify what needed to be done first. It helped me develop my story of where the testing was, why it might be behind, or why I didn’t think we were ready to start on certain activity. Even though I’m not a big fan of coming up with vacuous percent complete numbers, I recognize that in doing so it forced me to assess our status and develop a more meaningful story for where we were.

When you start to feel overwhelmed with a testing effort your managing, try the following:

• Ask someone you respect on the team if you can talk to them about it. Sometimes just explaining what you’re struggling with can provide you with insights that will help.
• Take that person’s feedback seriously, even if it’s not an approach to the problem you’d normally take.
• Develop a clear method for communicating your status and issues to the rest of the team. It doesn’t have to be a formal status report.
• Until you feel more comfortable with your status, continue to ask for help and continue to refine the story of your testing.

Running, debugging and analyzing load tests using JMeter

In a recent tips series, site expert Mike Kelly explored JMeter, a popular and free open source load testing tool. He uses JMeter frequently because it’s easy to use and supports several programming languages, including HTTP/HTTPS, SOAP, JDBC, LDAP and JMS.

In the first tip, Kelly shows how to run a load test on a proxy server. Using graphics and a step-by-step approach, Kelly describes the ins and outs of setting up the proxy server and related processes.

The second tip walks through running a load test from start to finish. He covers basic and advanced methods and features, such as record-playback and freeze frame.

Kelly’s tip on debugging load tests finishes the series. He suggests shortcuts for finding issues, identifying causes and resolving problems, shedding light on some elusive, yet common, flaws discovered in load tests and how to analyze and interpret results.

JMeter tip series

1. Running your first load test with JMeter
   JMeter is a highly recommended software tool for load testing open source Java applications. JMeter supports various technologies ranging from HTTPS - SOAP. Learn how run your first Jmeter test in this tip.
   

2. Recording and running software load tests with JMeter
   Recording JMeter load tests can simplify the creation of Samplers needed for your test plans. This tip explains how to record JMeter test results and analyze the data generated.
   

3. Tips for debugging your JMeter tests
   Learn how to interpret and debug JMeter HTTP proxy tests in this software testing expert tip. Mike Kelly describes the entire JMeter testing process in this three-part series that starts with running a JMeter test, interpreting results and finally debugging application test criteria.

If you have questions about JMeter that are not answered in this series, you can ask Mike Kelly for information and advice. Write to him at Ask the Expert.

Capturing feedback during sprint reviews

After sitting in a number of different sprint review meetings, with a number of teams, I’ve noticed that not all teams have a formal way of capturing feedback obtained in sprint review. Feedback can be any of the following:

• A defect noticed by anyone in the room during the demonstration of the software developed during the sprint
• Enhancement request ideas by anyone in the room for the product backlog based on what’s being shown during the review
• Feedback from the product owner on something that needs to be changed for the story to be accepted the story isn’t done until this change is made

That feedback should be captured and managed. In theory:

• Defects get tracked and managed: Different teams handle defects requests in different ways. For some, they become backlog items, for others they story isn’t done until they are fixed, and for others they get tracked as part of a release.
• Enhancement requests are captured and managed: Enhancement requests are almost always handled as backlog items. They get reviewed and prioritized along with all the other stories.
• Acceptance feedback is addressed before the sprint “officially” ends. It might be tracked as a new “task” for the story before it’s completed.

In practice, no one tracks any of these. Nothing gets written down, and two hours after sprint review all the feedback is either completely forgotten or is just hearsay.

Don’t let this happen to you.

One way to address this is to have someone in the sprint review meeting enter stories, defects, and tasks as they come up. Have them do this using the same tools you always use note cards, JIRA, whatever… At the end of the sprint review, have that person review everything they captured to ensure they captured it all. Regardless of what processes you use to actually address these issues, you now have the ability to because you’ve captured this information.

Rapid prototyping for faster collaboration

This weekend, I was building houses for developers. As we were framing the first house, I suggested that the location of the door might be improved if it were shifted from where the original plans stated it should be. After a very short debate, both the product owner and the developer agreed, and the updates for the plans began.

As the new designs started to emerge, we began to understand that by moving the door we’d have a 3 to 5 inch strip of wall between the edge of the wall and the door. This created a problem. Since the entire front of the house was originally whiteboard, it no longer made sense financially or functionally to have a three to five inch strip of whiteboard: you’d never write on that strip and you’d end up with a lot of scrap whiteboard if you were to cut for that space.

Let me lay out the scene in a different way and you tell me if this all seems familiar:

The actors:

• A product owner, who has a clear internal vision of what he wants, and strong opinions.
• A developer, who knows what is and is not possible, and the trade-offs involved in what’s being asked for.
• A tester or potential user, whose opinion is occasionally asked for, but whose feedback is more often than not, given knowing it’s going to cause friction.

The requirements:

• On the front of the house we require a door, a light, and a space for whiteboard.
• We have a standard that says anywhere there’s a seam between building materials we need to put up trim to make it look nice.

What’s open for debate:

• The size and placement of the door, the light, and the whiteboard, all with the goal of balancing functionality and aesthetics.

When you think about the problem in these terms, it’s a classic software development problem. And after thirty minutes of discussing what our options were to solve this problem, I decided to leverage a classic software development solution to this problem – I suggested that the product owner and the developer create a mock-up that they could interact with.

Using various building supplies laying around our workspace, we did the following:

• We laid one of the framed out walls on the flat on the floor.
• On top of that, we laid out two sheets of siding and a sheet of whiteboard.
• On top of that, we laid out our door, trim, and a cordless drill which represented our light fixture.

We had a full-size very rough model of what we were talking about. At this point, instead of going back and forth trying to discuss the trade-offs in design by saying things like “If you move the door over three inches you’ll…<insert bad thing here>” the product owner and developer could just move stuff, and the trade-offs were immediately apparent. No detailed explanations where required – everyone was on the same page.

When most people think of wire framing, paper prototypes, or other forms of interface mock-ups, I think they envision expensive tools and long lead times so trendy designers can get it “just right.” And sometimes that’s true. But there’s value having the ability to pull together something physical – something you can interact with – in a quick and dirty way.

Once we made our final decision, the developer went back and updated the formal plans which provided a wire — frame picture of the final house with precise measurements. You still need those final design specs. But you also need tools that allow for faster collaboration. You need the ability for the product owner and the developer to push and pull on the same objects at the same time.

What do you use to do that type of rapid prototyping on your projects?

Testing estimates in Scrum, taking it one story at a time

I really enjoy like doing small projects where I get to put hands on a keyboard and execute tests. Ideally for me when I’m doing this, I’m either doing exploratory testing or performance testing. There’s a challenge that comes with doing this however. For these small projects to work well for both me and my clients, I need to be really good at estimating my time. If someone asks if I can test their application for certain risks, I need to be fairly certain of how much testing I’ll need to do to shed light on those risks and how much it’s going to cost them. Often when I provide these types of estimates, I do the following:

• provide the usual disclaimers about what I can and can’t do as a tester (there’s no such thing as complete coverage, just because I test an area of the application doesn’t mean I’ve found all the issues, this is what I mean when I talk about test coverage, etc..)
• provide a scope summary of what I think they’re asking me to do
• provide a range for how long I think it will take (for example, 5 to 10 hours)
• provide a due date for when I commit to having it done
• provide a rate for what it will cost (either hourly, fixed bid, or other - but I’d never price my work by the defect, sorry uTest)

If I’m working with a team I’ve worked with in the past, all of this fits in a couple of a paragraphs in an email. That’s because they won’t need the disclaimers. If I’m working with someone new, it might be a one or two page statement of work.

Here’s why I think this is important…

I see a lot of testers working on sprint teams struggle with estimating their testing for stories. In my mind, it’s no different then the process I just outlined above.  Assuming you don’t know in advance which stories will be delivered with other stories, then each story is it’s own little project waiting to happen. Based on this risks involved with that story, you outline your testing scope, provide an estimate (low/high) of what it will take to do your first pass of testing, and provide an idea of costs. (Costs might not be dollars, they might be trade offs.)

However, it’s my experience that many times testers can get overwhelmed by the scope of a sprint. They feel they need to “test everything” - even if they know that’s not possible - so they don’t bother to estimate the work. Or they forget to draw a distinction between testing the individual story and the overall release the story is potentially going to production with. This can cause confusion around scope and timing.

In my mind, part of the value of breaking the work up into stories is that you get little chunks of (hopefully) independently testable code. This allows me to more easily esitmate my testing work, because I can focus on just those features and risks involved with the individual story. Testing for the overall release, would be something I might estiamte seperately. With that testing, instead of looking at issues with the individual stories, I’d be looking at issues around integration and other overall quality criteria like performance, consistency, security, etc….