You Do Not Need to Know Exactly What You Want Before Building Software

This article reflects professional analysis and industry research. Individual results vary.

The upfront planning myth has cost firms billions in wasted work. The myth says you must define every requirement before anyone writes code. This was the core idea behind Waterfall. Most teams dropped it, and for good reason. Markets and user needs shift faster than a spec can.

The Myth: Define Every Requirement Before Development Starts

Winston Royce set out Waterfall in a 1970 paper. Royce himself called it a flawed approach. It splits a project into steps that follow each other. First you set the requirements. Then comes design, then implementation, then verification, then maintenance. Each phase must end before the next one starts. The requirements are fixed upfront. Later changes then get costly, or they cannot be made at all.

The logic here sounds good. If you know just what you need before building starts, you avoid rework and cost overruns. The problem is simple. Knowing just what you need, before you have seen anything working, is almost impossible. Users find out what they really need by using things, not by reading spec documents.

The Evidence Against It: CHAOS Data and the Agile Manifesto

The Standish Group CHAOS Report has tracked project outcomes since 1994. Their 1995 report looked at 8,380 projects across 365 American firms. It found 31.1 percent of projects were cancelled before they ended. Another 52.7 percent came in over budget, over time, or with fewer features than the spec asked for. Only 16.2 percent were done on time and on budget. Waterfall was the leading method back then.

The 2020 CHAOS Report showed better outcomes for Agile projects than for Waterfall ones. Agile projects were far more likely to be delivered on time and on budget. The gap between the two was not small.

The Agile Manifesto came out in February 2001. Seventeen software people signed it. They include Kent Beck, who made Extreme Programming. They also include Martin Fowler and Ward Cunningham, who made the wiki. Jeff Sutherland and Ken Schwaber signed it too. Those two made Scrum. Each had reached a similar view from years of work. Locking all requirements upfront, the Waterfall way, made the wrong software again and again. The Manifesto says it plainly: "Responding to change over following a plan." That was not a matter of taste. It was a written reply to decades of failed projects.

The IBM System Journal published research in 1997 on how much requirements move. It showed change at an average rate of 1 to 3 percent per month over the life of a project. A project with a twelve-month build cycle faces 12 to 36 percent change in requirements. That runs from the time the documents are written to the time the system ships. Software built to locked requirements from month one ships with the needs of month one, not those of month twelve, when it actually launches.

What Is Actually True: Iterative Development Produces Better Software

Iterative work accepts a simple truth. You learn what to build by building it and seeing it. A working prototype shows needs that no document could predict. Users who touch real software voice needs they could never put into words in an interview.

In practice, Agile development starts with a clear view of the problem and the functionality that matters most. You then build in short iterations, typically two to four weeks. You gather feedback from real users, and you adjust course as you go. The requirements are still there. They are simply held loosely, so you can incorporate what you learn.

This does not mean you start without any plan. It means you accept that the plan will change. So you build a process that absorbs change cheaply, rather than one that treats every change as a failure. The most expensive bugs come from building the wrong thing correctly, to a fixed specification that later turned out to be wrong.

Google studied how software teams work in Project Aristotle in 2016. The research found the best teams valued safety and learning over rigid process. Teams that could adapt as they learned did better than teams that stuck to fixed plans.

Frequently Asked Questions

Q: Does Agile mean no planning at all?

A: No. Agile development calls for constant planning, just in shorter cycles with more room for new information. The Agile Manifesto values responding to change over following a plan. It does not tell you to avoid plans. The difference is how you treat the plan. It is a guide that updates as you learn, not a contract that locks what you build.

Q: Are there situations where more upfront planning is actually right?

A: Yes. Safety-critical systems often need stricter upfront specs. They need stricter checks too. A change in live use there could cause real harm. Think of aviation software and medical device firmware. Financial settlement systems count as well. Each one works under rules that ask for documented needs and formal checks. Most business software does not face those limits.

Q: How do I give a developer a clear brief without specifying everything upfront?

A: Write user stories, not feature specifications. A user story names the person, their goal, and the reason behind it. For example: "As a sales manager, I need to see which leads have not been contacted in seven days, so I can follow up before they go cold." That brief keeps the build open while the requirement stays clear. It also makes it easy to test whether the feature delivered what was needed.