Using product requirement documents and user stories appropriately without creating a mess.

You’ve written two dozen user stories in Trello for a new feature. With each new story, it’s becoming harder to see the forest from the trees. So you start linking, tagging, and organizing your stories. But this doesn’t give you the bird’s eye view. In frustration, you decide to perform a reorganization via a Google doc. Unbeknownst, you’ve created a maintenance nightmare. With two places for information, how do you keep your user stories and the Google doc in sync with changes? What information should be in one versus the other? Worse, when something is out of sync, which source is the truth?

You start asking yourself:

• Have I created a product requirements document (PRD) by accident with my Google doc?

• Should I be writing a PRD or adding more details into my user story?

• What’s the difference between a PRD and a user story?

• Isn’t PRD used in the waterfall model of software development?

• Have I created some abomination of Agile + waterfall (i.e., Agilefall?)

Fear not. You’re not alone wrestling with these questions. To tackle our problems, we have to first understand the similarities and differences between product requirement document and user story. But let’s begin with some history.

Process requirement document (PRD): What the f*ck are they?

In the waterfall model of software development, activities are sequential, hence the name: waterfall. The first stage is often called requirements gathering and documentation. In this stage, individuals work together to create a written document: product requirement document (PRD). This document’s purpose is to define the “how” before a line of code is written. [Aside: Other documents also existed in conjunction such as business requirements document (BRD), functional requirements document (FRD), system requirements document (SRD), or [Fill in the blank] requirements document, depending upon the granularity of the PRD.]

Agile manifesto threw all that documentation out the window with “Working software over comprehensive documentation”, right?

Unfortunately, if that’s the way you’ve been operating, you’re wrong. You see, the Agile manifesto never states: “No product requirement document.” Instead, it’s written as principles to help guide trade-offs. The authors wrote . . .

. . . while there is value in the items on the right (e.g., comprehensive documentation), we value the items on the left more (e.g., working software). [additions added]

What’s comprehensive documentation and is that the PRD?

When the Agile manifesto was written down in 2001, I believe it was an attempt to create principles that highlighted the importance of conversation, dialogue, and working software. Less coding via documentation, where engineers had to write down every possible change, complete documentation reviews, and obtain documentation approvals, all before writing a single line of code. That’s because requirements gathering and documentation meant technical system and software requirements, completed by engineers, not by non-technical individuals. Product managers weren’t the ones writing requirements. From “Managing the development of large software systems” by Royce, a computer scientist and aeronautical engineering.

Occasionally, I am called upon to review the progress of other software design efforts. My first step is to investigate the state of the documentation, If the documentation is in serious default my first recommendation is simple. Replace project management. Stop all activities not related to documentation. Bring the documentation up to acceptable standards. Management of software is simply impossible without a very high degree of documentation. As an example, let me offer the following estimates for comparison. In order to procure a 5 million dollar hardware device, I would expect that a 30 page specification would provide adequate detail to control the procurement. In order to procure 5 million dollars of software I would estimate 1,000 page specification is about right in order to achieve comparable control. [emphasis added]

To fight this “bloat” in documentation, one of the software engineers who signed the Agile manifesto, introduced the use of user story as a way to use simple language to articulate ideas and create space for shared discovery and development of the requirements.  But user story, as a tool, is often misunderstood and misused. People treat user story as equivalent to PRD or requirements. That’s wrong. The better analogy is {user story : conversation} as {PRD : documentation}.

What am I supposed to do with PRD and user story?

1. Discard PRD as a document.

2. Replace it with three other documents/techniques:
   a) Press releases (a.k.a customer letter) to address the vision, explain the why, and frame the outcome,
   
   b) User story maps to organize and present the big picture with user journey or flows to address a specific user/persona, and
   
   c) Enhance your user stories to document technical specifications by incorporating visuals for an user interface, non-functional requirements (e.g., system performance, availability, reliability needs), and constraints

Sample “Enhanced” User Story with Specifications

Principles for replacing PRD

• If you need to write technical specifications (e.g., in a contract), don’t do the above. But then again, you aren’t really working in an Agile environment.

• If you’re thinking of replacing PRDs and don’t know how to execute all 3 steps, don’t attempt to replace the PRD. Instead, keep the PRD and start the press release. It’s the easiest for people to understand and adopt.

• The enhanced user story is trying to mimic detailed software requirement documents by adding 1 - 3 small sections. However, the intention in specifying more is to help the conversation, not replace the conversation. Nothing can replace the conversation because you aren’t a developer.

• If you do have a developer background (e.g., trained as an engineer, can develop), you can go forward with writing detailed specifications. However, you’ll run into other problems the above doesn’t address such as autonomy and creating shared understanding. Topic for another day.

• The goal of the user story map, user journey, and user flow is to tell the entire story visually, and not any specific user story. Think flow diagram to keep it simple. Sync user story to this visual flow.

  

• Keeping the PRD can be the right answer, if you’re consistently answering yes to the following questions.

  • Are you working on solving specific problems given to you by others with higher authority without understanding why this is an important problem relative to other problems?

  • Are you exploring the details for implementing a specific idea or solution without knowing what other ideas or solutions were considered, excluded, and why?

  • Do you lack a plan to test specific assumptions or hypothesis after the feature or product launches?

  • Is failure measured more often by failure to achieve the desired output rather than desired outcome or learning?

  • Do you work frequently alone, via documentation (e.g., slides, word) with infrequent communications with people in different roles (e.g., legal, engineering, design, sales)?

  • Are you negotiating contractual requirements and specifications with an outsourced development team?

  • Are meetings effective (e.g., no clear action item, conflicting opinions with no clear decision-making approach) because issues aren’t surfaced and resolved?

  • When people disagree, are disagreements frequently about personality, working-relationships, and people, rather than problems, ideas, values, and solutions?

• Breaking the PRD is just the beginning. Working collaboratively using more modern techniques to uncover problems, ideate solutions, validate hypotheses, and overcome failure is HARD. It’s not just that there are new tools or techniques to learn, but it’s structure is fighting against many inherent problems, such as:

  • Good people struggle between control and autonomy: A good CEO wants to give smart the people he/she hasi hired the autonomy to solve tough problems. The CEO often doesn’t have the time or the specific knowledge to solve the problem. But there’s an inherent money-to-time problem. If the smart person or team takes too long to solve the problem, what’s the CEO to do other than take more direct control? Even if the direct control sometimes doesn’t help, can the CEO sit there and do nothing when failure is at the door? Fighting against this human condition is critical to enabling collaboration across teams and it’s a constant struggle.

  • People are uncomfortable with large failures: Failure is often held up as something to be proud of and learn from. Think of Facebook’s famous old tagline: “Move fast and break things.” But ask yourself how you personally feel about your failures? Does it feel good when you trip up? Does the process of struggling feel enjoyable? Are you aware of loss aversion and your own frequent actions to avoid pain? While I believe people can be trained to become more comfortable with small failures and increase their appetite the size of a failure, we can never remove our aversion to failures. Furthermore, larger failures are a different beast. No one will likely say, the lessons learned from failures that resulted in the loss of life is a worthwhile learning experience as long as we don’t repeat this mistake again. Fighting against this human condition to avoid failure is a constant struggle and it goes against the try, fail, and learn mantra.

  • Everything is time constrained: Even the riches company or person is still constrained by time. All other constraints, e.g., money, people, talent, legal, etc., can be translated to time. Not enough money? What you mean is not enough money right now, but theoretically, you could get more somewhere down the road with time. Not legally allowable? Well, you could spend money and time to change the laws. Lack enough collaborators and you could hire, which takes time and money. This time constraint and the stresses caused to complete something within a certain timeline causes people to use processes and tools, comprehensive documentation, contract negotiation, and project plans rather than hold conversations and have interactions. Think how you act when you’re short on time and busy with 5 different activities. Are you having great conversations and interactions? Fighting against this time constraint so we don’t regress back to documentation and negotiations, but maintain open conversations and interactions is hard.

Source:

• Why We Should All Stop Creating Requirements Documents by James Hulgan

• Is the Product Requirements Document Dead? A Debate by Colin Lernell

• How to write a good PRD by Marty Cagan

• Waterfall model via Wiki

• Iterative and incremental development via Wiki

• Managing the development of large software systems by Winston Royce

• User Story Mapping by Jeff Patton

• IEEE Recommended Practice for Software Requirements Specifications