At each of the companies where I have worked a debate has always raged about how to document new products specifications. As VP of Engineering, I am frequently asked to produce a template for Requirements Documents. On the other hand, Agile does away with requirements, in favor of user stories. This, in turn, is in conflict with the business team, who wants to know six months ahead of time what they can promise to customers.
The first step towards reconciling these various perspectives is to understand that Software Specification is a Process not a document: the value of a specification comes mostly from the process of creating it, and less so, from the final artifact. For one, the final specification rarely captures the features that were excluded, nor the business justifications behind any given feature.
The Specification Process comprises 3 different phases with different purposes and different deliverables.
- The first phase is Scoping: this phase typically takes place weeks before the start of the release. The output of the scoping phase is an estimate from the Engineering team that a certain bag of features can be delivered by a given date, with a given set of resources.
- The second phase is the Release Planning, ideally starting(shortly) before the official start of the release, where the engineering lead, with input from the product manager, creates the release plan, breaks out the release into iterations, and defines the major features to be built in each iteration
- The third phase involves the detailed specification of the features/user stories for each iteration.
Scoping
In my world of enterprise software, the customers, and the business team, want to know months in advance what features will be available by when. Both the release date and the features are determined before the start of the project (sometimes weeks before) and must be met. This is not Agile, but it is reality – see my earlier blog “Setting Expectations about Formal Releases with the Business Team”
In order to produce a reliable estimate of what will be delivered when, the Engineering team needs a complete list of features, with a degree of specificity that only needs to be good enough for the Engineering team to appreciate the degree of difficulty of each task.
For example, the spec for a user registration page on a web site could be as simple as:
- User enters Username, Password first time, Password second time.
- The Username must be unique
- The 2 entries for the Password must be identical
… but it could get a lot more complicated
- The password must meet “strength of security” criteria
- As the user types in the password, the strength of security of the password will be computed and displayed graphically
- The registration server must handle up to 2,000 registrations per minute with a response time of 3 seconds or less
- System availability must be 99.99% uptime
The two scenarios are vastly different. However, the Engineering team does not need to know a lot more than the bullets above to engage in a discussion with the business team about the scope of the project. If the application’s software stack has not already been validated for performance or reliability, the second project is going to take weeks, compared to hours for the first one. Even the little visual indicator of password strength can add days to the scope of the project (if AJAX needs to be added to the app, or if the team does not have a graphic designer readily available).
While the spec can be very short and still allow the Engineering team to provide scope estimates, one should not underestimate the time it will take to scope. For example, if system performance is significantly increased, scoping will involve design and probably prototyping.
The scoping estimates are typically done based on experience by comparing the new project to previous ones, estimating the number of functional points, etc.
Release Planning / Iteration Sequencing
Release planning, or iteration sequencing, is an overlooked and underrated activity, and yet it often signifies the difference between perceived success and failure. Agile suggests that the user stories most important to the customers should be developed first. This is indeed the primary guide in sequencing activities within a release. However, other important factors need to be considered. For example:
- Eliminating technical risks for some of the important features
- Confirming ease of use and usability by mocking up or prototyping key components of the user interface so that they can be shown to customers for feedback early in the release cycle, thus leaving time for modifications.
- Integration of new libraries, tools, or partners
- Performance validation
By going through the release planning exercise, the team drills down further in the specifications, gets a more refined appreciation for the scope of the project and thus confirms, or infirms, the original scoping estimate. If necessary, adjustments can be made before the project starts. Early preventive action is always a good thing!
In addition, release planning is important to ensure availability of critical resources whether human, or physical.
Finally, a proper release plan will align the coding effort with the integration and testing strategy. For example, it is simpler to test API calls when you implement both sides of it, or to test a DAO, when you simultaneously code the UI front end for it.
“Intra-Release Specification”: Detailed User Stories
Once a release has started, detailed user stories must be provided to the Engineering team prior to the start of each iteration – so that the iteration can be scoped at the start of the iteration,by the developers, and the features can be implemented during the iteration.
While interactions between Product Management and developers are encouraged during the iteration, having well-thought out user stories ahead of the iteration greatly improves efficiency.
By understanding that specifying product requirements is a process, rather than a document, both business and engineering teams will work effectively, by delivering the proper level of information to each other at the right time. In the next blog, I’ll cover tricks and best practices of this process.