Finesse of “Backlog Refinement Meeting”
✥ ✥ ✥
In an agile setting, the enterprise must be poised to quickly activate programs on work that is ripe for reaping value, and should avoid working too far ahead. To respond quickly to a market opportunity, all inputs to the effort must be set up for deployment: needs must be understood, the Product Increment must be well envisioned, and delivery must be timed to meet peak market value. When opportunity knocks one wants to be ready for it so as not to let the opportunity slip by. There is a “last responsible moment” after circumstances and the march of time take away the ability to exercise one’s options.
However, it’s also important to defer work that won’t generate value for a long time. Working prematurely on a long-term deliverable can have a high opportunity cost.
Further complicating this is that some deliverables have hard dependencies on others. Some Product Backlog Item (PBI) may build on an earlier product increment and it’s important to plan ahead, so the right foundations are in place when the opportunity for the big market win presents itself. This may imply that you deliver some PBI earlier than you otherwise would, because though an early release may generate lower value for that item it may set up a Product Increment with much larger ROI.
As time goes on you learn more about the market and about dependencies. However, as time goes on, people make decisions, or the market makes decisions for you. These decisions are likely to constrain the options you may have had open to establish a market presence or to take advantage of a window in the lifetime of the product where there was still flexibility to establish a product direction — flexibility that may disappear with growing feature richness and complexity in the product, a missed opportunity in the market, or losing out to a competitor by waiting too long. Many of these foibles cannot easily be foreseen.
The Scrum Team (particularly the Product Owner and Developers) should meet regularly to ensure the Product Backlog is properly ordered, suitably broken down, and that Product Backlog Items are properly estimated by those who will eventually do the work (Pigs Estimate).
The effort should focus particularly on items near the top of the Product Backlog. This effort should be particularly attentive to dependency relationships between PBIs as well as the dependency of PBIs on external market dates (e.g., holiday sales seasons) as well as dates when resources may become available to make it possible to build the PBI (e.g., taking delivery on raw materials or enabling technology from a supplier). The Scrum Team should annotate PBIs on the backlog with estimates and value attributions.
✥ ✥ ✥
Refinement includes detailed requirements analysis; working towards a Granularity Gradient in the Product Backlog’s structure; bringing estimates up-to-date; re-ordering items to reflect dependencies between them or constraints related to the timing of a PBI release; and, in general, updating the Product Backlog to reflect any new information available to the team.
One good first-order ordering technique from the lean canon is the Design Structure Matrix. ,  Make a matrix each of whose axes are labeled with the PBIs, putting an “X” in the cells where there is a dependency between items:
Then sort the matrix so that it is a lower-left diagonal matrix. That is, you pull the last responsible moments forward — you never defer them. Such was the original, pre-XP sense of the phrase “last responsible moment.”  The items will be properly ordered with respect to their dependencies. Further, pulling these decisions forward leads the team to dive into the associated work, which will flush out emergent requirements and increase the level of insight into the problem being solved. The passage of time alone is no guarantee that any new insights will come along: the item actively must be taken into design for emergent requirements to become evident.
Deal with hard external dates using common sense. If the matrix cannot be sorted then there is a circular dependency and the items should be investigated with more scrutiny.
With the fundamental dependency constraints in hand, use the remaining ordering freedom to optimize market alignment and value opportunities. Knowing the Development Team’s forecasts of relative costs, and the anticipated value the item will generate at the time of delivery, you can adjust the ordering of items to optimize value. Just changing the order of a given set of items can often double the realized value.
Temper the refinement with Granularity Gradient. Focus on refining the PBIs for the upcoming two or three Sprints and work on the rest if time permits and if you have the information to make informed, responsible decisions.
In the early days of Scrum this activity took place during Sprint Planning. As history progressed the work was spread between Sprint Planning and a weekly meeting sometimes called “The Wednesday Afternoon Meeting” or “The Product Backlog Refinement Meeting.” In contemporary practice it is common to spread the work even further, in short, daily meetings. Each team can adjust the frequency and duration of these activities as necessary, but the total amount of face time that the Product Owner requires from the Developers should not in general be more than 10% of the working time. All such activities should be scheduled in advance: it is not work the Product Owner may request on demand.
For more on estimates, see Estimation Points.
 —. “Design structure matrix.” Wikipedia (accessed 2 November 2017).
 Steven D. Eppinger and Tyson R. Browning. Design Structure Matrix Methods and Applications. Cambridge, MA: MIT Press, 2012.
 Glenn Ballard. “Positive versus Negative Variation in Design.” In Proceedings of the 8th Annual Conference on the International Group for Lean Construction (IGLC-8), Brighton, UK, 2000.
Picture from: https://pixabay.com/en/firewood-wood-holzstapel-stack-287015 (under CC0 License).