Finesse of “Backlog Refinement Meeting”
✥ ✥ ✥
Agile enterprises must be poised to respond quickly to capitalize on opportunities to create value, and should avoid working too far ahead. To respond quickly to a market opportunity. The team needs to understand the stakeholder needs, and all of the ducks must be lined up to enable work to proceed on the items at hand.The Product Owner must have communicated how Product Increment supports the vision. The Product Owner has scheduled delivery to meet peak market responsiveness, to create the highest value. When opportunity knocks one wants to be ready for it so as not to let the opportunity slip by. There is a “last responsible moment” after circumstances and the march of time take away the ability to exercise one’s options.
However, it’s also important to defer work that won’t generate value for a long time. Working prematurely on a long-term deliverable can have a high opportunity cost if it moves other viable product offerings out of their prime market window.
Further complicating this is that some deliverables have hard dependencies on others. Some Product Backlog Item (PBI) may build on an earlier product increment and it’s important to plan ahead, so the right foundations are in place when the opportunity for the big market win presents itself. This may imply that you deliver some PBI earlier than you otherwise would, because though an early release may generate lower value for that item it may set up a Product Increment with even greater value.
As time goes on the team learns more about the market and about dependencies. However, as time goes on, people inside the company make development and market decisions, or the market itself makes decisions that may upset existing plans. Every new decision is a constraint that could limit future product and market possibilities. For example, one may lose the opportunity to be first in a market by deferring the introduction of some feature in that market. The team may order PBIs to gain the highest value given today’s understanding of the team and the market, but tomorrow may bring a window of opportunity that could create even higher value with a different backlog ordering. It is often difficult for the team to foresee these problems far in advance.
The Scrum Team (particularly the Product Owner and Development Team) should meet frequently to properly order the Product Backlog and to break down the most imminent large PBIs into smaller ones. The Developers should maintain current estimates for the Product Backlog Items that they will eventually implement (Pigs Estimate).
The team should focus particularly on items near the top of the Product Backlog. Pay particular attention to dependencies between PBIs, as well as to dependencies of PBIs on external market dates (e.g., holiday sales seasons) as well as dates when resources may become available to make it possible to build the PBI (e.g., taking delivery on raw materials or enabling technology from a supplier). The Scrum Team should annotate PBIs on the backlog with estimates and value attributions.
✥ ✥ ✥
Refinement includes detailed requirements analysis; working towards a Granularity Gradient in the Product Backlog’s structure; bringing estimates up-to-date; re-ordering items to reflect dependencies between them or constraints related to the timing of a PBI release; and, in general, updating the Product Backlog to reflect any new information available to the team.
One good first-order ordering technique from the lean canon is the Design Structure Matrix. ,  Make a matrix each of whose axes are labeled with the PBIs, putting an “X” in the cells where there is a dependency between items:
Then sort the matrix so that it is a lower-left diagonal matrix. That is, you pull the last responsible moments forward — you never defer them. Such was the original, pre-XP sense of the phrase “last responsible moment.”  If it is a lower-left diagonal matrix, then every item will come after all of those on which it depends. Further, pulling these decisions forward leads the team to dive into the associated work, which will flush out emergent requirements and increase the level of insight into the problem at hand. The passage of time alone is no guarantee that any new insights will come along: the team must actively take an item into design and implementation to make emergent requirements appear.
Deal with hard external dates using common sense. If it is impossible to sort the matrix then there is a circular dependency and the team should review the items with more scrutiny. In the worst case, the team may combine two mutually dependent items into a single, larger item.
With the fundamental dependency constraints in hand, use the remaining ordering freedom to optimize market alignment and value opportunities. Knowing the Development Team’s forecasts of relative costs, and the anticipated value the item will generate at the time of delivery, you can adjust the ordering of items to optimize value. Just changing the order of a given set of items can often double the realized value.
Temper the refinement with Granularity Gradient. Focus on refining the PBIs for the upcoming two or three Sprints and work on the rest if time permits and if you have the information to make informed, responsible decisions.
In the early days of Scrum this activity took place during Sprint Planning. As history progressed teams started to spread the work between Sprint Planning and a weekly meeting sometimes called “The Wednesday Afternoon Meeting” or “The Product Backlog Refinement Meeting.” In contemporary practice it is common to spread the work even further, in short, daily meetings. Each team can adjust the frequency and duration of these activities as necessary, but the total amount of face time that the Product Owner requires from the Developers should not in general be more than 10% of the working time. The Scrum Team should schedule all such activities in advance: it is not work the Product Owner may request on demand.
For more on estimates, see Estimation Points.
 —. “Design structure matrix.” Wikipedia,https://en.wikipedia.org/wiki/Design_structure_matrix, 17 February 2016 (accessed 2 November 2017).
 Steven D. Eppinger and Tyson R. Browning. Design Structure Matrix Methods and Applications. Cambridge, MA: MIT Press, 2012.
 Glenn Ballard. “Positive versus Negative Variation in Design.” In Proceedings of the 8th Annual Conference on the International Group for Lean Construction (IGLC-8), Brighton, UK, 2000.
Picture from: https://pixabay.com/en/firewood-wood-holzstapel-stack-287015 (under CC0 License).