Agile & Waterfall: Fitting Round Pegs into Square Holes

Large enterprises and government agencies increasingly turn to Agile for their software projects.  This is a difficult shift, with many struggling to fit their new round agile pegs into their existing square waterfall holes. Cultural, structural, and procedural impediments make “being agile” a challenge.

Agilists may be critical and call this “fragile-agile” or “agile-fall.”  We should commend these efforts and find ways to bridge this dual-operating model reality.  When working with those trying to bridge the divide, I share the Disciplined Agile mantra:

“Start where you are.  Do the best you can with the situation you face.  And always strive to get better.”

This article explores some of the complexities of “going agile” in a traditional enterprise environment.  My goal is to recognize and highlight some of these challenges, provide context, and offer suggestions.

Adopting the Agile Mindset

Agile is a mindset.  The methodologies, frameworks, and practices build the path to agility.  But following them does not guarantee the required changes to the culture and way of working.

Agile emphasizes building collaborative, empowered, high-performing teams that deliver solutions iteratively and incrementally.  Achieving these objectives sounds easy but requires unwinding long-held beliefs and practices.

Creating an empowered and collaborative environment shifts decision-making and accountability from management to the delivery team.  David Marquet described how he stopped giving orders as a submarine commander in “Turn This Ship Around.”  Instead, he gave intent, enabling the crew to think, engage, and take ownership.

Active collaboration throughout the value delivery process—including business and all technology teams—reduces process friction and improves outcomes.  Eliminating the “us versus them” mentality creates a shared purpose.  Co-locating teams breaks down the barriers to collaboration and engagement.

Incremental and iterative delivery means delivering usable functionality in stages and continuing to evolve the solution based on feedback.  Simply adopting a 2-week development cycle without demonstrating even a tiny working process yields little value.  Similarly, not soliciting or providing meaningful feedback limits the ability to learn, adjust, and ultimately be successful.

Moving from Fixed to Prioritized Scope

The triple constraint dominates project thinking—scope is fixed, and changes will have corresponding impacts on schedule and cost.  The iron triangle undermines our ability to be agile by creating the expectation that:

  • Requirements and scope must be defined up-front;
  • Changing scope is bad; and
  • All requirements must be delivered for the project to be successful.

Agile inverts the triple constraint and nullifies these beliefs.  Project duration can be time-boxed, and scope can be adjusted to fit the available time.  This problem is solved by focusing on what is necessary and can be accomplished.  Vacations match this pattern–there are many things to do, but only so many days.

Prioritizing scope and focusing on delivering small increments of value resolves this challenge.  The product backlog is a single, ordered list of everything that needs to be delivered.  The list should be structured so that each item provides usable functionality like staking building blocks.

When I worked for a competitive telecom company, we had no choice but to follow these agile practices even though we used a modified waterfall development lifecycle.  New product launch dates were fixed.  The functionality to support the launch had to be delivered on time.  Other features were delivered in successive releases based on when they were needed.

We were not building the plane while flying it.  It was a well-timed and choreographed process of building functionality as needed.  It also allowed us to limit the investment in products that failed to gain momentum.

Changing the Roles

The product owner and scrum master play critical agile roles that do not have analogs on traditional projects.  Incorporating these roles into the standard project’s organizational structure can be challenging. Finding people with the skills, experience, and influence is even harder.

The product owner is the voice of the customer on the project team and is responsible for the product’s success—a greater purview than project success.  They collaborate with stakeholders to build the roadmap and prioritize the product backlog.  They work directly with the team to write user stories, define acceptance criteria, approve completed work, facilitate product demonstrations, and synthesize feedback.

The scrum master is the servant leader of the agile team.  They are not project managers or functional managers.  Their role is to help the team be agile and mature their practices.  They demonstrate the new mindset and behaviors, facilitate the ceremonies, and work externally to remove impediments.

Establishing context-sensitive expectations for these roles is invaluable.  Stakeholders, customers, and development teams must understand how everyone works together in this new environment.  Whole team training is one way to create a consistent and shared understanding of the roles and how people fit in.

Regular Customer Engagement

Good stakeholder engagement is the leading cause of project success.  And poor engagement leads to failure.  Agile values collaboration and regular engagement.   Its practices establish multiple touchpoints to enable these principles:

  • Product owners work with stakeholders to develop the roadmap and backlog to confirm prioritization and alignment;
  • Co-locating the product owner with the development team creates the opportunity for a virtuous cycle of input, feedback, and adjustment; and
  • Demonstrations at the end of each sprint are invaluable. They build trust, create engagement, and provide the team with rapid feedback.

Moving from the traditional practice of engaging stakeholders at the beginning and end of a project to a regular, continuous engagement model should be easy.  But it requires breaking many old habits.

Procurement

Procurement for software projects is always challenging.   Fixed-delivery contracts reduce the buyer’s risk but require precise specifications.  Customer-driven changes in scope come at a high price.  Cost-based agreements provide more flexibility but lack delivery assurance.

Fixed delivery is an agile anti-pattern contradicting the “welcoming changing requirements” principle.  Cost-based contracts are more agile-friendly but can shift too much risk to the buyer.  Crafting an effective agile agreement requires creativity.

A Lean principle is developing long-term vendor relationships based on trust.  This contrasts traditional, transactional procurement practices that primarily focus on price.  Managed service provider arrangements can build an agile-friendly environment.

Funding the team and creating well-defined objectives is a path to agile contracting.  The Federal government uses modular contracts that support iterative delivery, provide flexibility, and mitigate risks by isolating issues.  Organizations adopting agile contracting may unintentionally introduce anti-patterns despite their best intentions.

Testing

Traditional software testing practices are inefficient.  There are multiple hand-offs, lags, and wait states.  Consequently, errors are found late, the cost of rework is high, and everyone is frustrated.  As Harold Dodge said, “You cannot inspect quality in.”

Agile adopts a test-first mentality.  Acceptance criteria are documented along with the features and user stories.  Developers write and successfully execute robust unit tests before moving to the next item.  The product owner approves each story as it is completed.  Stakeholders then provide feedback at the sprint demonstration.

Testing sooner builds quality into the product. Changing the mindset and practices is the first step.  Investing in test automation tools that significantly reduce the time and effort required for testing is a game changer.

© 2023, Alan Zucker; Project Management Essentials, LLC

See related articles:

To learn more about our training and consulting services or subscribe to our newsletter, visit our website: https://pmessentials.us/.

Image courtesy of:  Merilyn.com