Apr 1, 2024
Embracing True Agile
This article explores what agile truly is and what modern teams get wrong about agile!
TPM
Agile
Introduction
What Agile is not!
If you've spent any time in tech, you've seen it. A team proudly declares they're "Agile" because they have a Jira board, run daily stand-ups, and work in two-week sprints. Yet, beneath the surface, a waterfall is raging. They spend weeks trying to sequentially finalize every single requirement, then debating every pixel of the design, and mapping out a rigid, year-long roadmap before a single line of production code is written.
Here's something that might surprise you: I've spent the last decade watching brilliant engineers, product managers, and even C-suite executives completely miss the point of agile development. They've reduced it to sprint planning meetings, Jira boards, and daily stand-ups—while still thinking like waterfall traditionalists under the hood. It’s a process that adopts the ceremonies of Agile without embracing its core spirit: responding to change and providing maximum value to the customer, as early as possible.
If you find yourself on an agile project where most of the requirements are written before the development begins, or there is an attempt to create many thorough and detailed specifications and designs before programming, know that waterfall thinking has unfortunately invaded into the project. It is not a healthy agile project, regardless of claims.
Ideas such as “lets write all user stories before starting to program” or “lets finalize all the detailed architecture and API design before starting to program” are examples of unhealthy waterfall thinking.
Why is waterfall thinking so failure prone : The Data Doesn't Lie
Before we dive into my process, let's start with some sobering statistics. The 2020 Standish Group Chaos Study shows that though Agile Projects are 3X more likely to succeed than Waterfall projects, yet Agile projects, on average, have only a 42% success rate. In contrast, Waterfall projects lag significantly behind at 13%.
(Sources: https://www.agilegenesis.com/post/agile-vs-waterfall-comparing-success-rates-in-project-management
Here's what's really concerning: many teams calling themselves "agile" are actually achieving waterfall-level failure rates. Why? Because they're still thinking sequentially. Most organizations continue practicing what I call "agile theater"—maintaining Jira boards and holding standups while operating with waterfall mindsets.
I see this pattern everywhere:
Teams write exhaustive user stories before touching any code
Architects spend months creating detailed technical specifications
Product managers define every feature requirement upfront
Development doesn't start until the "design phase" is complete
Sound familiar? That's waterfall thinking wrapped in agile terminology.
The fundamental issue with waterfall isn't just the sequential phases—it's the assumption that software requirements can be fully understood and specified upfront. Research consistently shows that software projects experience 25-30% change in requirements. For larger projects, that number can skyrocket to 50%.
Recent statistics reveal that 31.1% of software projects are canceled before completion, with 52.7% exceeding budgets by 189%. The root cause? 39% of project failures stem from poor requirements gathering, which suggests teams are still front-loading specifications instead of embracing iterative discovery. The problem intensifies when considering that 45% of features in traditional requirements are never used.
Only about 7% of the features built using waterfall approaches is always used, and 13% is frequently used. Another 16% is rarely used.
Software development isn't manufacturing or real-estate construction. It's knowledge work in a domain of high uncertainty and rapid change. In software, there are market shifts, new user demands, and unforeseen technical challenges. This has been exacerbated with the rise of Generative AI. Clinging to the original plan when reality changes is the fastest way to build a product nobody wants. Change is a constant in software projects.
Choose your approach
I choose my approach (in most cases) depending on two parameters : 1. Probability of Change & 2. Cost of Change. Look at the diagram below to get an idea of how you can tailor your project management approach depending on the project context.
A lot of times, you would find yourself adopting a "hybrid" approach - a combination of predicability while being open to change!
How to do Agile right!
How to do Agile right!
So What is Agile really?
In contrast to the sequential “waterfall lifecycle” - agile involves early programming and testing of a partial system, in repeating cycles(sprints or iterations). It also normally assumes that development starts before all the requirements are defined in detail; continuous feedback is used to clarify and improve the evolving specifications.
We rely on short, quick development steps, feedback and adaptation to clarify the requirements and design. To contrast waterfall values promoted big upfront speculative requirements and design steps before programming. Agile development methods usually apply time-boxed iterative and emergent development, employ adaptive planning, promote incremental delivery and include other values and practices that encourage agility - rapid and flexible response to change.In this approach, development is organized into a series of short, fixed-length (for ex. 2 weeks) mini-projects called sprints(or iterations) : the outcome of each is a tested, integrated and executed *partial system.*Each iteration includes its own requirement analysis, implementation, and testing activities. The result of each iteration is an executable but incomplete system, it is not ready to deliver into production. The system may be eligible for production deployment after many iterations, for ex: 10 or more, depending on the scope of the system. Mind you, the output of an iteration is not an experimental or throw-away prototype, but it is a production grade subset of the final system.
You might have heard about multiple methodologies followed in Agile : Scrum, Kanban, Extreme Programming, DSDM, Feature-Driven Development, Crystal, Lean, ASD. The objective of this article is not to delve into the specifics of any of these methodologies, but to make you understand the foundational mindset of agile that is common across all these processes - short, time-boxed iterations with emergent refinement of plans, requirements and design. They promote practices such as simplicity, embracing change, lightness, communication & feedback, self organising teams and more. This is not about endless backlogs or rigid sprint rituals — it’s about delivering value early and often, reducing risk, and refining the system iteratively. Agile doesn’t remove good practices like risk management and quality management; it makes them more continuous, lightweight, and iterative.
How to manage tensions between agile flexibility and organizational constraints?
How to manage tensions between agile flexibility and organizational constraints?
Agile by nature is iterative and adaptive, but leadership often requires defined commitments to make strategic decisions.
One of the most common challenges I see in program management in organizations with Agile is that there is little room for flexibility. It might make you happy to think of flexibility of agile and being open to change, but another thing to manage the fixed resources of time, budget and human resources. How do you deal with this tension between agile flexibility and lightness with organizational constraints when leadership wants you to given them a fixed timeline or ETA? How do you balance leadership’s need for certainty (timelines, resources) with the uncertainty and adaptability of Agile.
Here is what I do : I don’t tell leadership “Agile doesn’t do deadlines. Instead, I give them a structured roadmap with confidence intervals and show data-driven tracking. Example: “We’ve modelled delivery velocity over the past 6 sprints; based on that, I can commit that core APIs will be production-ready by June, with 70% confidence. If we need higher confidence, we must descope X or add Y resources.
Create rolling roadmaps that show near-term commitments (fixed) and longer term (mid-long term) direction.
Near-term: high-confidence, detailed planning (next 1–2 quarters, broken into sprints).
Mid/long-term: directional, roadmap-level commitments, refined as we learn.
This creates a balance of predictability + adaptability.
Use SAFe or Dual-track agile - maintain a discovery track(flexible) and delivery track(more predictable)
Manage upward communication strategically : report on value delivered, rather than features completed. Ultimately the goal is to create value, not complete projects or features.
Establish regular checkpoints reviews to adjust scope while maintaining timeline/resource constraints.
Prioritize ruthlessly - focus on always finding the 20% most essential, followed by a 40% secondary. Keep the remaining 40% in todo.
Create buffers.
Use CI/CD, automated testing to maintain quality while moving fast.
Educate leadership on the “triple constraints” of scope, time, and cost.
Fixed resources and timeline mean scope is the variable.
I work with stakeholders to define MVPs and critical vs stretch goals, so we can guarantee a minimum outcome within the fixed box, and iteratively add enhancements if capacity allows.
Show historical velocity data to ground timeline discussions in reality .
Proactive Risk & Dependency Management
Maintain a risk register and highlight critical path items early.
Mitigate by parallelizing workstreams, staging delivery, or escalating trade-offs early.
This helps leadership understand constraints before surprises occur.
FAQ
FAQ
01
What kinds of projects have you managed?
02
What industries have you worked in?
03
What technical skills do you bring to the table?
04
What project management methodologies are you most familiar with?
05
Do you have experience managing remote or distributed teams?
06
How do you handle project risks and escalations?
07
What is your leadership style?
08
Are you open to freelance consulting / side projects ?
01
What kinds of projects have you managed?
02
What industries have you worked in?
03
What technical skills do you bring to the table?
04
What project management methodologies are you most familiar with?
05
Do you have experience managing remote or distributed teams?
06
How do you handle project risks and escalations?
07
What is your leadership style?
08
Are you open to freelance consulting / side projects ?
Apr 1, 2024
Embracing True Agile
This article explores what agile truly is and what modern teams get wrong about agile!
TPM
Agile
Introduction
What Agile is not!
If you've spent any time in tech, you've seen it. A team proudly declares they're "Agile" because they have a Jira board, run daily stand-ups, and work in two-week sprints. Yet, beneath the surface, a waterfall is raging. They spend weeks trying to sequentially finalize every single requirement, then debating every pixel of the design, and mapping out a rigid, year-long roadmap before a single line of production code is written.
Here's something that might surprise you: I've spent the last decade watching brilliant engineers, product managers, and even C-suite executives completely miss the point of agile development. They've reduced it to sprint planning meetings, Jira boards, and daily stand-ups—while still thinking like waterfall traditionalists under the hood. It’s a process that adopts the ceremonies of Agile without embracing its core spirit: responding to change and providing maximum value to the customer, as early as possible.
If you find yourself on an agile project where most of the requirements are written before the development begins, or there is an attempt to create many thorough and detailed specifications and designs before programming, know that waterfall thinking has unfortunately invaded into the project. It is not a healthy agile project, regardless of claims.
Ideas such as “lets write all user stories before starting to program” or “lets finalize all the detailed architecture and API design before starting to program” are examples of unhealthy waterfall thinking.
Why is waterfall thinking so failure prone : The Data Doesn't Lie
Before we dive into my process, let's start with some sobering statistics. The 2020 Standish Group Chaos Study shows that though Agile Projects are 3X more likely to succeed than Waterfall projects, yet Agile projects, on average, have only a 42% success rate. In contrast, Waterfall projects lag significantly behind at 13%.
(Sources: https://www.agilegenesis.com/post/agile-vs-waterfall-comparing-success-rates-in-project-management
Here's what's really concerning: many teams calling themselves "agile" are actually achieving waterfall-level failure rates. Why? Because they're still thinking sequentially. Most organizations continue practicing what I call "agile theater"—maintaining Jira boards and holding standups while operating with waterfall mindsets.
I see this pattern everywhere:
Teams write exhaustive user stories before touching any code
Architects spend months creating detailed technical specifications
Product managers define every feature requirement upfront
Development doesn't start until the "design phase" is complete
Sound familiar? That's waterfall thinking wrapped in agile terminology.
The fundamental issue with waterfall isn't just the sequential phases—it's the assumption that software requirements can be fully understood and specified upfront. Research consistently shows that software projects experience 25-30% change in requirements. For larger projects, that number can skyrocket to 50%.
Recent statistics reveal that 31.1% of software projects are canceled before completion, with 52.7% exceeding budgets by 189%. The root cause? 39% of project failures stem from poor requirements gathering, which suggests teams are still front-loading specifications instead of embracing iterative discovery. The problem intensifies when considering that 45% of features in traditional requirements are never used.
Only about 7% of the features built using waterfall approaches is always used, and 13% is frequently used. Another 16% is rarely used.
Software development isn't manufacturing or real-estate construction. It's knowledge work in a domain of high uncertainty and rapid change. In software, there are market shifts, new user demands, and unforeseen technical challenges. This has been exacerbated with the rise of Generative AI. Clinging to the original plan when reality changes is the fastest way to build a product nobody wants. Change is a constant in software projects.
Choose your approach
I choose my approach (in most cases) depending on two parameters : 1. Probability of Change & 2. Cost of Change. Look at the diagram below to get an idea of how you can tailor your project management approach depending on the project context.
A lot of times, you would find yourself adopting a "hybrid" approach - a combination of predicability while being open to change!
How to do Agile right!
So What is Agile really?
In contrast to the sequential “waterfall lifecycle” - agile involves early programming and testing of a partial system, in repeating cycles(sprints or iterations). It also normally assumes that development starts before all the requirements are defined in detail; continuous feedback is used to clarify and improve the evolving specifications.
We rely on short, quick development steps, feedback and adaptation to clarify the requirements and design. To contrast waterfall values promoted big upfront speculative requirements and design steps before programming. Agile development methods usually apply time-boxed iterative and emergent development, employ adaptive planning, promote incremental delivery and include other values and practices that encourage agility - rapid and flexible response to change.In this approach, development is organized into a series of short, fixed-length (for ex. 2 weeks) mini-projects called sprints(or iterations) : the outcome of each is a tested, integrated and executed *partial system.*Each iteration includes its own requirement analysis, implementation, and testing activities. The result of each iteration is an executable but incomplete system, it is not ready to deliver into production. The system may be eligible for production deployment after many iterations, for ex: 10 or more, depending on the scope of the system. Mind you, the output of an iteration is not an experimental or throw-away prototype, but it is a production grade subset of the final system.
You might have heard about multiple methodologies followed in Agile : Scrum, Kanban, Extreme Programming, DSDM, Feature-Driven Development, Crystal, Lean, ASD. The objective of this article is not to delve into the specifics of any of these methodologies, but to make you understand the foundational mindset of agile that is common across all these processes - short, time-boxed iterations with emergent refinement of plans, requirements and design. They promote practices such as simplicity, embracing change, lightness, communication & feedback, self organising teams and more. This is not about endless backlogs or rigid sprint rituals — it’s about delivering value early and often, reducing risk, and refining the system iteratively. Agile doesn’t remove good practices like risk management and quality management; it makes them more continuous, lightweight, and iterative.
How to manage tensions between agile flexibility and organizational constraints?
Agile by nature is iterative and adaptive, but leadership often requires defined commitments to make strategic decisions.
One of the most common challenges I see in program management in organizations with Agile is that there is little room for flexibility. It might make you happy to think of flexibility of agile and being open to change, but another thing to manage the fixed resources of time, budget and human resources. How do you deal with this tension between agile flexibility and lightness with organizational constraints when leadership wants you to given them a fixed timeline or ETA? How do you balance leadership’s need for certainty (timelines, resources) with the uncertainty and adaptability of Agile.
Here is what I do : I don’t tell leadership “Agile doesn’t do deadlines. Instead, I give them a structured roadmap with confidence intervals and show data-driven tracking. Example: “We’ve modelled delivery velocity over the past 6 sprints; based on that, I can commit that core APIs will be production-ready by June, with 70% confidence. If we need higher confidence, we must descope X or add Y resources.
Create rolling roadmaps that show near-term commitments (fixed) and longer term (mid-long term) direction.
Near-term: high-confidence, detailed planning (next 1–2 quarters, broken into sprints).
Mid/long-term: directional, roadmap-level commitments, refined as we learn.
This creates a balance of predictability + adaptability.
Use SAFe or Dual-track agile - maintain a discovery track(flexible) and delivery track(more predictable)
Manage upward communication strategically : report on value delivered, rather than features completed. Ultimately the goal is to create value, not complete projects or features.
Establish regular checkpoints reviews to adjust scope while maintaining timeline/resource constraints.
Prioritize ruthlessly - focus on always finding the 20% most essential, followed by a 40% secondary. Keep the remaining 40% in todo.
Create buffers.
Use CI/CD, automated testing to maintain quality while moving fast.
Educate leadership on the “triple constraints” of scope, time, and cost.
Fixed resources and timeline mean scope is the variable.
I work with stakeholders to define MVPs and critical vs stretch goals, so we can guarantee a minimum outcome within the fixed box, and iteratively add enhancements if capacity allows.
Show historical velocity data to ground timeline discussions in reality .
Proactive Risk & Dependency Management
Maintain a risk register and highlight critical path items early.
Mitigate by parallelizing workstreams, staging delivery, or escalating trade-offs early.
This helps leadership understand constraints before surprises occur.
FAQ
01
What kinds of projects have you managed?
02
What industries have you worked in?
03
What technical skills do you bring to the table?
04
What project management methodologies are you most familiar with?
05
Do you have experience managing remote or distributed teams?
06
How do you handle project risks and escalations?
07
What is your leadership style?
08
Are you open to freelance consulting / side projects ?
Apr 1, 2024
Embracing True Agile
This article explores what agile truly is and what modern teams get wrong about agile!
TPM
Agile
Introduction
What Agile is not!
If you've spent any time in tech, you've seen it. A team proudly declares they're "Agile" because they have a Jira board, run daily stand-ups, and work in two-week sprints. Yet, beneath the surface, a waterfall is raging. They spend weeks trying to sequentially finalize every single requirement, then debating every pixel of the design, and mapping out a rigid, year-long roadmap before a single line of production code is written.
Here's something that might surprise you: I've spent the last decade watching brilliant engineers, product managers, and even C-suite executives completely miss the point of agile development. They've reduced it to sprint planning meetings, Jira boards, and daily stand-ups—while still thinking like waterfall traditionalists under the hood. It’s a process that adopts the ceremonies of Agile without embracing its core spirit: responding to change and providing maximum value to the customer, as early as possible.
If you find yourself on an agile project where most of the requirements are written before the development begins, or there is an attempt to create many thorough and detailed specifications and designs before programming, know that waterfall thinking has unfortunately invaded into the project. It is not a healthy agile project, regardless of claims.
Ideas such as “lets write all user stories before starting to program” or “lets finalize all the detailed architecture and API design before starting to program” are examples of unhealthy waterfall thinking.
Why is waterfall thinking so failure prone : The Data Doesn't Lie
Before we dive into my process, let's start with some sobering statistics. The 2020 Standish Group Chaos Study shows that though Agile Projects are 3X more likely to succeed than Waterfall projects, yet Agile projects, on average, have only a 42% success rate. In contrast, Waterfall projects lag significantly behind at 13%.
(Sources: https://www.agilegenesis.com/post/agile-vs-waterfall-comparing-success-rates-in-project-management
Here's what's really concerning: many teams calling themselves "agile" are actually achieving waterfall-level failure rates. Why? Because they're still thinking sequentially. Most organizations continue practicing what I call "agile theater"—maintaining Jira boards and holding standups while operating with waterfall mindsets.
I see this pattern everywhere:
Teams write exhaustive user stories before touching any code
Architects spend months creating detailed technical specifications
Product managers define every feature requirement upfront
Development doesn't start until the "design phase" is complete
Sound familiar? That's waterfall thinking wrapped in agile terminology.
The fundamental issue with waterfall isn't just the sequential phases—it's the assumption that software requirements can be fully understood and specified upfront. Research consistently shows that software projects experience 25-30% change in requirements. For larger projects, that number can skyrocket to 50%.
Recent statistics reveal that 31.1% of software projects are canceled before completion, with 52.7% exceeding budgets by 189%. The root cause? 39% of project failures stem from poor requirements gathering, which suggests teams are still front-loading specifications instead of embracing iterative discovery. The problem intensifies when considering that 45% of features in traditional requirements are never used.
Only about 7% of the features built using waterfall approaches is always used, and 13% is frequently used. Another 16% is rarely used.
Software development isn't manufacturing or real-estate construction. It's knowledge work in a domain of high uncertainty and rapid change. In software, there are market shifts, new user demands, and unforeseen technical challenges. This has been exacerbated with the rise of Generative AI. Clinging to the original plan when reality changes is the fastest way to build a product nobody wants. Change is a constant in software projects.
Choose your approach
I choose my approach (in most cases) depending on two parameters : 1. Probability of Change & 2. Cost of Change. Look at the diagram below to get an idea of how you can tailor your project management approach depending on the project context.
A lot of times, you would find yourself adopting a "hybrid" approach - a combination of predicability while being open to change!
How to do Agile right!
So What is Agile really?
In contrast to the sequential “waterfall lifecycle” - agile involves early programming and testing of a partial system, in repeating cycles(sprints or iterations). It also normally assumes that development starts before all the requirements are defined in detail; continuous feedback is used to clarify and improve the evolving specifications.
We rely on short, quick development steps, feedback and adaptation to clarify the requirements and design. To contrast waterfall values promoted big upfront speculative requirements and design steps before programming. Agile development methods usually apply time-boxed iterative and emergent development, employ adaptive planning, promote incremental delivery and include other values and practices that encourage agility - rapid and flexible response to change.In this approach, development is organized into a series of short, fixed-length (for ex. 2 weeks) mini-projects called sprints(or iterations) : the outcome of each is a tested, integrated and executed *partial system.*Each iteration includes its own requirement analysis, implementation, and testing activities. The result of each iteration is an executable but incomplete system, it is not ready to deliver into production. The system may be eligible for production deployment after many iterations, for ex: 10 or more, depending on the scope of the system. Mind you, the output of an iteration is not an experimental or throw-away prototype, but it is a production grade subset of the final system.
You might have heard about multiple methodologies followed in Agile : Scrum, Kanban, Extreme Programming, DSDM, Feature-Driven Development, Crystal, Lean, ASD. The objective of this article is not to delve into the specifics of any of these methodologies, but to make you understand the foundational mindset of agile that is common across all these processes - short, time-boxed iterations with emergent refinement of plans, requirements and design. They promote practices such as simplicity, embracing change, lightness, communication & feedback, self organising teams and more. This is not about endless backlogs or rigid sprint rituals — it’s about delivering value early and often, reducing risk, and refining the system iteratively. Agile doesn’t remove good practices like risk management and quality management; it makes them more continuous, lightweight, and iterative.
How to manage tensions between agile flexibility and organizational constraints?
Agile by nature is iterative and adaptive, but leadership often requires defined commitments to make strategic decisions.
One of the most common challenges I see in program management in organizations with Agile is that there is little room for flexibility. It might make you happy to think of flexibility of agile and being open to change, but another thing to manage the fixed resources of time, budget and human resources. How do you deal with this tension between agile flexibility and lightness with organizational constraints when leadership wants you to given them a fixed timeline or ETA? How do you balance leadership’s need for certainty (timelines, resources) with the uncertainty and adaptability of Agile.
Here is what I do : I don’t tell leadership “Agile doesn’t do deadlines. Instead, I give them a structured roadmap with confidence intervals and show data-driven tracking. Example: “We’ve modelled delivery velocity over the past 6 sprints; based on that, I can commit that core APIs will be production-ready by June, with 70% confidence. If we need higher confidence, we must descope X or add Y resources.
Create rolling roadmaps that show near-term commitments (fixed) and longer term (mid-long term) direction.
Near-term: high-confidence, detailed planning (next 1–2 quarters, broken into sprints).
Mid/long-term: directional, roadmap-level commitments, refined as we learn.
This creates a balance of predictability + adaptability.
Use SAFe or Dual-track agile - maintain a discovery track(flexible) and delivery track(more predictable)
Manage upward communication strategically : report on value delivered, rather than features completed. Ultimately the goal is to create value, not complete projects or features.
Establish regular checkpoints reviews to adjust scope while maintaining timeline/resource constraints.
Prioritize ruthlessly - focus on always finding the 20% most essential, followed by a 40% secondary. Keep the remaining 40% in todo.
Create buffers.
Use CI/CD, automated testing to maintain quality while moving fast.
Educate leadership on the “triple constraints” of scope, time, and cost.
Fixed resources and timeline mean scope is the variable.
I work with stakeholders to define MVPs and critical vs stretch goals, so we can guarantee a minimum outcome within the fixed box, and iteratively add enhancements if capacity allows.
Show historical velocity data to ground timeline discussions in reality .
Proactive Risk & Dependency Management
Maintain a risk register and highlight critical path items early.
Mitigate by parallelizing workstreams, staging delivery, or escalating trade-offs early.
This helps leadership understand constraints before surprises occur.
FAQ
What kinds of projects have you managed?
What industries have you worked in?
What technical skills do you bring to the table?
What project management methodologies are you most familiar with?
Do you have experience managing remote or distributed teams?
How do you handle project risks and escalations?
What is your leadership style?
Are you open to freelance consulting / side projects ?