The difficulties of scaled agile projects

There are many projects which are Agile in name but chaos in reality #HoskWisdom

The Agile framework is like a person with a hammer, all problems look like nails and all projects look like Agile projects #HoskWisdom

 

Agile/Scrum used on large/enterprise projects with multiple Scrum teams, scaled Agile increases the complexity of a project and if not managed, adding more people can slow projects because there is a cost to adding more people to a project and increasing complexity.

Many agile project require an increase in velocity to meet an earlier completion date, the common method is to add more people to the project.

These Brooks quotes, author of the Mythical man month are apt
Adding manpower to a late software project makes it later. Fred Brooks
The bearing of a child takes nine months, no matter how many women are assigned. Fred Brooks

Adding more resources (usually developers) to a project would seem sensible.  If one team have a velocity of 25 story points Sprint (Story points gets internally translated to days by 91 percent of the population #HoskWisdom).  So if there are 4 teams a velocity of 100 story points per sprint.

The reality is teams and individuals don’t start delivering 100 percent instantly because there are other factors to consider.

Increasing people on a project increases complexity of working, collaborating and communicating #HoskWisdom

These other factors have been explained by Brooks Law

Brooks Law

 Brooks points to the main factors that explain why it works this way:
  1. It takes some time for the people added to a project to become productive. Brooks calls this the “ramp up” time. Software projects are complex engineering endeavors, and new workers on the project must first become educated about the work that has preceded them; this education requires diverting resources already working on the project, temporarily diminishing their productivity while the new workers are not yet contributing meaningfully. Each new worker also needs to integrate with a team composed of several engineers who must educate the new worker in their area of expertise in the code base, day by day. In addition to reducing the contribution of experienced workers (because of the need to train), new workers may even make negative contributions, for example, if they introduce bugs that move the project further from completion.
  2. Communication overheads increase as the number of people increases. Due to combinatorial explosion, the number of different communication channels increases rapidly with the number of people.[3]Everyone working on the same task needs to keep in sync, so as more people are added they spend more time trying to find out what everyone else is doing.
  3. Limited divisibility of tasks. Adding more people to a highly divisible task such as reaping a field by hand decreases the overall task duration (up to the point where additional workers get in each other’s way). Some tasks are less divisible; Brooks points out that while it takes one woman nine months to make one baby, “nine women can’t make a baby in one month”.
The problems associated with adding new teams is ramping up, individuals need to learn/understand
  • Business logic
  • Project processes
  • Best resources for situations
  • Project politics
  • Collaborate with others

A scrum team takes 2 sprints to get up to  speed, usually 50% efficiency for sprint 1 and 60/70% in Sprint 2.  When you add the added overhead of trying to collaborate with other teams this can slow new teams further and slow existing teams.  

Deadlines are the common cause of adding more people to a project which adds more pressure to the project and the newly arriving people.

A big overhead is integrating the work from multiple teams into one solution, even with continuous integration there will be problems when bringing solutions together

 

Sutherland one of the creators of Scrum mentions scaling in this article

InfoQ: Now that more and more larger organizations are using Scrum, the need for scaling increases. Can you give some suggestions how organizations can scale Scrum?

Sutherland: Scrum is designed to grow like a biological system. To create a baby, you need one good cell that replicates. If this cell is bad you have a lot of problems. So the challenge is to first get a single team successful to use as a model for other teams, then replicate it. As you grow, Scrum as large follows the same basic principles as Scrum in the small. If it doesn’t you have introduced waste into the system. You will build out organ systems, just as in the human body that have some specialization and need some training on how to handle cross team coordination, elevating impediments all the way to senior management, avoiding the introduction of unnecessary teams that can compromise performance, and scrumming the entire organization.

We have published several papers showing that a properly scaled Scrum can provide linear increases in global velocity, something never seen before in software development. However, you need to do Scrum really well to make this happen.

Hosk thoughts on scaling agile projects

The first rule of large agile projects is not to do large agile projects #HoskWisdom

Large Agile or Scrum projects are difficult but this doesn’t mean Agile/Scrum are not right for large projects because large projects are difficult and have similar problems no matter the framework or methodology used.

Agile projects have an extra dimension because quick decisions are needed from empowered product owners to give Scrum direction and feedback

Having multiple teams adds the need for more communication and collaboration and managing dependencies between teams.

There are lots of different scaled agile frameworks, many of them discussed in article below

Comparing scaling agile frameworks

Most of the frameworks try to solve problems with shared resources between teams and collaboration between teams (Scrum of Scrums meeting).

If you want existing teams working efficiently, you must remove project level impediments before adding new teams.  When a team works well then try adding one more team at a time.

There are other reasons for project troubles

Is Safe Evil

Conclusion

Big projects are difficult, hard and challenging and having people with experience of large projects helps.  There is pressure with big agile projects because sprinting and collaborating create conflicts and complexity.

Stick with the processes and aim to keep improving your processes and relationships.  Challenging projects are the most satisfying to deliver because you have to work hard and earn the achievement of a successfully delivered project.

A few last great quotes from Fred Brooks on why projects are delivered late
Fifth, when schedule slippage is recognized, the natural (and traditional) response is to add manpower. Like dousing a fire with gasoline, this makes matters worse, much worse. More fire requires more gasoline, and thus begins a regenerative cycle which ends in disaster. [page 14]

How does a project get to be a year behind schedule? One day at a time. Fred Brooks

picture from here

Study tips for the Scrum Certification

 We don’t need an accurate document, we need a shared understanding – Jeff Patton

I became a certified Scrum master yesterday, I love the title of Scrum master, it’s by far the most silly title, mixing rugby and ninja skills. There could be some alternatives

  • Scrum Ninja
  • Scrum master general
  • Scrum lord
  • Head of Scrumperation
  • International Scrum
  • Michelin starred Scrum Vicar

Why get certified

Certifications can be divisive, people who aren’t certifications can get angry about the silliness of them and can be found shouting “They don’t mean anything“.

I heard a good quote “Scrum masters who are certified are usually crap

What we should remember is a certification doesn’t mean anyone is good at their job, it means they have studied on how a tool works, which is different from how to use a tool.

The benefits of certification

I am partial to getting certified because it gives a deadline and creates a sense of urgency around learning a topic.

Studying for a certification gives you a broad in-depth knowledge of a topic, it doesn’t guarantee you will use that knowledge or you good at your job.    If you view knowledge as a tool then it‘s your job to make sure you understand what the tool is, how it works and when to use it.

I have written about CRM certifications What are the benefits of CRM certifications

Tips on passing the PSM 1 certification

The scrum guide has all the answers, the certification is multiple choice questions based on a 16 page document.  Read this small document many times and make notes.

Here are some useful pages on passing the Scrum certification

This book was quite useful and it has some practice questions

Scrum Narrative and PSM Exam Guide: All-in-one Guide for Professional Scrum Master (PSM 1) Certificate Assessment Preparation

Test yourself

I view studying for a certification as two steps, the first step is learning theory and ideas and the second step is testing yourself to reinforce the knowledge and be able to retrieve it at will.

Once you have read about Scrum you need to keep testing yourself until you can recall the information easily.

practice these exams until you get 100 percent (every time)

https://www.scrum.org/Assessments/Open-Assessments

15 questions

http://www.bostonagiletraining.com/Sample-Scrum-Master-Certification-Practice-Exam

Mikhail Lapshin 80 questions are great because they explain the answers by quoting the scrum guide.

PSM1 – Learning mode

finally this is a useful page on product owner

http://www.scrumcrazy.com/New+New+Product+Owner+-+The+Videos

 

 

Looking at the ideas behind SCRUM

“No Heroics. If you need a hero to get things done, you have a problem. Heroic effort should be viewed as a failure of planning.”
Jeff Sutherland, Scrum: The Art of Doing Twice the Work in Half the Time

I have looked at the concepts behind Scrum.

There are many projects which are Agile in name but chaos in reality #HoskWisdom

What is SCRUM?

Start with Wikipedia SCRUM definition

Scrum is an iterative and incremental agile software development framework for managing product development

 

Scrum is a framework which shares values with the Agile Manifesto

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

 

SCRUM and Agile for a time was seen as a magic solution to IT projects, a way to deliver a project successfully and quickly.  There has been a backlash and growing discontentment with Agile and SCRUM.

Here are links criticising Agile/Scrum framework

My experience of working on Agile projects is many of those projects are Agile in name but turn out to be a Waterfall-Agile hybrid often called WaterScrumFall.

Delivering Software with Water-Scrum-Fall

Here is a good point to compare WaterFall and Scrum

Agile vs Waterfall: Comparing project management methods

My experience of mixing Waterfall and Scrum is those projects struggle but not because of the way of delivering the project, these projects struggled because of people and the collaboration between the customer and the project team is weighted to one side (customer), successful projects are always a collaboration.  The mix of project style wasn’t the sole cause of the problems with the project.

Mixing Agile/Scrum with Waterfall removes the benefits of Scrum removing many of the opportunities to inspect and adapt.  One of the major disadvantages is the loss of transparency and time-boxing events.

The Scrum guide online is a simple 16 pages and is adamant you may not change any meetings, artifacts or meeting lengths

Hosk Experience of Agile

I worked on many Agile projects in name but only one was Scrum project which adhered to all the rules.

The project which stuck to all concepts was successful and enjoyable.  Estimating, the retrospective and the daily scrum all worked well and the team bonded. One of the benefits of a Scrum project is the transparency and clear understanding and regularity of the Scrum events and artifacts.

The Agile project in name but not in execution were disorganized and lacked clarity and direction.  The people on the project were not sure when things were happening and the purpose of events.

Little consideration of project was suitable for Agile\Scrum framework or if the customer could execute an Agile project.

One of the most frustrating part of a poorly executed Agile project is a daily scrum which goes on and on and on.  They should not go exceed the 15 minute timebox

Good practices

I have worked on Agile and Scrum projects but not thought about the concepts and ideas behind it.

What is Scrum and Agile

“Adaptive methods are called Agile.  There are many Agile frameworks.  The most famous Agile framework is Scrum.”

The Scrum Master Training Manual: A Guide to the PSM Exam
By Nader K. Rad, Frank Turley

 

Agile is a framework and Scrum is a flavour of it.  The focus on adaptive, prepares the customer the project will need to adapt and change.  The longer the project continues, the more you learn about the business requirements and more feedback from the customer on the iteration/solution created.

Scrum forces customer to prioritize the items on the product backlog, you can add the high priority items to the sprint, which should give the most business value.

The priorities and scope changes with time and feedback, Scrum acknowledges this will happen and encourage

Change is inevitable but customers don’t expect or like  it when it happens to their project. #HoskWisdom

When should you use Scrum?

  • You should use Scrum on projects where its suitable to do incremental and iterative development.
  • The product owner must have a good understanding of the business
  • Where a product is hard to define upfront
  • Where the customer is willing to devote time to the project
  • Customer is willing to make quick decisions

Agile Manifesto

I mentioned the Agile manifesto earlier but lets focus on two points.

Customer colloboration

Successful projects involved a good relationship between the project team and the customer.  The project should be a collaboration between the customer (business knowledge expert) and the technical team (solution experts).

Projects which don’t work well involves unsuccessful relationships between customer and project team dominated by one side, instead of a collaboration, failed projects are often involve one side dictating the solution.

When the customer (business experts) dominate you often end up with poor technical solution.

When the technical experts dominate the product doesn’t help the users achieve the business requirements or contribute to business goals.

Working Software

Sometimes people forget the goal of an IT project is to create a working solution to help the users achieve their business goals.

The Agile manifesto puts it right up front and focuses on getting a working product to the customer quickly to enable open and honest feedback.  Scrum welcome customer feedback which might disrupt the current iteration.

The difference between the planned product and working software can be huge and the sooner the customer can see it and try it the sooner the product can be changed to give the customer an effective product.

Software innovation, like almost every other kind of innovation, requires the ability to collaborate and share ideas with other people, and to sit down and talk with customers and get their feedback and understand their needs.

Bill Gates

Development Team

In Scrum all developers are equal, have the same title and are all focused on delivering the sprint.

The development team are self organised and cross functional, enabling them to deliver a sprint without any outside assistance.

Scrum focuses on team work and empowering the development to find solutions to problems.

Progress monitoring

Splitting the projects into sprints allows the project to never fall to far behind schedule and if it does the customer knows quickly and a discussion can be had.  This limits bad news to never being too bad before it‘s discussed with the customer (if it‘s bad).

Burndowns and velocity monitor the speed of the project and daily scrums make sure everyone is pulling their weight and problems are cleared.

The daily scrums ensure there is constant communication in the sprint team, progress is monitored and problems are raised with solutions found.

Any problems only have the potential to be one sprint long.

Product backlog

The product backlog is a prioritized list of requirements to be done.  A list of deliverable’s grouped into sprints and prioritized (sprint backlog)

The concept of a prioritized list of deliverables avoiding prioritising and say all the deliverable’s are all high priority.

The sprint backlog of a sprint is the items are not changed once the sprint is started.  This means the customer cannot add new items to a sprint and cause the sprint to be delayed.  This process stops users squeezing more functionality in and extended sprints.

The working software and feedback keep happening at the end of every sprint.

Successful projects involved a good relationship between the project team and the customer.  The project should be a collaboration between the customer (business knowledge expert) and the technical team (solution experts).

Projects which don’t work well involve unsuccessful relationships between customer and project team, instead of a collaboration, failed projects are often involve one side dictating the solution.

When the customer (business experts) dominate you often end up with poor technical solution.

When the technical experts dominate the product and hurts the users achieve the business requirements or contribute to business goals.

Sprint events

Each sprint has a number artifacts

Sprint

  • The items to be delivered during the sprint

Sprint planning

  • Done at the start of the sprint by the whole sprint team.  The estimating is done by group vote, which gives an accurate average

Daily scrum

  • A daily 15 minute meeting
  • progress is monitored
  • impediments are discussed and resolved
  • standup so not to take too long

Sprint Review

  • Showing the user the functionality of the sprint
  • Feedback

Sprint Retrospective

  • Discuss what worked well in the sprint
  • Discuss what didn’t work well in the sprint

Looking at the framework all the events and activities seem like a great idea and are likely to help a project be successful.  They give opportunities to inspect and adapt, to get feedback.

The reason many Agile projects go badly because often steps are missed and the Scrum team don’t understand what they are doing or why they are doing it.

Retrospective

One method for improving a team and individuals is to analyse past performance and work out what worked and what didn’t.  This reflection often doesn’t happen because people don’t have time or don’t want to do it.

Scrum ensures you learn from past performance by scheduling a retrospective after every sprint.  It looks at

  • People
  • Relationships
  • Process
  • Tools

What worked, what didn’t and how it can be improved for the next sprint.

Review

This quote is inspiring and scary

“We welcome changes in Scrum and encourage them to be demanded, because it increases satisfaction of the customer and will create a final product that better matches the needs of the customer Jeff Sutherland

The quote comes from the book – Scrum: A revolutionary approach to building teams, beating deadlines and boosting productivity

The quote initially seems like it could create a nightmare project with customer potentially giving lots of negative feedback but the goal of the project is create a product which is effective for the end users.  Reviewing the product and getting feedback is a core part of collaboration with the customer and will increase the chance of creating a product the customer wants.

Considering the concepts of Agile and Scrum

Looking a Agile/Scrum I can see many aspects of successful projects.  If you flip it round and look the common problems with IT projects

  • Product not doing what the customer wants or needs
  • Projects falls behind schedule
  • Customer  changing, adding or removing requirements
  • Customer and developer team are not collaborating well
  • Communication between the developers

Scrums uses many tools and techniques to resolve many of the common problems with IT projects.  The goals of Agile projects are positive and if executing well the benefits can be great, faster projects, working product shown to the user and feedback given often.

“Scrum incorporate the concepts of continuous improvement and minimum viable products to get immediate feedback from consumers, rather than waiting until the project is finished

Jeff Sutherland

Scrum: A revolutionary approach to building teams, beating deadlines and boosting productivity

Common language

A benefit of Scrum is it gives a common language for everyone to use.  Common rules and processes must be followed by the customer and project term as outlined by the Scrum framework itself which insists on it.

It makes the customer prioritise requirements and stop adding or changing requirements half way through a sprint.

Summary

Scrum is made up of many good practices such as adaption, incremental/iterative development and getting feedback as soon as possible.

A goal of Scrum is to get feedback from the customer on a product as quickly, so can be changed to create a product useful to the customer.

The individual components of Agile/Scrum are good practices which makes me wonder why it generates such negativity.

Many companies sold Agile to customers as a super weapon to lowers costs, deliver projects quicker but Agile is suited on projects which can be delivered incrementally and where people are trained and experienced delivering Agile\Scrum projects.

The Agile framework reminds me of someone who has got a hammer, suddenly all problems look like nails and all projects look like Agile projects.

Many companies did Agile/Scrum projects without training people in the framework, if you don’t have some people with experience the project could hit problems without someone to guide the project back on course.

Companies need to make sure employees are given time to learn new skills and are not on project work 100 percent of their time.  Agile projects can be relentless and it‘s possible the tasks are all focused on sprint backlogs with no maintenance, code refactoring, builds or house keeping tasks factored in.

Resources

The Scrum Master Training Manual: A Guide to the Professional Scrum Master (PSM) Exam

This ebook is free and gives a good overview

Scrum: A revolutionary approach to building teams, beating deadlines and boosting productivity

Written by Jeff Sutherland who was one of the creators of Scrum

Scrum and Xp from the Trenches 2nd Edition

You can buy it from Amazon or you can download it for free here

Confessions of a Scrum Master

free download here