agile

Are You Lost In All The High Priorities?

April 4, 2020 by Greger Wedel

As software product organisations grow, they can become the victims of their own sales success. Many high-priority customers and features put pressure on the engineering organisation to deliver, but even when using all the traditional ways to prioritise and optimise execution, you feel progress is slow and the overall “delightfulness” of your product is not what it should be.

The root cause is simple: as you grow in people and priorities, you have lost that focus you had naturally when the company was a small startup, but the commonly understood criteria for prioritisation and the execution capabilities of a more mature product organisation are not yet firmly established. The solution though, is not simple: on the top level, you know what the key priorities are, but then as you break down and distribute responsibilities, each part of the company seems to have a reasonable amount of work, and your organisation MUST now be big enough to execute on multiple things in parallel.

The Challenges

There are typically two key challenges in this situation. The first is that although the ownership of each priority is distributed, the delivery of the priority involves many parts of the organisation. This cross-team alignment typically involves the same people in each team and as they have been assigned different priorities, they will order their tasks differently. Still, everybody will try to satisfy other peoples’ needs, thus they will start up many tasks in parallel, and all priorities will slow down due to switching costs and that time is distributed across many tasks.

While the first can be seen in large, mature organisations as well, the second challenge is typically a startup/scale-up challenge: mature organisations have normally figured out their criteria for prioritising features and backlogs, how to groom the backlog efficiently, how to align dependencies across teams, basically, how to get things done through distributed mechanisms that don’t need escalations to management. In a less mature organisation, all these things are still being shaped. So, the organisation is typically less capable of executing many things in parallel.

You need to do something to ensure that the organisation increases total capacity through establishing the mechanisms that will work for you, your culture, your people, and your products. One approach that I have used successfully in both large and small organisation is what I call slicing, pacing, and themed releases. This is not a steady-state recommended scheme, but rather a mechanism to fast-track super-high priority product deliverables that are critical to your company. It can be used for a strategy pivot, building a new and different product type in a mature organisation, or to fast-track product-market fit in that critical phase where you know the high level, but need a fine-tuned product that really solves customer problems.

Slicing

The principle is pretty basic, but hard to execute on: define a very clear, very concrete priority that slices through all parts of your organisation; include at least one representative from sales, finance, sales engineering, support, UX/CX, product management, as well as from each of the engineering teams you need deliverables from. A related strategy is known as the “startup-in-a-startup” approach. However, if you isolate this new team as an internal startup, you get innovation, but you get no organisational learning in the wider organisation. This is where the first “slicing” happens: you create a slice through your organisation with hyper-focus on solving one and only one problem at the time, but you keep everybody in their respective organisations and with the mandate to shape the priorities of their home teams.

Make the on-boarding journey from signing-up to paying for our service into an interactive and delightful experience that drives conversion from today’s 1% to 5%.
Example of a charter for a slice

You should establish a “situation room” for the team, physically or virtually through a tool like Miro. This is where you have a visual representation of the concrete end-to-end problem to solve and where the team is responsible for continuously updating with plans and results as they go. It can be useful to have a concrete and representative customer used as a benchmark. The slice participants will each be responsible for working with their home teams to come up with ideas and solutions, and to bring back to the situation room the actual work done and show how it solves the defined end-to-end problem.

The end-to-end problem to solve should be finite, concrete, and need refining and iteration. The first task of the team should be to attack the assigned problem area and start breaking it down, and find our how they will attack the problem. Here, you will go different directions depending on how many engineering teams you need to rely on. If you are able to gather a group of full-time developers to work on the problem and you don’t have other dependencies, you can do agile sprints with at least bi-weekly demos to stakeholders (customer(s)!). If you have a more complex setup with multiple teams involved, you will need to introduce the second type of slicing and do pacing. This second type of slicing is done when you use the end-to-end problem and the situation room to agree to a set of common things to tackle across all the teams for the upcoming sprint. This way you make sure that the switching costs are low and that people work on the same things in the same order. If you have a back-end API team, this team should e.g. do a quick mock API to realise the proposed UX/UI, and then iterate together with the front-end team as they learn more about how to solve the problem.

Themed Release

If the problem you are solving is isolated from the rest of the products in your organisation, or it’s your entire product, you can quickly release functionality the way you are used to (and through devops and CI/CD for Software-as-a-Service products). However, if your slice is a part of a bigger product or platform where also other functionality will be released in parallel, you may be able to better scale the impact in your organisation and customer base through doing a themed release. Your slice is then equal to the release theme (to frame for other stakeholders what all this new functionality is solving), and you should release at regular intervals, not too often (as nobody will really pay attention), and not too far apart (to get the feeling of accomplishment and push towards a milestone in the team). 4-6 weeks can be a good choice as you can have 1-2 stakeholder demos and iterations to deliver something that has clear customer value.

Pacing

Finally to pacing: the more teams involved, and the more people involved, the more you will get drift in the project. Product engineers have a tendency to want to solve problems generically, so that it can be used by all customers. This is a good thing, but it introduces the risk that one team can spend a whole week on delivering something another team needs, when they could have done something simpler on Monday, unblock the other team, and then continue on making the feature more robust and generic the rest of the week. These opportunities for increased overall velocity can only be discovered across the teams. One way of doing pacing is to have pacing meetings that have a similar format to standups, but where the sole focus is to identify drift and focus on quick tasks that can unblock other teams.

If you haven’t done slicing, pacing, and themed releases before, you should probably start with only one. You will have most of your engineering resources already allocated to other priorities, so the “allocatable time” of your teams is probably far less than you want to think. Once the organisation understands how to execute on slicing and a themed release, you can evaluate running 2-3 such themed releases in parallel. Typically, over time the themed releases should be less and less an extraordinary effort, and more an approach to aligning product management, priorities, and engineering execution.

The Three Deadly Sins of Scale-Ups

February 22, 2020 by Greger Wedel

So, you managed to pull it through. You found your product-market fit (or at least partially), got customers to pay, and secured your funding to expand. Your tight-knit team where everybody knows what everybody is doing will now double and triple in size as you add more people to the team to support your growth. Congratulations! Make sure you take time to celebrate because you are about to embark on a new and equally challenging journey!

The people you hired for the first phase were most likely generalists who thrive when things are unclear and problems just need to be solved without any process or long decision processes. You now need to decide who to hire for the next phase. Do you hire more generalists? Smart people who are individualists who will figure it out or do you hire people from larger organisations who can help you put things into process and scale up? Chances are that you are already establishing an HR organisation, and you are about to distribute the control of who you hire to those you have just recently hired. Most likely, the new HR organisation will put together an interview process, hiring criteria, and individual managers will start making hiring decisions. You are supposed to hire quickly, and priority number one is to hire the best people, but not necessarily the ones you need. This is deadly sin number one: not having explicit and detailed criteria as well as hands-on control over who you hire in this phase. The people who you hire now will define your company in years to come.

Many years ago, during the dot-com boom, I worked in Silicon Valley and got exposed to the cynical view that:

“A-people hire A-people, and B-people hire C-people.”

Through the best of intentions, you are now hiring (hopefully) smart people who can help you reach the goals of scaling up your company. If not, you will get a lot of people who are good, but not star performers, and then they will hire people who are not that good. Be careful, who do you need and what do you want for your organisation? Smart or best are not good hiring criteria, what will they prioritise and how will they shape your organisation?

Anyway, some of these new hires will come from larger organisations (you need that). However, this is where you risk doing deadly sin #2. As you grow, people will loose touch with management decisions, who is doing what, how to get things done, and as growth is typically chaotic, you will get dependencies across different parts of the organisation that will slow things down or even prevent people from even attempting to do the right thing. The response is always the same (and especially from people who come from larger organisations): We need more knowledge sharing through presentations, newsletters, demos, documentation, presentations, wiki pages, you name it. AND, we need processes that can help us get things done: A program management meeting, let’s hire project managers, we need more coordination meetings, who can make sure we sync up across teams, how do we make decisions, who can describe how we are supposed to do X?

You were used to having an organisation where everybody just made it happen, now, you have people who see it as theie job not to make it happen, but to coordinate, call meetings, write summaries and project plans, and structure your Jira projects. And who do they engage? Exactly the people who used to spend all their time making it happen, so now they have to spend time in meetings, report on progress, and update tickets. And they don’t think things are going in the right direction…

It is all done with the best of intentions and there IS a need to coordinate more, document things, and create some processes. And after all, this is what worked for organisation X (replace X with your favourite large tech company). The two main problems are that:

this is often overwhelming to people, and you risk them disengaging from many of the things that are not directly related to them and
you are introducing bureaucracy (i.e. a predictable process), but is it at the right place (helping you scale) or do you create barriers for getting the right things done? Also, your early generalists are bickering, this is not what they like. Where they earlier just fixed the problem, they now have to follow a process and they feel they waste their time. You risk loosing some of the star performers that enabled you to do the right things early on

So, deadly sin #2, overwhelm everybody with knowledge sharing that they don’t really know if they need and not controlling strictly where you introduce the right process, so in sum you frustrate people and slow down execution.

This is where you are about to enter the ranks of established companies, and your employees will be able to complain about how much red tape and meetings you have, and how difficult it is to just do the right thing because there are so many people who need to have an opinion. Now we are getting to the worst of the deadly sins, #3: introducing top-down processes of program management, managers, decision processes, gating criteria, and other mechanisms that are meant to get overview and control. It is in the nature of people to seek control, of course the organisation needs to be in control. The move towards the top-down control is reinforced by the people you hired. People who are very comfortable with establishing processes and deliverables to get in control.

But what is really the alternative?! Not putting in place ways to get decisions done, coordinate with others, and share knowledge will also slow down your organisation. We hear about empowerment of employees and establishing independent teams that can make decisions without consulting everybody else. But is that desirable? Can you avoid top-down control and still move forward as one company? So, should you avoid it? You want the ability to deliver on your corporate goals, you want to be able to estimate delivery dates, and you want to report on the progress, right?

De-coupling of dependencies between teams and individuals is the only scalable alternative to top-down control. It requires careful design of organisation, responsibility, and accountability.

This is really, really hard, but it is the only way you can get employee empowerment and the engineering and product velocity you want. If that is what you prioritise. The top-down control and planning may be something you prioritise, and you are willing to sacrifice velocity to get it. However, if speed is what you need, you need to organise small teams that are not dependent on top-down planning processes, where dependencies on other teams can be loosely coupled, and where the teams and the individuals are comfortable with making decisions and deliver on what they believe is the right things to deliver.

Teams will in reality never be independent, there will always be some other teams that will be impacted by decisions or that rely on what is delivered by others. Also, you want people to run in the same direction. This means that you need two things to get a de-coupled organisation:

a strong sense of strategy and purpose down to team level that guides decisions and
technical and organisational mechanisms that make it simple to understand how to interact with other teams (i.e. loosely coupled).

Designing a loosely coupled organisation is not trivial, especially when the urge to get that overview, that top-dow control, is so strong. However, if somebody pushes a change to a repository, the right documentation is updated, and the right people are notified, and they know how to change and update their own code, then you are onto something!

The key challenge is that the urge and the ways to implement top-down control are generic and well-known, while the path to de-coupled organisations is always specific to the organisation and requires careful and step-wise implementations that are tested and continuously improved. This is why the three deadly sins are so connected: the people you hire will determine whether you are able to scale through de-coupling or top-down mechanisms.

A Practical Approach to Engineering Leadership

August 17, 2019 by Greger Wedel

In my previous post on the new engineering leadership skills needed in a devops world, I called for a more systematic approach to developing leadership capabilities in engineering. But I gave no answers. In this post, I’ll share my practical approach to developing leadership capabilities in an engineering organisation.

When developing great software, there are some fundamental questions all engineers have:

What can I decide?
Who can make decisions on things I cannot decide?
Who do I need to talk to or involve when resolving an issue?
Who determines the targets for what we make, how we make it, and how we go about doing it?
What is expected of me and what kind of rewards do I get when I meet expectations?

In a small startup team, you don’t really need a lot of these things explicit. The goals, actions, and rewards are mostly shared across the team, and everybody contributes where they see a need. The entire team has the same priorities, the tradeoffs are decided upon continuously, and changing course is done quickly.

However, you cannot grow much as an organisation without introducing slightly different priorities in different teams or roles. You are starting to introduce the cost of coordination. For a while, you can align everybody around the same priorities through some kind of coordination process or meeting that gives you the “top 5 things to work on this week”. But to allow rapid growth you need to at some point allow different people or teams to focus on different priorities. I have seen numerous approaches to solving the resulting coordination problem, and most either introduce too much process or too little.

There is no single answer that works for everybody in every situation. However, if we go to the fundamentals of how to build great software, there are four different areas that always have to be covered in some way:

Build high quality code, designs, and other tangible stuff
Knowing what to make to solve the customer’s problem
Knowing how to technically solve the problems and deliver something that will work over time
Knowing how to go about making the tangible stuff like code and actually delivered what’s agreed upon

The skills that go into each of these four areas are very different. And the more people involved, the more coordination and collaboration is needed to make sure everybody is pulling in the same direction. Ideally, you want everybody to be 100% productive, but that is not possible as long as there is some kind of dependency between what people are supposed to do. How do you get the productivity percentage as high as possible?

Typically, each of these responsibilities translates into one of four roles (not individuals):

Developers build stuff
Backlog owners manage the priorities and the definitions of what is to be made
Technical architects determine how to technically solve problems
Engineering managers lead people, the process of development, assign the right person to the right task, and ensure velocity

These are roles, not individuals, so many engineers will wear multiple hats. Also, there is one missing part here: the service or product. Each organisation has to break down its services and products (and thus code bases) and group them in a sensible way. Once that is done, it should be possible to identify the developers, backlog owner, the manager, and the architect for each. An engineer can be a developer on one code base/service, an architect on another, as well as manage a deliverable.

This break down into modules and components is absolutely critical to the productivity of your engineering organisation and subject to Conway’s law.

Unless you match the identified ownerships and roles with your modules and components and get that right relative to your organisational structure, you are introducing inefficiencies in your software architecture and how people communicate. This is a continuous process and needs to be adapted to the changing needs with respect to both features and the underlying system architecture.

Going back to the questions at the top of this post (that most people have in a software development organisation), it should now be possible to answer the questions. You don’t have to go through a central manager or (..!) a committee or a process, you can reach out directly to the person in charge who can make a decision. E.g. if you are a backlog owner and you need more engineers to work on your backlog, you can talk to a manager. If you need an evaluation for how to implement a certain functionality, you talk to the architect. If several components need to communicate through an API, you know exactly whom to invite to discuss this.

You thus get an organisation that can resolve most day to day matters without central coordination.

Typically, you will have triplets of lead people for each of your main components: a manager, an architect, and a backlog owner. The development team can thus be productive by getting help from each of these three. Ideally, all three roles/people are co-located physically for highest productivity since high velocity requires quick and frequent touch points.

There are two other roles missing in this picture: if the team is a product team, you need a product manager role (who could also be the backlog owner, dependent on skill set). If your component or product has a UI, you should also have a UX lead who can work with the backlog owner and product manager to get clarity on the deliverables. See also an earlier post on the backlog and the product owner role.

This model is also good for training and career development. There are four areas where skills can be built dependent on individual interest. As engineers grow from juniors to become more senior, they will take on more roles and expand what they take ownership of. Some may want to take on manager roles and lead teams. Some may want to deep dive in a very narrow technical area and become lead architect.

Remember that any changes in an organisation will take time. You need to repeatedly clarify the expectations, the opportunities, show that real authority to make decisions come with these roles, and systematically encourage people to take ownership. And if people are uncertain, you need to exemplify what concrete tasks and actions are necessary and expected.

The reward is an empowered engineering organisation that can deliver great software with high velocity!

Wanted: a New Set of Engineering Leadership Skills

June 30, 2019 by Greger Wedel

A while ago I did a series on how key roles in the engineering organisation changes as a result of introducing devops practices. I covered the software architect, the program manager (at that time I was at Cisco and some of the projects were huge), and the product owner, but left out the engineering manager and the product manager as other priorities called. In this post, I circle back to the key skills necessary for engineering leadership, independent of the role you have.

What makes good leadership is sometimes seen as magic. It is difficult to get it right. On the other hand, engineering leadership can sometimes seem like black magic! You typically don’t have the sales or the financial aspects, but you are supposed to lead people who are rumoured to be motivated by weird things regular people don’t even know what is, and technology development is so fast that it seems that last week’s new stuff is now old. And in software development, you are supposed to deliver a product that at best seems fuzzy, at a pre-defined time, and with a limited number of engineers where your most productive individual can deliver 10, 20, or even 30 times of the least productive individual.

One challenge is that you can bend pretty much everything: what is shipped, the quality of the features that are shipped, the quality of the code, and the technical debt you leave for later by cutting corners. And in addition you are supposed to be innovative and solve the customer’s problems in a “delightful” way. A good engineering leader understands the customers, the business, as well as how to solve the customers’ problem. Getting all this right is very challenging and why I love building things in code!

I did a quick review of various engineering leadership programs and found MIT’s Engineering Leadership Program to be one of the most explicit on what they believe engineering leadership is. They emphasise six core capabilities where five of them are common with general leadership: core personal values and character, relating to others, making sense of context, visioning, delivering on the vision, and technical knowledge and reasoning.

We now know that a key characteristic of high-performing teams is trust among its members, and humans are more capable of trusting others when they feel safe. Any change introduces uncertainty, which again can reduce the feeling of being safe. In an environment of developers used to code and solve problems in front of their screen, the introduction of devops can be a dramatic change. We know from the introduction of agile that tension can raise quickly. Devops is an even more dramatic change in how developers work. Thus, anything that can be done to increase the feeling of being safe is critical! Simon Sinek has done a lot of work on trust leadership and this quick introduction to his thoughts on creating a circle of safety is a good use of your time.

An interesting difference between Europe and the US is how management education in the US is seen as something separate from the professional degree (MBA or engineering leadership programs). In Europe, leadership is embedded in professional degree programs, and it is more often assumed that a manager has a professional education and then promoted based on merit to leadership positions. There are pros and cons with each model. In Europe, it is often a problem that the only career path is management and the best people in their area become managers. You risk loosing a great specialist and get a lousy manager. However, if the transition works out, you get a very efficient manager who knows what is important and can guide others in doing their best. Regardless of model, it is important to promote people to leadership positions who have a real passion for accomplishing great things through other people, not only doing things themselves!

Devops is an approach to software development that is only possible if your product is a service that you operate and control, typically from a public or private cloud. This is called Software as a Service (aka SaaS). One of the key characteristics of devops is that the team writing the code is also responsible for operating the service. This allows modification of code as an immediate response to issues, requests, improvement needs, learnings, and feature requests. Jonathan Rosenberg, CTO of Cisco Collaboration, wrote a very good post explaining what this really means.

However, what are the requirements of a good engineering leader when you develop Software as a Service? Let’s look at some of the relevant things that change with devops:

Classic project management becomes difficult as there is no project, just many small chunks of work being done in parallel
Realising customer needs and desired business outcomes become a more direct deliverable as the whole product development process is condensed into the devops process
The team’s responsibilities get much bigger, and the type of people and number of roles expands where QA, operations, security, and other functions need to be taken care of by an integrated team
It may make sense to organise product management, product marketing management, technical marketing engineers, as well as support into the same group
The architecture of what you build typically shift from one integrated piece of code into many smaller, loosely coupled components (called micro-services). This has an impact on how the teams are organised and how they work
The continuous deployment pipeline and your test infrastructure become critical assets
Reliability and uptime become important in areas where you earlier could be lax (like your build server)
The team becomes responsible for delivering on key business metrics

This list is by no means exhaustive, but offers a flavour of the type of changes a team experiences when introducing devops. There is one immediate thing that is obvious: the scope of the engineering leader’s work has grown tremendously, both in topic areas and type of people and deliverables. This calls for a very different type of engineering manager than the classic “first among peers” approach where the most senior developer also gets a manager responsibility.

In the new devops-centric engineering organisations, HR and senior leaders need a much more systematic approach to educating its engineering leaders and train, coach, and mentor people to reach a broad set of leadership skills, and who also thrive on the challenges of leading software organisations.

The Program Manager in the Agile/DevOps Organisation

March 24, 2016 by Greger Wedel

This is another post in my series on transitioning from waterfall to agile and devops processes. This post covers the program manager role, a rather unassuming role in some organisations, an important role in others, but nevertheless a role where the right person can potentially make a big impact. There are a few different types of program managers, so let me be clear on the type I will discuss here. Microsoft uses the title for product managers with a horizontal responsibility, i.e. functionality that should go into multiple products. Other organisations use the title for a role that is similar to a product manager tole. Typically, a program manager of this type has an independent responsibility for business results within a certain area. A very different program manager role is a project manager for a deliverable that crosses multiple independent products or teams. Often, a program manager of this type will work with multiple project managers, one for each product involved in the deliverable, but does not have an independent ownership of business results and thus becomes more like an advanced project manager. In Cisco, program managers are often of this last type (and clearly lacks the first type), and alas, some larger products may have their own program manager working alongside the project manager to manage and coordinate outside the product team.

I may cover the consequences for organisations not having the “program manager with business ownership” role in a later post, but here I will cover the “program manager as an advanced project manager” role.

Of the many roles in a development organisation, the program manager role may be the role that involves the most painful change in mindset and daily tasks when a large organisation moves to agile and devops. The traditional tools of a program manager is no longer of much use: defining deliverables, setting and tracking milestones, managing dependencies, measuring progress against the plan, reporting status and escalating issues to stakeholders… But why is that? The reason is that the traditional program management model and associated tools assume a planning horizon (e.g. ship within 6 months), a clear view of what is “inside” and “outside” of the boundaries of what is managed (e.g. we need to ship a and b and rely on c and d to deliver x and y for us to be successful), a desire and need for a pre-deterministic plan for what is “inside” (e.g. we are going to ship a,b,c, and d), and that any deviation from the plan, including “outside” changes, are bad and should trigger a corrective action or some kind of mitigation.

ALL of this is wrong for agile and devops! For one, changing plans is desired, that is called learning. Also, the boundaries of what is “inside” and “outside” may be shifting as the teams learns and gets feedback, and reacting to change is something that each member of the team should do immediately. Planning should be done iteratively based on learnings and changing requirements, and what is actually shipped is not fixed as teams commit to a prioritised set of tasks (backlog), not a fixed set of deliverables.

Alright, so the program manager role is dead, is that what you say?! Yes, the OLD program manager role is dead, really dead. However, the reasons for why the program manager was needed are still there: ensure that multiple teams and projects with independent execution paths coordinate priorities and execute efficiently so that real cross-product value can be delivered to the customers with the right quality!

Can program managers re-invent themselves? I really don’t know, some of them may be stuck in old ways of doing things. There are of course many flavours of agile that companies may choose to adopt, like SAFe, where you get clear definitions (http://www.scaledagileframework.com/rte/) of what a program manager is supposed to do. The problem I have with such frameworks and role definitions is that activities, tasks, and goals are often defined in terms of artefacts and ownership of activities defined by the framework. Thus, the role quickly becomes statically defined, which is contrary to the whole point of agile: constant improvements reacting to learnings. By the way, as a rant on the side: that is probably also the biggest problem with frameworks like SAFe in general. They capture some important learnings and concepts, but become stale and not a framework for change.

I really like to get to the core of what we are doing and what we are trying to achieve. E.g. “Manage risks and dependencies” is an abstract statement, and it involves too many implicit definitions of what a risk is, how you manage it etc. But looking at the problems I have seen in agile and devops, a typical program manager personality can contribute buckets to some of them (but constantly change HOW to contribute to them as the organisation evolves):

1. What do we want to deliver and when? In large programs, multiple product managers need to agree to a set of ordered business priorities. Without clear and shared priorities across the products that are to be delivered together, we get absolutely nowhere. This is the role of the product managers (or rather ironically the role of the other type of program manager in organisations that have those). As covered in the post on the backlog, I believe in a true and single owner of a backlog (delivering on the priorities), and the backlog being defined as the task list a development team uses in their iterations. When a team of product managers come together to define shared business priorities, the team of product owners need to come together to tie their backlogs together (as described in my backlog post). A group of peers with priorities outside the group has always problems with closing decisions because conflicting product-specific priorities will interfere. A program-level product manager (aka solution product manager aka that other type of program manager.. puh!) can be used to drive the program/solution priorities and resolve the conflicts between the program-level and product-level priorities. Likewise, a program-level product owner can work with the product-level product owners to tie the backlog items together so that co-development can be done (where co-development means actual coding done in the same iteration with developers talking to each other). This program product owner role can be filled by a program manager (assuming some technical knowledge).

Solution Product Manager ——— Solution Product Owner ——— Solution Program Manager

|| || ||

Product Managers Product Owners Project Managers

The flow is thus: define and prioritise shared business priorities —> define interlinked product-level backlog items to deliver on business priorities —> plan co-development where needed. I tend to talk about “backlog items that are flagged for co-development”, which basically means that there will be another (named) developer in another team (or more) who should be involved when writing and testing the code for that backlog item.

<Whining rant> When writing this, I really feel the urge to go back to a startup environment and the simplicity of making decisions and pushing things out the door… The less people who don’t code, the higher feature velocity you get. But then again, you typically have just one or a few products and limited coordinations needs… </Whining rant>

Another important thing in devops is metrics and A/B testing. This input should be continuously brought into the business priority process. Often there is a breakdown in communication between engineering (doing metrics and A/B testing) and product management (setting the priorities). I have yet to see program managers really making a difference here, strong architects and engineering leads tend to basically take ownership of parts of product/service improvement, but I believe this is an area of real value creation where a good program manager can make a difference.

Another important aspect of program-level work is to set the right targets and make sure engineers can focus on getting things done. It is important to go to the right level of detail on the program-level of what a specific program backlog item really means. Getting to the right level of detail on wireframes, prototypes, and requirements through demos, transfer of knowledge sessions, and white-boarding is important to make sure that the engineers from all of the teams involved in co-development have a shared understanding of what needs to be done. If there is one person responsible for driving this on behalf of the product owners, things seem to move faster.

Of course, another important part of this phase is to get the architecture right, engaging the senior engineers in each of the teams and preferably with a senior architect with program-level knowledge and ownership to lead the process. In Cisco, there is a separate technical leadership career path where director-level technical leaders (principal engineers) are expected to define and supervise architecture in larger programs and also across programs and business units (I’m btw one of those).

2. How do we tie together iterations to do co-development? Once the priorities have been agreed to on the program level, the engineering managers and product owners need to coordinate and identify when and how to do any needed co-development. This will include backlog grooming, developers planning an iteration together, sharing builds, and coding together possibly on a day to day basis. Two and two teams can manage this nicely on their own, but if three or more product owners and engineering groups need to coordinate, there is often the need for a facilitator. Again, this is more of a solution-level product owner type of role, not a program manager, but a technical program manager would fit nicely in this role.

3. How do we bring together the products, validate that iterations delivered planned outcomes, and qualify the process towards release/deployment? Once an iteration has been done, there is a need to bring builds of each of the products (or services) together to do testing and validation of the deliverable on a solution-level. In a true devops environment with continuous deployment, you will get this validation quickly, but if you deliver products and not devops services, you will need some solution-level live test environment. Solution-level testing needs to be done (again, preferably done as part of the iteration…), and if everything is ok, the builds can be deployed to a wider audience, either internally or to early trial customers. This is a critical stage as solution-level bugs will show up. There is a need for a strong person managing the maturity of features delivered and determining whether to allow the features to go beyond the initial solution test bed. In a devops approach, you will have developed features with #debug flags and feature-toggles, and features can be turned off if they are deemed not mature enough for deployment to a wider audience. The program manager can be an excellent watchdog in making sure that the right quality is achieved before releasing features to a wider audience.

4. Once a development cycle reaches an end and we want to ship, how to get things out the door? We want engineering to have a rhythm, a two-week pace that will give us some small, new improvements and features for every iteration. Ideally, engineering will not miss a beat when the products are released. However, more likely there will be a need for one or two hardening iterations at the end of a development cycle to clean up bugs, improve stability, and fix remaining issues. Once this has been done, all the code repositories should get a release branch, and the bulk of the team should pick up new feature development for the next release, while some time should be allocated to fixing bugs on the release branches and merging back to trunk. Dependent on the products/services, this final step before releasing products to customers can be long or short, but preferably it should be as short as possible. Typically, tasks will be final stress testing, solution-level manual testing, documentation finalisation, knowledge transfers to customer service people and sales engineers, release notes, bug scrubbing, as well as other steps to ship the software. This whole process play more to the traditional strengths of a program manager who is an obvious candidate to own this step of the process.

As you can see, with references to artefacts like backlog and list of business outcomes and to various roles, I get dangerously close to defining a framework of my own 🙂 The difference is that I here describe a(n ideal?!) system of people and deliverables that MAY work, but that must be adapted to people’s experiences, preferences, and beliefs through an agile process where people bounce off each other and change together. If not, the deficiencies inherently found in any human activity will settle and some important things will continue to be dropped, people will have overlapping responsibilities, and teams will work in slightly conflicting directions without the continuous improvement that agile promises.

But in sum, this post is an argument against the opinion that agile and devops mean that you don’t have to do any planning, documentation, or process management. On the contrary, some of your planning and process management need to be even more diligent than in traditional waterfall. So, don’t sack your program managers before you see how they can adapt…!