(intro to this post on devops)
It has been a busy few months. Since my last post I have transitioned to a new role as part of day to day development (see linkedin.com/in/greger for more details) in the Collaboration Infrastructure Technology Group (UC, telephony, telepresence, conferencing, etc) in Cisco. Earlier, as part of Office of the CTO, I influenced the medium to long-term technology and architectural strategy, but as I had limited involvement in the day to day operations, it was a challenge to develop architecture that bridged near-term goals with longer-term strategy. Now, I’m less involved in long-term technology strategy, but on a daily and weekly basis involved in designs, decisions, and priorities made as part of development. Quite refreshing and comes with a bunch of interesting challenges!
If you have followed my blog, you have probably understood that I see architecture as something tightly embedded in day to day development, but more than just the sum of tactical, short-term steps. Also, you have probably detected my desire to see product management team and architects partner and work on the same thing: identify the whats and hows of building great products and services. Agile has for a long time changed my organisation and the last year or so, devops processes have really started to impact how we work. We have hardware projects, traditional waterfall product releases, and cloud services. This means that in my technology group we are currently using various flavours of waterfall, agile, and devops processes in parallel. It puts a strain on coordination and especially challenges existing roles in the teams. This post is about some of these changes.
Classic product organisations used more or less waterfall processes, and many so-called agile development organisations are using agile terminology to execute on a mostly waterfall mindset. However, with an increased focus on devops processes, the combined product/service organisation is facing another set of challenges in combining everything from hardware products and their software loads through legacy, monolithic, hosted applications to devops microservices. Tying these together to deliver robust, scalable services that offer great user experiences is a challenge along many dimensions: culture, people, competence, programming languages, tools, project management, goal setting, measurements, management, and roles.
One particular challenge is that not only does what we do change, our roles have to change as well. This is easy in a small organisation, but in a big organisation, the expectations other people have to what you are supposed to do is very important to your ability to get things done. So, let’s say I’m used to architects writing specs, then I will likely continue to ask for specs. Other people may not think an architect should write specs anymore, and they will ask for other things and maybe ignore the specs. This creates frustration and tension.
I believe most people in the industry are starting to understand the fundamentals of how devops are changing the roles of the development team and the operations team, and that they basically need to become one integrated team (with some new roles and new definitions of existing roles). However, the role of the product manager is also changing in the process, as is the project manager’s role, the engineering director’s, as well as marketing, sales, support, and partners. They no longer have releases to plan around and need to change how they work. I may cover some of these other roles in later posts, but in this post I will concentrate on the architect’s role.
First of all, a few words on what an architect is. In my understanding, an architect is a developer, a member of the engineering organisation, with a special responsibility for understanding the short-term and long-term business and technical requirements and making sure that the right technical deliverable is delivered at the right time with the right quality. Sometimes this is to get it done and deliver something less ideal from a code point of view. Sometimes this is to tell the product owner that some labour-intensive refactoring needs to be done in order to deliver a feature. My personal view is that a good architect also truly understands the value of the product to the end users and customers, and that he or she is a partner to the product manager or owner in defining and evolving the product. An architect can be a senior member of a scrum team (in Cisco, these are called technical leaders), or a more dedicated role that covers multiple scrum teams (senior technical leaders or principal engineers). In larger organisations, system architects typically span multiple products or services to create functioning solutions. In my view, regardless of scope, to be effective, the architects should be embedded in the engineering organisation. Some architects lead development teams, others have a more standalone role. However, the most effective teams seem to always three roles covered when decisions are made: the product owner (representing the vision and the problems to be solved by the product), the architect (representing how to deliver with the technical integrity of the product in mind), and the development manager (representing the developers, and making the trade-off decisions).
The architect is thus a member of the product leadership team. And with agile and devops, this role becomes even more challenging with the need to engage in more areas. Here are the some of the main areas of challenges:
- Software architecture and code structure: There are two big challenges I want to mention here. First of all, devops requires a modularisation into micro-services, and getting the right grouping of functionality and defining clean, scalable APIs are critical. Second, the whole thinking of how you develop a feature or a set of features need to change. Every time you commit code, you cannot break anything. This influences how you architect the feature, how you test it, and you need to think in terms of A/B testing, feature toggles, and metrics, metrics, metrics(!), and not in terms of feature branches and unit and integration testing.
- Software development processes: An architect needs to engage in code repository management, branching strategy, build ande delivery pipeline, test automation, A/B testing, feature toggles, 360 reviews, and much, much more. The architect needs to police the process and make sure that scrum teams working on various micro-services have the same view on the process and don’t break the builds. Very few other people will have the code knowledge and competence required to identify where the issues are and address them. The architect becomes critical to making sure that the devops machinery keeps working.
- Requirements and product management: A shift from waterfall to devops is a shift in power from product management to the engineering organisation. Where product management used to ask for features based on market knowledge and research, a high speed of introducing features, doing A/B testing, and measuring the impact and quality relegates all product managers (except those who are very technical) to focusing on the go to market aspects of the service, not the requirements. The architect’s role in this becomes critical as the architects are instrumental in defining what to measure, and what you measure, you improve (whether this power shift has to happen, is a separate topic…)
- Project management: Traditional project (or program) management should totally disappear. The software development process, the build pipeline, the metrics, the scrum standups, the scrum of scrum meetings, kanban boards, and the definition and usage of the APIs across the services, together with the automated tests together define and drive the project. The traditional tools of a project manager, like Gantt charts, dependency tracking, weekly project status meetings, read-outs to executives, escalations, and decision meetings based on analysis, are of absolutely no use in a properly executed devops process (note that badly executed devops processes will give ample room for such old tools and processes…). The architect needs to engage in the overall execution of the devops project to ensure that there is a constant evolvement of proper architectural frameworks, guidelines, automation, gates, micro-service designs, API/protocol designs, and much more.
- Architecture: Devops does not mean architecture work is not necessary. On the contrary, as the architecture is constantly and organically evolving, the architecture that emerges can either be a disaster or a robust construction. However, the “ivory tower” architect is history as the emerging architecture emerges both with and without a plan, and it can only be shaped by having constant touch with what is being developed. A spec or design document will be obsolete within a week.
- Management: I see a lot of very competent, senior architects who are brilliant at designing a flow or a new protocol, but who are lacking in managerial skills. Running efficient meetings with action items, pro et contra evaluations, closing decisions, setting deadlines and targets, chasing deliverables, and driving consensus are just examples of typical managerial skills that becomes more important for an architect.
There is so much more to how devops is changing an architect’s work, as always, the devil is in the details, but these categories have been helpful for me in understanding some of the challenges in evolving architects towards “devops ready.” The shift is similar already with agile development processes, but becomes more visible with devops.
Do you believe there are other important things that impact the architect’s role in devops? I’m happy to hear from you!
My next post covers the importance of one shared backlog backlog with all the actual work items visible, as well as the product owner’s role. Not directly focused on devops only, but relevant to devops as a good backlog grooming process is essential to getting anywhere.
RSS - Posts