architecture

A Production-Quality Flutter Starter App, Take Two

January 21, 2020 by Greger Wedel

Take two of the production-quality Flutter starter app after nearly seven months of tremendous progress in the Flutter community. The starter app has been updated to the latest Flutter releases, best practices, and improved in many aspects.

When I open-sourced my work in July 2019 as a result of an investigation into the state of the Flutter tools and eco-system for production-quality mobile apps, I also decided to write an article about it and share it on Medium. I had not expected the overwhelming response and, as of today, the nearly 7,000 reads, 63 stars, and 22 forks. It must have filled a gap in people’s desire to learn more about Flutter.

I typically tinker more over holidays, so in mid-December I decided to do an overhaul of the starter app as I had a number of things I wanted to improve and update. First of all, I updated the app to use the latest Flutter version, 1.12.13+hotfix.5 (Dec 10, 2019), as well as update the app to the latest Android SDKs, Xcode, and tooling. Many of the packages I had used also had newer versions. I learned that keeping track of newer versions is important as the amount of work to update everything after a while can be daunting. Indeed is Flutter a fairly new and rapidly changing project, but Android and iOS are also changing rapidly, so to upgrade everything and make sure that everything worked properly was a few hours of work. I also realised that some of the packages I had relied on, like flutter_crashlytics, were no longer maintained and new packages had showed up that were maintained better (firebase_crashlytics).

One of the painful things about Flutter is the fact that Flutter is an overlay and framework on top of Android and iOS (and other platforms now), which means that a Flutter project is really three different projects where the Flutter project includes tooling that updates the underlying Android and iOS apps. When you build an app, you really build the Android and the iOS projects separately, and when you want to upgrade these underlying projects, you can use flutter create . to update the project files to the latest templates. My experience was that this didn’t work smoothly, probably because I had to tinker with the platform projects to introduce auth0, crashlytics, as well as Google Maps. Upgrading to AndroidX, the new 29 API, and the latest Xcode updates were not painless as I don’t have any experience with the platform-specific technologies.

Internationalisation is important to get right in larger projects from day 1. The IntelliJ/Android Studio plugin and package flutter_i18n still seemed to be maintained, and a newer version resolved the issue with lack of default language for countries with multiple languages (beyond English). Another thing that is important to have established early in a project with multiple developers and UI/UX people, is the templating/themes for the UI. I thus added a more complete theme that I generated from https://rxlabz.github.io/panache_web/.

In the first version of the app, I was primarily interested in the “plumbing” of making a production-quality app, so I sort of ignored the UI. I now put some effort into making a half-decent UI, and below you can see the updated looks. The new theme is cleaner, and although the UI is fairly simple, it is now more pleasant and can more easily be extended and changed.

I also wanted to test out the use of Google Maps, and in the process of showing a map, I added a new widget type as a library widget in the ui/widget/ folder. I found the AnchoredOverlay widget that I used for showing the Google map shared by one of the Flutter maintainers. As the new locations flow in, the map overlay is updated to show the map of the latest coordinates.

I also did several cleanups throughout the app. It turned out that I didn’t have the Podfile under version control (guess nobody built the iOS version…) and my .gitignore missed a few files. Also, I had hard-coded includes of files several places, something I learned was not best practice (you should include from your app package, not from the file system). My “keep-it-dense” C origin showed up and made me get rid of trailing commas, something I learned the Dart formatting relies on to get the widget code formatted nicely, so I added back the commas 🙂

The flutter-auth0 package was not released as an official package back in July, 2019, and I sort of maintained the repo while the original developer was busy elsewhere. The effort showed an interest from folks to get the package released, and Denny Segura luckily re-engaged and got it released. I thus replaced the use of a local package with the released version and updated the code to use the new flows and methods.

One of the things I was not happy with in the original starter app was testing. I had some unit tests, including some mocking using mockito, but I had no widget tests, and I really wanted to have a look at the integration test capabilities of Flutter. I thus put some effort into improving the mocking capabilities, and introduced a map of mock objects into the main appstate, thus making it super-simple to leverage the state updates through the widget tree to turn on and off mocking. I haven’t seen this pattern before, so I would be really happy to get feedback on how wise that approach is!

The added benefit of these mocking capabilities was that I was able to use the flutter_driver and add proper integration tests by leveraging the inter-process communication between the observatory (the instrumented app) and the test process. This fully integrated testing on a simulator as part of the Flutter framework is super-nice! The two main drawbacks are that the tests need to be written in Dart and that the Flutter driver cannot interact with system dialogs (yet). The first problem is a process and people problem: Either you need the developers to develop the integration/UI tests as part of their development work (should be feasible once it’s set up like I have) or you need your QA people to use Dart. They probably already know Python, so they should not have a problem with Dart. The lack of interaction with the system dialogs caused some problems until I created the mock pattern mentioned above for libraries that interact with external entities. Hopefully, this is something that will be possible to add for the Flutter team.

A side note on the flutter_driver: The documented way to run flutter drive is really cumbersome as it always starts the observatory (instrumented) app and then run the tests. See the app’s README for how you can run the instrumented app in hot reload mode, while re-running your tests separately. Thanks to Maurice McCabe for explaining how to do this!

One of the hot topics of the Flutter community is state management. In July, there was no clear winner, and after evaluating several state management approaches, I used my experience (and desire for clean and simple architectural patterns in larger teams) and chose scoped_model. Later, state management has become a bit more nuanced. When I did my research starting about a year ago, all the state management solutions solved two problems: 1. how to put state data into buckets and cleanly separate different type of data into different buckets, so that state change notifications did not lead to repainting of all widgets, but only the widgets that used the data that changed, and 2. how to actually propagate changes throughout the widget hierarchy and make these buckets of data state available to the right widgets.

With the refresh of my app, I was curious to understand provider, and I wanted to replace scoped_model with provider for state management as a talk at Google I/O quickly made provider the hottest (and recommended) approach to state management. As scoped_model and provider were built by the same people (guy), I was able to quickly wire up provider and replace scoped_model within 30 minutes. However, that was when I understood that while scoped_model solved both of the two problems above, provider is really made to solve problem #2 above, i.e. how to manage state data distribution throughout the widget tree (and life cycle). First, I found it really cumbersome that I had to make explicit choices whether I wanted to listen to changes or only access the methods of the state objects. But, when the dime suddenly fell down, and I realised that provider is a much more powerful tool and that it forces you to do the right thing, I started looking at problem #1 again. With scoped_model, I did a very simple thing and made a single bucket of state data, called appstate. With provider, I was forced to look again at this design because I needed to refresh a widget when there was a new GPS location available, but I only needed to get access to the global appstate to run a method in that same widget (and not refresh the widget).

As a result, I split out a new locstate state model, thus more cleanly separate the location data from the global appstate. If you look at the models (lib/models/) in the app, you will see that they are very simple. They are basically basic classes with a ChangeNotifier mixin: class AppStateModel with ChangeNotifier{}. This gives me the notifyListeners() method, the only thing needed to notify about a state change. In the updated starter app, I have on purpose NOT introduced a state management system to solve problem #1, but used provider to propagate the state properly.

In my research, MobX came up as a state management solution that plays really well with provider and where you can structure your state data (“stores”) in much more advanced ways than what I have done in the starter app.

In line with the original app and my desire to make it ready out of the box, I have committed the Google Maps API keys to the repo (though throttled the quotas), and I have also set up Codemagic to build the latest version of the app. If you want to install on Android, you can use the app-release.apk file. Unfortunately, it is non-trivial to build something you can install on iPhone, but you can easily build it yourself and run it in a simulator.

I hope this update to the starter app is useful and…

Happy fluttering!

A Practical Approach to Engineering Leadership

August 17, 2019 by Greger Wedel

In my previous post on the new engineering leadership skills needed in a devops world, I called for a more systematic approach to developing leadership capabilities in engineering. But I gave no answers. In this post, I’ll share my practical approach to developing leadership capabilities in an engineering organisation.

When developing great software, there are some fundamental questions all engineers have:

What can I decide?
Who can make decisions on things I cannot decide?
Who do I need to talk to or involve when resolving an issue?
Who determines the targets for what we make, how we make it, and how we go about doing it?
What is expected of me and what kind of rewards do I get when I meet expectations?

In a small startup team, you don’t really need a lot of these things explicit. The goals, actions, and rewards are mostly shared across the team, and everybody contributes where they see a need. The entire team has the same priorities, the tradeoffs are decided upon continuously, and changing course is done quickly.

However, you cannot grow much as an organisation without introducing slightly different priorities in different teams or roles. You are starting to introduce the cost of coordination. For a while, you can align everybody around the same priorities through some kind of coordination process or meeting that gives you the “top 5 things to work on this week”. But to allow rapid growth you need to at some point allow different people or teams to focus on different priorities. I have seen numerous approaches to solving the resulting coordination problem, and most either introduce too much process or too little.

There is no single answer that works for everybody in every situation. However, if we go to the fundamentals of how to build great software, there are four different areas that always have to be covered in some way:

Build high quality code, designs, and other tangible stuff
Knowing what to make to solve the customer’s problem
Knowing how to technically solve the problems and deliver something that will work over time
Knowing how to go about making the tangible stuff like code and actually delivered what’s agreed upon

The skills that go into each of these four areas are very different. And the more people involved, the more coordination and collaboration is needed to make sure everybody is pulling in the same direction. Ideally, you want everybody to be 100% productive, but that is not possible as long as there is some kind of dependency between what people are supposed to do. How do you get the productivity percentage as high as possible?

Typically, each of these responsibilities translates into one of four roles (not individuals):

Developers build stuff
Backlog owners manage the priorities and the definitions of what is to be made
Technical architects determine how to technically solve problems
Engineering managers lead people, the process of development, assign the right person to the right task, and ensure velocity

These are roles, not individuals, so many engineers will wear multiple hats. Also, there is one missing part here: the service or product. Each organisation has to break down its services and products (and thus code bases) and group them in a sensible way. Once that is done, it should be possible to identify the developers, backlog owner, the manager, and the architect for each. An engineer can be a developer on one code base/service, an architect on another, as well as manage a deliverable.

This break down into modules and components is absolutely critical to the productivity of your engineering organisation and subject to Conway’s law.

Unless you match the identified ownerships and roles with your modules and components and get that right relative to your organisational structure, you are introducing inefficiencies in your software architecture and how people communicate. This is a continuous process and needs to be adapted to the changing needs with respect to both features and the underlying system architecture.

Going back to the questions at the top of this post (that most people have in a software development organisation), it should now be possible to answer the questions. You don’t have to go through a central manager or (..!) a committee or a process, you can reach out directly to the person in charge who can make a decision. E.g. if you are a backlog owner and you need more engineers to work on your backlog, you can talk to a manager. If you need an evaluation for how to implement a certain functionality, you talk to the architect. If several components need to communicate through an API, you know exactly whom to invite to discuss this.

You thus get an organisation that can resolve most day to day matters without central coordination.

Typically, you will have triplets of lead people for each of your main components: a manager, an architect, and a backlog owner. The development team can thus be productive by getting help from each of these three. Ideally, all three roles/people are co-located physically for highest productivity since high velocity requires quick and frequent touch points.

There are two other roles missing in this picture: if the team is a product team, you need a product manager role (who could also be the backlog owner, dependent on skill set). If your component or product has a UI, you should also have a UX lead who can work with the backlog owner and product manager to get clarity on the deliverables. See also an earlier post on the backlog and the product owner role.

This model is also good for training and career development. There are four areas where skills can be built dependent on individual interest. As engineers grow from juniors to become more senior, they will take on more roles and expand what they take ownership of. Some may want to take on manager roles and lead teams. Some may want to deep dive in a very narrow technical area and become lead architect.

Remember that any changes in an organisation will take time. You need to repeatedly clarify the expectations, the opportunities, show that real authority to make decisions come with these roles, and systematically encourage people to take ownership. And if people are uncertain, you need to exemplify what concrete tasks and actions are necessary and expected.

The reward is an empowered engineering organisation that can deliver great software with high velocity!

A Production-Quality Flutter Starter App

July 18, 2019 by Greger Wedel

Production-ready flutter apps are rare species…

Flutter promises a lot. It has also quite a few new interesting characteristics that are attractive to a CTO. When is it production-ready for your next app and how does it stack up against native apps and React Native? Will it survive and thrive over time? And for what type of projects is it suitable already now?

Jan 22, 2020
Since this post was written, I have updated and improved the starter app quite a bit to evolve with the Flutter eco-system best practices and close some gaps I felt was there. There is still lots of value in this article, so I recommend you to read it first, and then move on to the new article describing the updates I have done!

When Flutter first came out of beta, I started playing with it and quickly achieved a lot and had great fun. But I couldn’t find much evidence and code that showed how to bring a large Flutter app into production and maintain it over time. As a CTO, I love playing with new technology, but I also want to understand maturity when it comes to flexibility, maintainability, support for features typically needed for large-scale production applications, and life-time cost vs other alternative eco-systems. I ended up pulling together my own Flutter starter app that includes some of the important production app features, just to see how to do it and what the challenges are. In this post, I present what I did. If the interest is big enough, I may cover specific features of the app in follow-up posts.

The resulting starter app can be found at https://github.com/gregertw/actingweb_firstapp. The README describes how to run it and start playing with it. It does not cover all the things I wanted to include (and that are needed in a production app), but it is a good start. Here’s a quick summary of what the app covers in a running starter app:

Separation of business logic in models and providers, and UI in a separate folder structure
Use of a global UI theme and custom icons for both iOS and Android
Use of scoped_model for app state management
Simple widget framework for handling logged-in, expired, and logged-out states
State management of login and login token using Auth0 including permanent storage for restarts
Localisation using i18n and the Android Studio/IntelliJ flutter i18n plugin to generate boilerplate
Testing using unit test framework and mocking
Use of a plugin that is not published (flutter_auth0)
Use of a OS native capability (location tracking) using a published plugin (geolocator)
Use of Firebase Analytics for usage tracking
Use of Firebase Crashlytics for crash reporting

Obviously, different people value different things, so let me be explicit on what I was trying to achieve. First of all, separation of code related to UI and business logic makes it easier to get good feature velocity as developers can work on the UI and business logic independently. This is not enforced by Flutter, so you need to be careful when building up the app structure. I created a global UI theme (good), created a provider for authentication using a separate folder with code files for business logic for Auth0 authentication (also good), but copied code from geolocator’s example code where widgets and logic were in a single file (bad).

State management is super-important to avoid high complexity, errors in the UI, slow rendering of the app, and the need to re-write state code as the application evolves. I tried out setState, inheritedWidget, BLOC, Redux, and scoped model (overview of state management approaches here). Of these, I evaluated setState and inheritedWidget to be too simple for a larger project, while BLOC and Redux are both viable choices for applications with lots of data. BLOC fits better in with the Flutter language constructs, but if you are used to Redux, that is probably your preferred choice. For this not-so-data-heavy app, I chose scoped model as a simple, but very powerful state library that fits nicely into Flutter thinking.

There are lots of great posts on Flutter widgets and how to build up your app, but very few that shows how to pull together login screens, storing tokens (also permanently), and handling logged in/out state. A mobile app typically needs an access token to connect to backend APIs. I explored using Firebase as an authentication service, but decided to go for Auth0 as it was easier to show a more generic approach to handling logins and access tokens. Using shared_preferences, I could store the access token across app and device restarts.

There are two things everybody knows should be part of any project, but that you typically add too late: testing and localisation. I wanted to test basic unit testing, mocking, as well as UI widget testing. I played with http_server to set up a mock web server (code is still in the test/ folder), however, I managed to do what I wanted with mocking of Auth0. The UI widget testing is still missing (PRs welcome!), but I added simple app state testing, as well as mocking of login and logout.
Localisation is in its basic form a mess in Flutter, with lots of boilerplate. The only viable option I could find was to use an Android Studio/IntelliJ plugin called flutter_i18n. It uses .arb files with translations for each language and auto-generates the boilerplate necessary to inject a localisation class S into the widgets.

One of the concerns about Flutter is how the platform-independent overlay interacts with the iOS and Android platforms to make apps that can run on both platforms. I thus wanted to both use an OS-native capability in my app (device geo location), as well as test out how to integrate code into the app that needs OS-specific code. The flutter-auth0 plugin is not published and the project was somewhat dormant, but had what I needed for Auth0 authentication and authorisation functionality, so I forked the project and used that plugin for Auth0 support. Callbacks are needed for the authn/authz flows, and this is handled with OS-specific code. Also, the geolocator plugin uses OS-specific location code, but is a published and supported library/plugin, so I could then test both.

Finally, I wanted to add analytics and crash reporting capabilities, as well as test out how to integrate with one of the backend-as-a-service offerings. I explored AWS Amplify and Google Firebase, but from a library support, only Firebase had mature enough libraries, so Firebase realistically was the only option (about six months ago). Any production-grade app needs this support, and while I have not actually put this app into production, the integration was simple enough and with Google having bought Fabric, the Crashlytics part “came for free” (though with a bit too complicated configuration).

Overall, I came out with a positive view on the readiness for production for Flutter apps. Not all apps can easily be implemented in Flutter, but if your app does not have fancy and very custom UI requirements and only uses standard OS-level functionality, you can get a LOT of velocity by having a single code base and a single team. Investing in Flutter is still a bet on the future.

On the downside, you still work in two eco-systems in parallel. Upgrading Xcode/iOS, to new libraries (e.g. AndroidX), and build system changes requires an understanding of the eco-systems and preferably Swift/Kotlin(Java). You need to continuously update to the latest technologies and libraries to avoid that you get stuck in incompatibilities between components. The library/plugin system is powerful and you can lock the dependencies, which is a good thing, but you need to investigate the maturity and history of what you choose to dependent on. If not, you may end up stuck with an important component that is not actively maintained. This means that your team should have a decent level of iOS and Android eco-system expertise in addition to Flutter to make sure you navigate well.

Comparing with React Native, it is an eco-system that has more maturity (but less focused direction), but with similar dependencies on the underlying OS. Flutter has quickly become a viable alternative. At this point, the choice is probably less about technology and more about strategy, people, and skills. If you want your frontend developers to develop mobile apps and they are already using React for web, then React Native is a logical choice. If you see mobile apps more in the context of desktop apps/unified web/desktop development, if you have more business logic, and you want UI/UX to be shared across web, mobile apps, and desktop applications, your business logic developers may be more comfortable in a language like Dart that is very similar to Java.
Many technology choices are of such a nature that engineers tend to forget that often the biggest concern is people and competences, not technical capabilities!

There is still one big missing piece in my evaluation: deployment pipeline and lifecycle management. Since I haven’t put the app into production, I have not tested code signing, beta testing, A/B testing, as well as upgrading the app. There is very limited information to be found on these aspects. However, as with the evaluation on AWS Amplify, I suspect that the only real choice for now (unless you like pain) is to go for Firebase and the eco-system around Google. The same holds for app messaging, a feature that I would love to the app, but I haven’t had the chance to do.

Feel free to interact and help me improve the starter app at https://github.com/gregertw/actingweb_firstapp! I would also love feedback if you are using it for your projects and get interesting experiences!

Using Pivotal Assumptions to Increase Velocity

August 29, 2015 by Greger Wedel

Pivotal assumption – “An assumption on business case, priority, minimum viable offer, or requirement that if false will fundamentally change designs, deliverables, or timelines.”

In my post on how organisational factors increase the bad impact of architectural debt, I referred to how the lack of clear business objectives and priorities slow you down and prevent important architectural decisions from being made. The result is that you need to plan along two or more paths in parallel, which obviously slows you down. Sometimes this double work is needed while you are waiting for a decision, and if all outcomes are likely to happen. However, very often one outcome is more likely than the others, it is just difficult to get consensus or a decision has a big impact on something else, so people want to avoid a rash decision.

In these cases, you will get dramatically increased velocity if you can make an assumption about what the decision will be. Teams will get more focus, people will stop doing double work, some things can get lower priority, other things float up to the top. I call this a “pivotal assumption” because a lot hinges on such an assumption, and if you make the wrong assumption, you have lost some time (though often less than you think, because executing on two things in parallel is very inefficient). But what makes a pivotal assumption different from the actual decision?

First of all, you need to make the decision and options explicit and clearly defined, and get people’s acceptance that this is a decision that needs to be made, but that it may take time. You also need to identify the things that do not hinge on the assumption by explicitly stating that “x, y, and z are not impacted by this decision, so we will plan these separately.” You want to do it this way as this will explicitly decouple x,y, and z from your pending decision and will get rid of uncertainty: is x impacted by the decision??

Then, you need somebody in a position to postulate the pivotal assumption. Let’s say you are making a new product for the 10-100 employee segment, and product management is not sure whether import of users from ActiveDirectory is needed in the first version of the product. A senior architect, engineering manager, or senior product owner needs to postulate that “we should execute from here on based on the assumption that ActiveDirectory import will not be needed in this phase.” All discussions and work related to ActiveDirectory import should then be put on hold.

There is one final, crucial element in using pivotal assumption. You need to avoid that it actually becomes the decision by identifying the facts or evaluations that need to happen before a decision can be made and pro-actively follow up. Everybody will be acutely aware that the longer you execute based on the assumption, the more costly it will be to make a decision that is not the same as the assumption. This is an incentive to get a conclusion. Also, I recommend that you actively look for indications that the pivotal assumption may be wrong, and that when it becomes clear that the pivotal assumption was wrong, you swiftly change course and treat throw-away work as sunk cost.

If you can systematically identify areas where people have different assumptions when doing their work, you can make explicit pivotal assumptions to get temporary focus, and then quickly close the identified decisions. This will dramatically increase your velocity. You may think that you get a lot of throwaway work, but truth is that without pivotal assumptions, you get double work, which has the same cost, but in addition you get also the unnecessary code into your codebase, increasing complexity and cost of maintenance. The psychological effect of not having clarity also adds to the cost of executing on two tracks in parallel.

I call this method “hypothesis-driven development” as you continuously identify assumptions, you gather facts and try to falsify your hypothesis. The organisation can get pretty good at identifying good assumptions and quickly close them. Be warned though, unless you pro-actively gather facts to try to falsify your pivotal assumptions, you end up adopting the assumptions as your decisions.

The Changing Development Organisation: The Software Architect

July 20, 2015 by Greger Wedel

(intro to this post on devops)

It has been a busy few months. Since my last post I have transitioned to a new role as part of day to day development (see linkedin.com/in/greger for more details) in the Collaboration Infrastructure Technology Group (UC, telephony, telepresence, conferencing, etc) in Cisco. Earlier, as part of Office of the CTO, I influenced the medium to long-term technology and architectural strategy, but as I had limited involvement in the day to day operations, it was a challenge to develop architecture that bridged near-term goals with longer-term strategy. Now, I’m less involved in long-term technology strategy, but on a daily and weekly basis involved in designs, decisions, and priorities made as part of development. Quite refreshing and comes with a bunch of interesting challenges!

If you have followed my blog, you have probably understood that I see architecture as something tightly embedded in day to day development, but more than just the sum of tactical, short-term steps. Also, you have probably detected my desire to see product management team and architects partner and work on the same thing: identify the whats and hows of building great products and services. Agile has for a long time changed my organisation and the last year or so, devops processes have really started to impact how we work. We have hardware projects, traditional waterfall product releases, and cloud services. This means that in my technology group we are currently using various flavours of waterfall, agile, and devops processes in parallel. It puts a strain on coordination and especially challenges existing roles in the teams. This post is about some of these changes.

Classic product organisations used more or less waterfall processes, and many so-called agile development organisations are using agile terminology to execute on a mostly waterfall mindset. However, with an increased focus on devops processes, the combined product/service organisation is facing another set of challenges in combining everything from hardware products and their software loads through legacy, monolithic, hosted applications to devops microservices. Tying these together to deliver robust, scalable services that offer great user experiences is a challenge along many dimensions: culture, people, competence, programming languages, tools, project management, goal setting, measurements, management, and roles.

One particular challenge is that not only does what we do change, our roles have to change as well. This is easy in a small organisation, but in a big organisation, the expectations other people have to what you are supposed to do is very important to your ability to get things done. So, let’s say I’m used to architects writing specs, then I will likely continue to ask for specs. Other people may not think an architect should write specs anymore, and they will ask for other things and maybe ignore the specs. This creates frustration and tension.

I believe most people in the industry are starting to understand the fundamentals of how devops are changing the roles of the development team and the operations team, and that they basically need to become one integrated team (with some new roles and new definitions of existing roles). However, the role of the product manager is also changing in the process, as is the project manager’s role, the engineering director’s, as well as marketing, sales, support, and partners. They no longer have releases to plan around and need to change how they work. I may cover some of these other roles in later posts, but in this post I will concentrate on the architect’s role.

First of all, a few words on what an architect is. In my understanding, an architect is a developer, a member of the engineering organisation, with a special responsibility for understanding the short-term and long-term business and technical requirements and making sure that the right technical deliverable is delivered at the right time with the right quality. Sometimes this is to get it done and deliver something less ideal from a code point of view. Sometimes this is to tell the product owner that some labour-intensive refactoring needs to be done in order to deliver a feature. My personal view is that a good architect also truly understands the value of the product to the end users and customers, and that he or she is a partner to the product manager or owner in defining and evolving the product. An architect can be a senior member of a scrum team (in Cisco, these are called technical leaders), or a more dedicated role that covers multiple scrum teams (senior technical leaders or principal engineers). In larger organisations, system architects typically span multiple products or services to create functioning solutions. In my view, regardless of scope, to be effective, the architects should be embedded in the engineering organisation. Some architects lead development teams, others have a more standalone role. However, the most effective teams seem to always three roles covered when decisions are made: the product owner (representing the vision and the problems to be solved by the product), the architect (representing how to deliver with the technical integrity of the product in mind), and the development manager (representing the developers, and making the trade-off decisions).

The architect is thus a member of the product leadership team. And with agile and devops, this role becomes even more challenging with the need to engage in more areas. Here are the some of the main areas of challenges:

Software architecture and code structure: There are two big challenges I want to mention here. First of all, devops requires a modularisation into micro-services, and getting the right grouping of functionality and defining clean, scalable APIs are critical. Second, the whole thinking of how you develop a feature or a set of features need to change. Every time you commit code, you cannot break anything. This influences how you architect the feature, how you test it, and you need to think in terms of A/B testing, feature toggles, and metrics, metrics, metrics(!), and not in terms of feature branches and unit and integration testing.
Software development processes: An architect needs to engage in code repository management, branching strategy, build ande delivery pipeline, test automation, A/B testing, feature toggles, 360 reviews, and much, much more. The architect needs to police the process and make sure that scrum teams working on various micro-services have the same view on the process and don’t break the builds. Very few other people will have the code knowledge and competence required to identify where the issues are and address them. The architect becomes critical to making sure that the devops machinery keeps working.
Requirements and product management: A shift from waterfall to devops is a shift in power from product management to the engineering organisation. Where product management used to ask for features based on market knowledge and research, a high speed of introducing features, doing A/B testing, and measuring the impact and quality relegates all product managers (except those who are very technical) to focusing on the go to market aspects of the service, not the requirements. The architect’s role in this becomes critical as the architects are instrumental in defining what to measure, and what you measure, you improve (whether this power shift has to happen, is a separate topic…)
Project management: Traditional project (or program) management should totally disappear. The software development process, the build pipeline, the metrics, the scrum standups, the scrum of scrum meetings, kanban boards, and the definition and usage of the APIs across the services, together with the automated tests together define and drive the project. The traditional tools of a project manager, like Gantt charts, dependency tracking, weekly project status meetings, read-outs to executives, escalations, and decision meetings based on analysis, are of absolutely no use in a properly executed devops process (note that badly executed devops processes will give ample room for such old tools and processes…). The architect needs to engage in the overall execution of the devops project to ensure that there is a constant evolvement of proper architectural frameworks, guidelines, automation, gates, micro-service designs, API/protocol designs, and much more.
Architecture: Devops does not mean architecture work is not necessary. On the contrary, as the architecture is constantly and organically evolving, the architecture that emerges can either be a disaster or a robust construction. However, the “ivory tower” architect is history as the emerging architecture emerges both with and without a plan, and it can only be shaped by having constant touch with what is being developed. A spec or design document will be obsolete within a week.
Management: I see a lot of very competent, senior architects who are brilliant at designing a flow or a new protocol, but who are lacking in managerial skills. Running efficient meetings with action items, pro et contra evaluations, closing decisions, setting deadlines and targets, chasing deliverables, and driving consensus are just examples of typical managerial skills that becomes more important for an architect.

There is so much more to how devops is changing an architect’s work, as always, the devil is in the details, but these categories have been helpful for me in understanding some of the challenges in evolving architects towards “devops ready.” The shift is similar already with agile development processes, but becomes more visible with devops.

Do you believe there are other important things that impact the architect’s role in devops? I’m happy to hear from you!

My next post covers the importance of one shared backlog backlog with all the actual work items visible, as well as the product owner’s role. Not directly focused on devops only, but relevant to devops as a good backlog grooming process is essential to getting anywhere.