Why would I want to replace my web app with a Micro-Frontend in 2021?
Here at Houghton Mifflin Harcourt, our UI development teams have been prototyping and building out a Micro-frontend and Monorepo solution to replace our current application suite over the past two years.
My colleagues Aislinn, Attila and Cathal have covered each step of the journey, from switching to a Monorepo to migrating our apps and all the travails in-between.
It has been a long, eventful journey and, now that it is going live to our customers and other development teams are interested in coming on board, I think it is time to order my thoughts on the subject and explain the decisions we made on the way.
But first a brief history to give some context to why we made the decisions we made.
What we do and who we make web apps for
HMH is an education company that provides online tools, content and workflows for both teachers and students. Teachers can plan their lessons, grade homework and manage their classes. Students can read their coursework and complete assignments.
Typically the year starts with administrators creating their students and classes, teachers build their lesson plans and finally students come in and begin their coursework.
This is the 3rd generation UI application that HMH have released since I joined the company; each new generation was built with knowledge we learned from the previous one, but also to a degree in reaction to the shortcomings of the previous application.
- Backbone and Underscore
Our first single-page application felt really ground-breaking to work on, a pure Javascript application, completely decoupled from our services. Backbone was a pleasure to work with and we could pick and choose between component libraries, build tools and testing frameworks.
I particularly liked the independent libraries and responsibilities; being able to switch how we managed states and views meant that we could take advantages of new technologies. Just substituting Underscore with Lodash in the model layer gave us a massive performance boost! In retrospect, this was the most important lesson I took away from this generation of our UI.
Freed from waiting on services and infrastructure development, we very quickly had a fully-featured application to deliver to our customers.
So far, so good.
We added a couple of new developers and they started building on the initial work. Although it went well, I noticed that new features were written in wildly different idioms, facilitated by Backbone which endearingly has a very relaxed attitude to how you can use it.
Different features demand different approaches, and this diversity produces the best result when the teams are both experienced and coherent. However at scale, with a mix of junior and senior developers, a lack of common patterns can lead to serious problems. Starting with a flexible framework like Backbone can quickly lead to unstructured and unmaintainable code.
2. Angular and Grunt
Now that we had successfully demonstrated that building a client-side application was viable, the business decided to follow this model. We quickly hired teams of new developers and began to develop the second generation UI.
It was obvious from the beginning that teams would have to work independently and that we needed patterns in place to help new hires and junior developers. We decided on AngularJS over Backbone; it had a full set of tools for everything from routing, views and libraries to testing. It came with excellent documentation and a big community.
Very soon we had rebuilt all the workflows from the previous application and added many more. However problems were emerging; teams were jostling to merge code for deadlines. Test failures were causing deployments to back up as teams rushed their code through the pipeline without cleaning up.
Something needed to change.
The solution was to split the application into multiple smaller applications and to use templates to integrate them into one big application at build time.
This allowed teams to keep their code in their own repo, deploy at will to test environments for review and then to merge everything together for a release. I was pretty pleased with the results but it didn’t solve every problem; individual apps depended on commit tags for versioning and testing was difficult.
Something else bothered me as well. Engineers would suggest new libraries or approaches and would be shot down because everything had to be written in Angular. New technologies were emerging and we were watching from the sidelines.
Finally the bombshell dropped! We had been tracking the beta candidates for Angular 2 and waiting for the new router that would bridge the versions, but suddenly Google announced that AngularJS would be dropped without a migration route.
This made for a pretty awkward conversation with my manager, concerning this huge application we had developed which was suddenly defunct. I quietly swore to myself that I would never choose an opinionated framework like Angular again, assuming I got the chance.
3. React and Webpack
We were happy to carry Grunt and Protractor over from the last project so at this point we decided our options were either the new Angular or React. We wrote some sample applications and decided to go with React, which allowed us to pick our own libraries for managing state and REST APIs.
At this point I ended up working on another project but I followed the development of the new React application with great interest. The team were highly skilled and motivated, pretty soon they had a beautiful application framework, documentation and component libraries.
Unfortunately, they found themselves in a situation where they had to track a moving target. Every sprint that they added a new feature, the many teams working on the older application would add three more to theirs. The release date slipped further and further away.
Eventually the situation came to a head and we made a decision to fold the new application into the existing one. We tested a library that allowed React components to be bound to the Angular framework and we were able to merge the two.
The good news was that all new features would be developed in React, much to the relief of all developers. On the other hand we now had a bloated Javascript payload and a lot of technical debt.
So what did we learn?
- Opinionated frameworks are great for rapid development and scaled-up teams, but you are now tied to the lifespan of that framework
- The best way for teams to be effective in a large application is to decouple the features in the application and allow for independent deployments.
- Nobody likes taking features away from customers. If you want to develop a new application, it has to be able to coexist with the existing one.
So what do Micro-Frontends offer?
We found ourselves in a situation where we had many small React applications living in a very mature Angular container. It seemed improbable but we had managed to make everything work together.
This is when we first came across the concept of a Micro-Frontend, in particular single-spa. It didn’t seem conceptually all that different from what we had arrived at in our own time but it offered some tantalising possibilities:
Isolating frameworks
In a rapidly-changing ecosystem like Javascript on the browser, you need to be able to slowly but continuously evolve your applications. Frameworks inhibit innovation. We have a list of approved technologies in HMH and we didn’t want to make front-end frameworks part of this.
Single-spa offered the capability of isolating our existing applications. They provide handy wrappers for the most popular frameworks, including our Angular and React versions. They also tested with forward-looking technologies like NextJs. No longer would our developers have to watch new technologies come and go.
Independent applications
Not only could you load independent applications into the Single SPA container, but using SystemJs and/or Webpack, you could split them into independently-hosted chunks and demand load them as the user journeyed around the application. Applications could even be built and deployed separately, only to be loaded together at runtime.
Common Coding Standards
This really comes into its own when combined with a Monorepo for all your apps, nevertheless it’s easy to lay down patterns around inter-app communication and provide templates and examples to onboard new developers.
Great! Where do I sign up?
We tentatively built some test applications in single-spa and then ported some of our legacy apps. Everything worked just as we had hoped. Slowly the project ramped up and we added capabilities like authentication, testing, localisation.
Once the project was stable and documented, we took it to the teams. Soon we had our existing features living alongside our new applications. To get the most out of the Micro-frontend and reduce the number of pull requests necessary to complete a story, we adopted a Monorepo in tandem with our new applications.
This month we will be shipping the Micro-frontend applications to users! It has been a long journey but very rewarding. Personally, I’m very excited about the possibilities this new architecture opens for us.
What did we learn about Microfrontends?
Every new technology comes with advantages and disadvantages. Here are some of the issues we have worked through over the past year or so.
Cross-cutting changes
As with micro-services, cross-cutting changes are difficult to deal with. We tried to mitigate this by setting up as many patterns as possible at the beginning; capabilities like localisation, accessibility and responsiveness are baked into our framework.
We have also accepted that changes have to roll out on a feature-by-feature basis. Each team undertakes to add the new changes when they can and the users may experience some discontinuities between applications in the short term. This can be effectively messaged by Product and Support in most instances.
Duplication of Code
Team autonomy can mean that sometimes teams need to have their own copy or version of what would normally be shared libraries. Single-spa facilitates the use of shared libraries with import maps or using the new federated modules feature of Webpack 5. We keep an eye on the larger libraries and track divergence.
Again, we consider this one of the costs of the Micro-frontend model. It’s worth it for teams to be able to build and deploy their own applications. Also, breaking the application up into chunks means that the user is unlikely to bear the brunt of this code duplication in a single session.
No shared state
This is an unnatural situation for anyone used to developing within a framework but, other than a user token at boot time, we have no shared state between applications. Applications coupled by state become monoliths, which is exactly what we are trying to avoid in the first place.
The developers of single-spa suggest that if two microfrontends are frequently passing state between each other, consider merging them. The disadvantages of microfrontends are enhanced when your microfrontends are not isolated modules.
There are a couple of recommended solutions here, from RxJs to using an Event Bus. Instead we have kept it simple; using caching, eventing and url parameters to loosely communicate between apps.
A lot of what is considered state in a front-end application is often just structured data from an endpoint, and this can easily be shared using a client-side cache.
We use LocalForage and, as we build out our mobile app capabilities, we may switch to using a Service Worker and local DB for our Rest APIs. For GraphQl data sources, the Apollo Client has good caching that can be also be persisted with LocalForage.
Scenarios like logout or ‘save unsaved work’ can be handled by a combination of the single-spa events and our own custom events. Other applications that are part of a workflow can often just append url parameters to convey the state of the workflow.
Deployment of apps can lead to untested combinations, caching issues
If your team is used to bundling and minifying applications, you may find that you have issues with deploying multiple code chunks for the same application. Caching issues can be tricky to diagnose as different versions of the code interact unpredictably.
Likewise, testing can be difficult if applications are deployed to production using different pipelines. This is no different to ‘testing in production’ for micro-service architectures. We have minimised our exposure to this problem so far but we are looking at enabling individual deployments with the use of feature flags, a code registry and multiple test environments.
Dependency management
Updates to the micro-frontend or the component framework can affect multiple applications. Very quickly it becomes difficult to execute all the test suites for any given change. Each application is also likely to have a very similar list of dependencies.
We have begun resolving this outside of the Micro-frontend, using a combination of Yarn Workspaces, Lerna and a Monorepo, however this is a subject for a completely different article.
So … why would I want to replace my web app with a Micro-Frontend in 2021?
With the obvious caveats that not every project is the same and not every team has the same priorities, these are the reasons why I would choose a Micro-frontend like single-spa:
- Your application is large and long-lived. If you have an application with a lifespan of more than two years, you will be rewriting it at some point to keep up with underlying platform and technology changes. And if that application is too large to rewrite in one go, then you have a serious problem.
- You have many teams working on your application, perhaps in different time zones with their own release cycles. A Micro-frontend will allow those teams to work and deploy independently.
- Your application is large enough that the Javascript has to be chunked and served in pieces based on the role of the user and their workflow. There are many ways to do this, but a Micro-Frontend is a great way of coordinating this effort.