Why microservices might be finished as monoliths return with a vengeance

The shift towards microservices started gaining momentum in the early 2010s, as tech companies recognized the limitations of monolithic architectures. However, many companies such as Amazon (Prime Video), Invision, Istio and Segment are moving back to monolithic architectures. This article will explore why many organizations fail when transitioning to a microservices architecture.

What is a monolith?

A monolithic architecture is straightforward: The user requests data and all business logic and data reside within a single service. However, monolithic systems face challenges, such as limited scalability, difficulty with deploying updates and a vulnerability to single points of failure.

To address this, many organizations have attempted to transition to a microservices-based architecture to leverage advantages such as abstraction and encapsulation, faster deployment, easier maintenance and closer alignment of each service with team ownership.

Why microservices?

In an ideal microservices architecture, each business domain operates as its own independent service with its own database. This setup offers benefits like better scalability, flexibility and resilience. Consider the diagram below.

The reality

However, recent trends show that many companies are moving away from this and sticking to a monolithic architecture. This is because it is difficult to achieve this level of harmony in the real world. The reality often looks like the diagram below.

Migrating to a microservice architecture has been known to cause complex interactions between services, circular calls, data integrity issues and, to be honest, it is almost impossible to get rid of the monolith completely. Let’s discuss why some of these issues occur once migrated to the microservices architecture.

Incorrect domain boundaries

In an ideal scenario, a single service should encapsulate one or more complete business domains so that each domain is self-contained within a service. A domain should never be split across multiple services, as this can lead to interdependence between services. The following diagram shows how a single service can contain one or more entire domains to maintain clear boundaries.

In complex real-world systems, defining domain boundaries can be challenging, especially when data has traditionally been conceptualized in a specific way. The following diagram shows how real-world systems often look in a microservice architecture when boundaries are not defined in advance or engineers add new services without considering domain boundaries.

If domains are not well-defined, the dependency on other services increases, which leads to multiple issues:

• Circular dependencies or excessive calls: When services are interdependent, they require frequent data exchanges.
• Data integrity issues: A single domain split across services causes deeply coupled data to be split across multiple services. 
• Vague team ownership: Multiple teams may need to collaborate on overlapping domains, leading to inefficiencies and confusion.

Deeply coupled data and functionality

In a monolithic architecture, clients often skip designated interfaces and access the database directly because enforcing encapsulation is hard in a single codebase. This can lead developers to take shortcuts, especially if interfaces are unclear or seem complicated. Over time, this creates a web of clients tightly connected to specific database tables and business logic.

When moving to a microservices architecture, each client needs to be updated to work with the new service APIs. However, because clients are so tied to the monolith’s business logic, this requires refactoring their logic during the migration.

Untangling these dependencies without breaking existing functionality takes time. Some client updates are often delayed due to the work’s complexity, leaving some clients still using the monolith database after migration. To avoid this, engineers may create new data models in a new service but keep existing models in the monolith. When models are deeply linked, this leads to data and functions split between services, causing multiple inter-service calls and data integrity issues.

Data migration

Data migration is one of the most complex and risky elements of moving to microservices. It is essential to accurately and completely transfer all relevant data to the new microservices. Many migrations stop at this stage because of the complexity, but successful data migration is key to realizing the benefits of microservices. Common challenges include:

• Data integrity and consistency: Errors during migration can lead to data loss or inconsistencies.
• Data volume: Transferring large amounts of data can be resource-heavy and time-consuming.
• Downtime and business continuity: Data migration can require downtime, potentially disrupting business operations. A smooth transition with minimal user impact is crucial.
• Testing and validation: Rigorous testing is needed to ensure migrated data is accurate, complete, and performs well in the new service.

Conclusion

The microservices architecture may look appealing, but transitioning from a monolith is challenging. Many companies find themselves stuck in a midway state, which increases system complexity causing data integrity issues, circular dependencies and unclear team ownership. The inability to utilize the full benefits of microservices in the real world is why many companies are returning to a monolithic approach.

Supriya Lal is the group tech lead for the commerce platform organization at Yelp.