Distributed Transactions are a common pattern used to ensure ACID properties are met between distributed resources. Since the late 70s, the fIrst generation of this model has been widely used in many large-scale applications that struggle with the difficulties of multiplexing many online terminals. This led to the emergence of the 1st generation TP Monitors (TPMs), such as Tuxedo (Now owned by BEA). The emergence of web based applications in the late 90s drove the creation of 2nd generation TPMs, in the form of JEE application servers, to address similar needs using more open and modern technologies as described in the following article by Subbu Allamaraju: Nuts and Bolts of Transaction Processing. The diagram below illustrates a typical transaction flow in a JEE environment:
Transactions in EJB Application Server
(Source: Subbu Allamaraju, Nuts and Bolts of Transaction Processing)
The increased business demand for greater scalability led many to the realization that the current transaction model is a bottleneck due to its inherit centralized approach. It also makes our systems quite brittle and complex due to the tight coupling that it introduces, as described in a paper by Pat Helland of Amazon.com: Life beyond Distributed Transactions: an Apostate’s Opinion:
Today, we see new design pressures foisted onto programmers that simply want to solve business problems. Their realities are taking them into a world of almost-infinite scaling and forcing them into design problems largely unrelated to the real business at hand. Unfortunately, programmers striving to solve business goals like eCommerce, supply-chain-management, financial, and health-care applications increasingly need to think about scaling without distributed transactions. They do this because attempts to use distributed transactions are too fragile and perform poorly.
This challenge leads us to the emergence of a third generation of TPMs, or what Gartner calls Extreme Transaction Processing (XTP).
What approaches are people taking today to overcome the limitations of previous generations? From Pat:
"..Because the performance costs and fragility make distributed transaction impractical. Natural selection kicks in,... applications are built using different techniques which do not provide the same transactional guarantees but still meet the needs of their businesses"
So the question is how to achieve high transactional throughput and performance without compromising consistency and reliability?
Pat Helland suggest the following principles:
- Instead of global transactional serializability we assume multiple disjoint scopes of transactional serializability
- Scalable Apps Use Uniquely Identified “Entities” [Nati: Entities represent data elements that are equivalent to Tuples in Space-Based Architecture terminology]
- Atomic Transactions Cannot Span Entities
- From the programmer’s perspective, the uniquely identified entity is the scope of serializability
- Messages Are Addressed to Entities [Nati: in other words, we use messages to manage the workflow between the disjointed transactions - this is pretty much the equivalent of a transaction coordinator].
- Entities Manage Per-Partner State (“Activities”) [Nati: in GigaSpaces terminology: each activity is executed within a collocated partition].
In order to avoid distributed transactions you must:
- Create/rearrange your object model using entities (objects with IDs), in a such a way that most of your state changes will happen inside the boundaries of a single entity. a change to a single entity is atomic.
- For those changes that exceed the boundaries of one entity, changes will not be atomic and the final desired change will be achieved using a sequence of atomic changes, every change request (activity) MUST be implemented to be idempotent because of the chance of retransmitting due to failures.
- To accomplish idem-potency of your activities (actions/methods) the entity itself SHOULD hold a history of committed activities per source(entity).
Combining these three principles allows you to safely change the state of the application without distributed transactions, while allowing for almost infinite scaling.
How does this apply to SOA?
A core principle in SOA, event-driven architecture (EDA) and grid computing is the notion of loose coupling of services. The distributed transaction model leads to tight coupling by definition, as it requires intimate integration and runtime dependencies among all of the services that are associated with a transaction. The suggested model of "disjoint transactions" fits in nicely with this type of new architecture. Here's Pat again:
Everything discussed in this paper is supportive of SOA. Most SOA implementations embrace independent transaction scopes across services. The major enhancement to SOA presented here is the notion that each service may confront almost-infinite scaling within itself and some observations about what that means. These observations apply across services in a SOA and within those individual services where they are designed to independently scale.
Pat ends with:
In a few years we are likely to see the development of new middleware or platforms which provide automated management of these applications and eliminate the scaling challenges for applications developed within a stylized programming paradigm. This is strongly parallel to the emergence of TP-monitors in the 1970s.
Guess what, Pat, the future is now!
In a the next post i'll discuss how Space Based Architecture (SBA) and GigaSpaces XAP as
the implementation of SBA follow similar principles to those described by Pat Helland in a very elegant way that fits perfectly with SOA, EDA and Grid.
Additional References:
- Shay Banon on Scalability Mythbusters
- InfoQ discussion on New patterns and middleware architecture needed for true linear scalability?
