In previous post I reviewed Pat Helland analysis on transaction processing in distributed scalable environment as outlined in his document "Life beyond Distributed Transactions: an Apostate’s Opinion"
We touched about the implicit contradiction between scalability and distributed transactions and suggested a new model which IMO becomes the third generation of TP systems.
Massimo Pezzini from Gartner named that third generation TP eXtreme Transaction Processing or XTP. Both referred to this model as an emerging trend. Beyond those two analysis I'm aware of other research in this area claiming pretty much the same thing.
In this post I'll focus on how GigaSpaces XAP and SBA implements the principles as was laid out in the previous post:
- Create/rearrange your object model using entities (objects with IDs), in a such a way that most of your state changes will happen inside the boundaries of a single entity. a change to a single entity is atomic.
In an SBA model this special case of objects is referred to as Tuples. The Tuple model is very different from a Tree model that other caching systems supports since a tuple doesn't hold any distributed reference to other objects. Holding references to other objects would end up with tight coupling between those objects. This tight coupling will eventually determine the locking/transaction boundaries and therefore the scaling granularity.
- For those changes that exceed the boundaries of one entity, changes will not be atomic and the final desired change will be achieved using a sequence of atomic changes, every change request (activity) MUST be implemented to be idempotent because of the chance of retransmitting due to failures.
In an SBA world this is achieved through Master/Worker pattern which if you would like is a distributed command pattern. In this model transactions are executed through a chain of commands - the submitter is responsible for triggering the first command which start a chain of events (workflow) between a set of workers each responsible for s specific step in the transaction flow(validation, matching, routing..), a failure in one of those steps lead to a rollback of the transaction associated with that step. At this stage another worker will be able to pick it up and continue the processing of the failed transaction from the same point it failed. Unlike the current transaction model the submitter only blocks till the first command is committed, the rest happens in the background. The later is probably the key difference between today's model in which the submitter blocks till the entire transaction with all its various resources is completed. Another difference with XTP is that the submitter remains loosely coupled from the other transaction participants. This is another key aspect that makes SBA much more aligned with SOA/EDA and Grid type of architecture.
Executing transaction that span between multiple entities/tuples:
In real life situations there are situations where a single transaction need to reference multiple data items under the same transaction scope. This could potentially lead to huge overhead. To minimize that overhead we often partition the data based on affinity in away that each partition will become a self sufficient unit. The affinity key is the common key that is used to associate between a set of entities/tuples. A typlical example is csutomer-id, protfolio etc. In this case transaction scope could span across multiple data items but wouldn’t need to exceed a partition scope.
- To accomplish idem-potency of your activities (actions/methods) the entity itself SHOULD hold a history of committed activities per source(entity).
This steps assumes that there is no hot backup that can continue the processing of a transaction from the exact point where it failed. It is therefore assumed that the way to overcome this limitation is re-issue the transaction in case of a failure. This can obviously lead to scenario where the same transaction or part of that transaction could be processed more then once hence why idempotency handling is required. In SBA architecture we assume that there is a hot (in-memory) backup which can continue the processing of a transaction from the exact point it failed. In this way we minimize the cases in which you would need to deal with idem-potency since we will deal with it implicitly. Obviously this is can be achieved as long as the entire state associated with a transaction resides within the space.
The forth rule:
So far we discussed the common principles which lead to XTP as Pat Helland and Massimo Pezzini layed out. At this stage I would like to introduce a forth principle: Minimize the amount of moving parts associated with a business transaction:
In the current Tier Based Approach we build our system from different set of building blocks that are separated into tiers such as messaging-tier, business-tier, data-tier etc. Often the implementation of those tiers is done in cylos i.e. each tier maintain its own clustering, high availability model and often even data model. While I would agree that it make sense to keep those different components logically separated it is also pretty evident that XTP implies strong runtime dependency between those units. For example - the messaging system need to be aware of how we partition the data in order that transactions will be routed to the appropriate partition, from a latency perspective any network hop is going to effect the latency with direct proportion to the amount of hops. When we apply high availability at each tier separately we by definition add redundant network hops and disk I/O since each system maintain its own high availability either through replication or through persisting the data into the disk. In SBA approach we address the contradiction between the loose coupling required at the application Service level and tight coupling at the runtime level through a concept that has been widely used in distributed technology - virtualization. In our specific scenario we separate between the API and the runtime implementation of that API. We create a common runtime model which address the clustering, high availability, load balancing, fail-over semantics across all tiers and then create API façade that expose the concrete set of functionality i.e. messaging/data-sharing/parallel-processing to the application level on top of that common runtime. In this way we can achieve minimum latency since we eliminate the cylos and redundant network hop associated with them, In addition to that we maintain the workflow associated with a business transaction in-memory under a context of a single partition and therefore reduce all the network overhead associated with it. High availability of the entire application is maintain without writing to disk through replication etc.
Another method in which we can reduce the amount of moving parts with SBA is through a new component that is refer to as processing-unit. A processing-unit is built as an application container (In our case it extends the Spring container). It contains all the individual component/services that have tight runtime(latency, fail-over, scaling) dependency. As such a processing-unit defines the unit of scale and failover of our application. In SBA the transaction is scoped within the boundaries of a processing-unit. The fact that all the components involved in a transaction are collocated eliminate the chances for partial failure and therefore the need for handling distributed transactions.
Summary:
Through the set of two last posts I tried to provide a summary of the rational behind the third generation Transaction Processing (TP model) referred to as XTP - eXtreme Transaction Processing . I later tried to outline based on Pat Helland analysis (Amazon), and Massimo Pazini (Gartner) the core principles behind XTP. In this post I described how Space Based Architecture SBA follows and extend those principles in a more concrete aspects.
At this stage I would like to summarize what is see as the common principles for achieving XTP :
- Rule #1 - Build your data model from a set of independent data items (Tuples)
- Rule #2 - Partition your different data items based on your data affinity (Customer-id, account-id,..)
- Rule #3 - Execute transaction as a set of individual steps scoped within a single partition.
- Rule #4 – Minimize the amount of moving parts involved under the context of a transaction
A full working example that illustrate how this pattern works using Spring and OpenSpaces abstraction is provided here.
This example illustrates a typical scenario of an Order Management System (OMS) where orders submitted to the system becomes a command that triggers a sequence of POJO Driven Bean each dealing with a specific state of that transaction (Parsing/Validation -> Execution/Matching -> Routing).
Julian Browne has previously posted success story of a real production Order Management Systems based on this model.
In The third part of this series of posts I'll try to suggest a simple migration path between the current J2EE transaction model and XTP.
