Synopsis

This document provides a pattern for doing multi-document updates or “multi-document transactions” using a two-phase commit approach for writing data to multiple documents. Additionally, you can extend this process to provide a  functionality.

Background

Operations on a single  are always atomic with MongoDB databases; however, operations that involve multiple documents, which are often referred to as “multi-document transactions”, are not atomic. Since documents can be fairly complex and contain multiple “nested” documents, single-document atomicity provides the necessary support for many practical use cases.

Despite the power of single-document atomic operations, there are cases that require multi-document transactions. When executing a transaction composed of sequential operations, certain issues arise, such as:

• Atomicity: if one operation fails, the previous operation within the transaction must “rollback” to the previous state (i.e. the “nothing,” in “all or nothing”).
• Consistency: if a major failure (i.e. network, hardware) interrupts the transaction, the database must be able to recover a consistent state.

For situations that require multi-document transactions, you can implement two-phase commit in your application to provide support for these kinds of multi-document updates. Using two-phase commit ensures that data is consistent and, in case of an error, the state that preceded the transaction is. During the procedure, however, documents can represent pending data and states.

Pattern

db.accounts.insert(   [     {
       _id: "A", balance: 1000, pendingTransactions: [] },     {
       _id: "B", balance: 1000, pendingTransactions: [] }   ])

db.transactions.insert(    {
       _id: 1, source: "A", destination: "B", value: 100, state: "initial", lastModified: new Date() })

var t = db.transactions.findOne( {
         state: "initial" } )

{
         "_id" : 1, "source" : "A", "destination" : "B", "value" : 100, "state" : "initial", "lastModified" : ISODate("2014-07-11T20:39:26.345Z") }

db.transactions.update(    {
         _id: t._id, state: "initial" },    {
              $set: {
         state: "pending" },      $currentDate: {
         lastModified: true }    })

db.accounts.update(   {
         _id: t.source, pendingTransactions: {
         $ne: t._id } },   {
         $inc: {
         balance: -t.value }, $push: {
         pendingTransactions: t._id } })

db.accounts.update(   {
         _id: t.destination, pendingTransactions: {
         $ne: t._id } },   {
         $inc: {
         balance: t.value }, $push: {
         pendingTransactions: t._id } })

db.transactions.update(   {
         _id: t._id, state: "pending" },   {
             $set: {
         state: "applied" },     $currentDate: {
         lastModified: true }   })

db.accounts.update(   {
         _id: t.source, pendingTransactions: t._id },   {
         $pull: {
         pendingTransactions: t._id } })

db.accounts.update(   {
         _id: t.destination, pendingTransactions: t._id },   {
         $pull: {
         pendingTransactions: t._id } })

db.transactions.update(   {
         _id: t._id, state: "applied" },   {
             $set: {
         state: "done" },     $currentDate: {
         lastModified: true }   })

Recovering from Failure Scenarios

The most important part of the transaction procedure is not the prototypical example above, but rather the possibility for recovering from the various failure scenarios when transactions do not complete successfully. This section presents an overview of possible failures and provides steps to recover from these kinds of events.

var dateThreshold = new Date();dateThreshold.setMinutes(dateThreshold.getMinutes() - 30);var t = db.transactions.findOne( {
        state: "pending", lastModified: {
        $lt: dateThreshold } } );

var dateThreshold = new Date();dateThreshold.setMinutes(dateThreshold.getMinutes() - 30);var t = db.transactions.findOne( {
        state: "applied", lastModified: {
        $lt: dateThreshold } } );

db.transactions.update(   {
         _id: t._id, state: "pending" },   {
             $set: {
         state: "canceling" },     $currentDate: {
         lastModified: true }   })

db.accounts.update(   {
         _id: t.destination, pendingTransactions: t._id },   {
             $inc: {
         balance: -t.value },     $pull: {
         pendingTransactions: t._id }   })

db.accounts.update(   {
         _id: t.source, pendingTransactions: t._id },   {
             $inc: {
         balance: t.value},     $pull: {
         pendingTransactions: t._id }   })

db.transactions.update(   {
         _id: t._id, state: "canceling" },   {
             $set: {
         state: "cancelled" },     $currentDate: {
         lastModified: true }   })

Multiple Applications

Transactions exist, in part, so that multiple applications can create and run operations concurrently without causing data inconsistency or conflicts. In our procedure, to update or retrieve the transaction document, the update conditions include a condition on the state field to prevent reapplication of the transaction by multiple applications.

For example, applications App1 and App2 both grab the same transaction, which is in the initialstate. App1 applies the whole transaction before App2 starts. When App2 attempts to perform the “” step, the update condition, which includes the state:"initial" criterion, will not match any document, and the  and  will be 0. This should signal to App2 to go back to the first step to restart the procedure with a different transaction.

When multiple applications are running, it is crucial that only one application can handle a given transaction at any point in time. As such, in addition including the expected state of the transaction in the update condition, you can also create a marker in the transaction document itself to identify the application that is handling the transaction. Use  method to modify the transaction and get it back in one step:

t = db.transactions.findAndModify(       {
              query: {
      state: "initial", application: {
      $exists: false } },         update:           {
                  $set: {
      state: "pending", application: "App1" },             $currentDate: {
      lastModified: true }           },         new: true       }    )

Amend the transaction operations to ensure that only applications that match the identifier in theapplication field apply the transaction.

If the application App1 fails during transaction execution, you can use the , but applications should ensure that they “own” the transaction before applying the transaction. For example to find and resume the pending job, use a query that resembles the following:

var dateThreshold = new Date();dateThreshold.setMinutes(dateThreshold.getMinutes() - 30);db.transactions.find(   {
          application: "App1",     state: "pending",     lastModified: {
      $lt: dateThreshold }   })

Using Two-Phase Commits in Production Applications

The example transaction above is intentionally simple. For example, it assumes that it is always possible to roll back operations to an account and that account balances can hold negative values.

Production implementations would likely be more complex. Typically, accounts need information about current balance, pending credits, and pending debits.

For all transactions, ensure that you use the appropriate level of  for your deployment.