Memory Image in Kotlin

When people start an enterprise application, one of the earliest questions is “how do we talk to the database”. These days they may ask a slightly different question: “what kind of database should we use — relational or one of these NOSQL databases?”.

But there’s another question to consider: “should we use a database at all?”

— Martin Fowler

What is Memory Image?

Memory image provides a reliable persistence mechanism where all application data resides safely in main memory. Yes: good ole’, volatile, random-access memory.

Provided you domain model fits in main memory (which is cheap and abundant nowadays) this approach yields significant benefits over the established database-centric approach:

• Substantially faster transaction and query processing times as the system operates at RAM speeds!
• No object-relation impedance mismatch to speak of as objects only reside natively in memory. No implementation-level ORM limitations bubbling up to the design
• Much richer domain models leveraging advanced language and platform features, unhampered by persistence concerns. DDD nirvana; the cure for anemic domain models 😉

Hmm… Please Elaborate

Rather than persisting domain entities as such (as is done, typically, on a database) in the memory image approach what gets persisted is the sequence of application events that modify the state of the entities in the first place.

Consider the following minimalistic bank domain model:

Here, a bank holds a collection of accounts each having a balance that changes in time as it responds to events such as:

• deposits
• withdrawals
• transfers

Each such an event can be modeled as a command that, when applied to an account, modifies the account’s balance to reflect the banking operation embodied by the command.

This could be modeled as:

Let’s take a look at a progression of commands and the evolving system state resulting from their successive application:​

The key idea behind the memory image pattern is:

Serialize all state-modifying commands on persistent storage.

Reconstruct in-memory application state by replaying the deserialized commands onto an empty initial state

Somewhat paradoxically, entity classes themselves don’t need to be persisted! (But they might, for snapshotting, as mentioned below.)

If your application data fits into available memory and your event history fits into available disk space then you’re in business.

Memory Image Processor

A memory image processor consumes a stream of application commands by applying each incoming command to a mutable object (here referred to as the system).

Because applying commands in memory is so fast and cheap, a memory image processor can run in a single thread and consume incoming command sequentially with no need to provide concurrent access to its system. This removes much of the complexity traditionally associated with transactions as conflicts arising from concurrent mutation simply do not occur.

Individual command application, however, can fail in the midst of a transactional sequence of mutations so the memory image processor still needs a way to rollback partial changes and restore system integrity in the face of invalid data and constraint violations.

Incoming commands should only be serialized upon successful completion. Obviously, if command serialization fails, the memory image processor will stop processing further commands until command serialization is restored.

Lastly (and crucially!), a memory image processor also services queries.

A query is another type of event which, unlike commands, does not mutate system state. Importantly, queries are serviced in multi-threaded mode so querying the system is efficient and concurrent. Because in-memory access is so fast, many queries can be satisfied without indexing. However, special-purpose, in-memory indexing can be easily implemented as dictated by application requirements.

The following class diagram summarizes the memory image processor:

👉 Because application restarts entail reprocessing the (potentially large) history of all mutating commands, system snapshotting can be used to serialize the entire in-memory state on demand. This enables faster restarts at the expense of losing the ability to time-travel.

Memory Image Processor in Kotlin

The above class diagram is materialized in Kotlin as:

Simple Example: Bank Domain Model

To exercise the above memory image processor, let’s revisit our bank domain model:

This is implemented in Kotlin as:

Simple Example: Testing The Processor

The following test exercises the memory image processor using the same sequence of commands outlined above:

Conclusion

Memory image provides a simple, straightforward way to achieve high performance and simplicity without the complications associated with persisting objects on a database (whether SQL or not.)

Kotlin is a uniquely expressive language in which to implement this architectural pattern.

Interested readers can inspect the working code at the kmemimg Github repository.