Time Travel

pg_mentat maintains a complete history of every assertion and retraction. Unlike conventional databases that overwrite data in place, pg_mentat is accumulate-only: retracting a fact records the retraction as a new event rather than deleting the original assertion.

This immutable history enables temporal queries – looking at the database as it existed at any past transaction.

Concepts

Transaction IDs

Every transaction is assigned a monotonically increasing 64-bit integer (tx). Transaction IDs serve as both: - A unique identifier for the transaction - A logical timestamp (total ordering of all changes)

Basis-t

The “basis-t” is the most recent transaction ID in the database. It represents the current state.

Datom Lifecycle

A datom [entity attribute value tx added] has five components: - added = true means the fact was asserted at transaction tx - added = false means the fact was retracted at transaction tx

The “current” value of an attribute is determined by finding the most recent assertion that has not been subsequently retracted.

As-Of Queries

Query the database as it appeared at a specific transaction. Only datoms with tx <= target are visible, and retracted datoms are excluded.

Via Query Input

SELECT edn_q(
  '[:find ?name ?age
    :where
    [?e :person/name ?name]
    [?e :person/age ?age]]',
  '{"asOf": 1050}'
);

Via Dedicated Function

SELECT mentat_as_of('default', 1050,
  '[:find ?name :where [?e :person/name ?name]]',
  '{}');

Use Cases

  • Audit – “What did the data look like when we made that decision?”
  • Debugging – “When did this value change?”
  • Reproducibility – “Re-run analysis against the same data snapshot”
  • Undo exploration – “What was the state before this transaction?”

Since Queries

Query the current datoms that were asserted after a specific transaction (Datomic’s since): a value asserted after T and since retracted or replaced does not show. To see every change after T, assertions and retractions, use a history pattern [?e ?a ?v ?tx ?added] with since.

asOf and since together give the assertions with since < tx <= asOf that were not retracted by asOf (Datomic’s (since (as-of db T) S)).

Before 1.18.0 since returned different results on each backend: pg also showed values asserted after T and later replaced, and the embedded and DuckDB engines also counted retractions after T as values. With :x/n 1 at t1, 2 at t2 and 3 at t3, [:find ?v :where [_ :x/n ?v]] since t1 now returns [3] everywhere (was [2 3] on pg, [1 2 3] on SQLite and DuckDB). pg rejected asOf with since in edn_pull; the embedded engine dropped since.

SELECT edn_q(
  '[:find ?e ?name
    :where
    [?e :person/name ?name]]',
  '{"since": 1050}'
);

Via Dedicated Function

SELECT mentat_since('default', 1050,
  '[:find ?e ?name :where [?e :person/name ?name]]',
  '{}');

Use Cases

  • Change detection – “What entities were modified since my last sync?”
  • Incremental processing – “Process only new data since last batch”
  • Event sourcing – “What happened since transaction T?”

History Queries

View the full history of an entity-attribute pair, including all assertions and retractions:

SELECT mentat_history('default', 10001, ':person/name');

Returns:

[
  {"value": "Alice Smith", "tx": 1001, "added": true},
  {"value": "Alice Smith", "tx": 1050, "added": false},
  {"value": "Alice Johnson", "tx": 1050, "added": true}
]

This shows that: 1. “Alice Smith” was asserted at tx 1001 2. “Alice Smith” was retracted at tx 1050 3. “Alice Johnson” was asserted at tx 1050 (a name change)

Transaction Range

View all datoms across a range of transactions:

SELECT mentat_tx_range('default', 1000, 1100);

Returns: All datoms (assertions and retractions) in transactions 1000 through 1100.

Transaction Log

Get structured transaction log entries with metadata:

SELECT mentat.log('default', 1000, 1010);

Returns: Array of transaction objects, each containing: - tx – transaction ID - tx_instant – timestamp - datoms – array of [e, a, v, tx, added] tuples

Diff

Compare two points in time:

SELECT mentat.diff('default', 1000, 1050);

Returns: Object with added and retracted arrays showing the net changes between the two transactions.

How Time Travel Works Internally

As-Of Implementation

As-of queries add a tx <= ?target_tx filter to every pattern in the generated SQL. The query compiler: 1. Adds AND d.tx <= $as_of to each table join 2. Applies retraction filtering: excludes an assertion when a retraction of the same entity, attribute and value exists with a later tx <= target. This hides a value retracted at or before the target whether or not a new value replaced it. (Before 1.18.0 pg hid only a cardinality-one value replaced by a newer one, so a plainly retracted value, or a retracted cardinality-many value, still showed as of a later transaction.)

Since Implementation

Since queries add AND d.tx > $since_tx and the same retraction filtering (any later retraction, no upper bound), so they return the current datoms asserted after since_tx. A history pattern gets only the tx bound and sees retractions too.

History Implementation

History queries bypass the current-value filter entirely, returning all datoms (both assertions and retractions) for the specified entity-attribute pair, ordered by transaction ID.

Combining Time Travel with Other Features

Pull with As-Of

edn_pull takes the same temporal options as edn_q as a third argument (since 1.18.0):

-- Alice as of transaction 1000: the values asserted at or before 1000 and
-- not retracted at or before 1000.
SELECT edn_pull('[:person/name :person/age {:person/friend [:person/name]}]',
                10001, '{"asOf": 1000}');

-- Only the current datoms transacted after 1000.
SELECT edn_pull('[*]', 10001, '{"since": 1000}');

A pull in the :find of a historical query reads the query’s basis:

SELECT edn_q('[:find (pull ?e [:person/age]) :where [?e :person/name "Alice"]]',
             '{"asOf": 1000}');

Before 1.18.0 edn_pull had no options argument and a find-pull under asOf or since returned current values.

Subscriptions with Since

Subscriptions notify on new transactions. Combine with since to process missed changes after reconnection:

-- On reconnect, catch up from last known tx
SELECT edn_q(
  '[:find ?e ?name :where [?e :person/name ?name]]',
  '{"since": 1050}'
);

Excision

Excision is the only operation that breaks the immutability guarantee. It permanently removes entities and all their history from the database. See SQL Function Reference for details.

Excision is gated by a per-partition flag (allow_excision) and protected against accidentally excising schema entities (entid < 10000).