pg_improv 0.10.0

This Release
pg_improv 0.10.0
Date
Status
Stable
Abstract
Improv multidimensional models over PostgreSQL tables, kept current incrementally
Description
pg_improv runs Improv's multidimensional modeling engine over your own tables. Bind table columns to categories (dimensions) and measures, define derived measures in a dimension-aware formula language (Revenue = Price * Quantity, SUM(Revenue OVER Time)), and read the results from ordinary result tables. Results are kept current incrementally by a background worker on logical decoding (the default; writers pay nothing), by statement triggers inside the writer's transaction, or by improv.recalc() in the caller's snapshot. improv.what_if previews edits without writing. LANGUAGE improv writes SQL functions as Improv formulas. The Improv desktop GUI, terminal UI and CLI open the same model with a postgres:// URL. Built with pgrx (Rust): install with cargo-pgrx.
Released By
gregburd
License
Apache-2.0 MIT
Resources
Special Files
Tags

Extensions

pg_improv 0.10.0
Improv multidimensional models over PostgreSQL tables

Documentation

README
Family finance with pg_improv
dataflow
dataflow
scalar
scalar
CHANGELOG
Changelog

README

Improv

A multidimensional spreadsheet in Rust, in the line of Lotus Improv and Quantrix. You model in named categories, items and measures instead of cell addresses, write formulas over dimensions (with the spreadsheet functions people know, Excel’s names: IF, ROUND, NPV, STDEV, TEXT and some 330 more, plus running totals and PREV), and every derived value recomputes incrementally through a differential-dataflow graph. The same model opens in a NeXTSTEP-style desktop GUI, a terminal UI, a CLI, a streaming pipe, an HTTP server, inside Postgres and inside DuckDB.

Docs CI

Documentation: the Improv guide on GitHub Pages (also on Codeberg Pages) · API reference (rustdoc) · Ways to use Improv · Function reference · Improv in Postgres · Improv in DuckDB

The Improv workspace: worksheets, formulas and the model browser

Revenue[Time, Product] = Price[Product] * Quantity[Time, Product]
Total[Product]         = SUM(Revenue OVER Time)

Price is broadcast over Time. Editing one Quantity cell recomputes one Revenue cell and one Total, not the sheet.

Ways to use it

Use it for Start with
GUI improv-gui building and exploring a model by hand: drag tiles to pivot, edit cells, add formulas, several worksheets at once improv-gui sample.db
TUI improv-tui the same in a terminal or over SSH improv-tui sample.db
CLI improv scripting a model, bulk CSV/SQL import and export, CI improv --help, man improv
Streaming improv stream pushing a live feed through a model and reading the changed results producer | improv stream m.db Revenue
Postgres pg_improv modeling data already in Postgres; results land in tables any SQL client reads CREATE EXTENSION pg_improv
SQL functions LANGUAGE improv writing a SQL function as a formula CREATE FUNCTION … LANGUAGE improv
DuckDB improv extension a formula or a whole model over a DuckDB table, Parquet or CSV, in one query LOAD improv
Server improv-server serving a model to other programs as JSON over HTTP improv-server sample.db

Each front end takes a model file or a Postgres model URL (postgres://user@host/db?model=plan). The guide’s Ways to Use Improv chapter explains when to use which.

The desktop GUI

The GUI: pivoting, paging, editing, a range Sum and a new formula

The terminal UI

The TUI: moving, editing an input and watching Revenue recompute, pivoting, paging

Improv in Postgres

pg_improv: a model over a sales table, an UPDATE flowing to the results, a LANGUAGE improv function, then the TUI editing the same model

Install

Pre-built CLI, TUI, GUI and server binaries for Linux, macOS and Windows are attached to each release. To build from source you need Rust 1.97 or newer:

git clone https://codeberg.org/gregburd/improv && cd improv
cargo build --release      # target/release/{improv,improv-tui,improv-gui,improv-server}

The GUI needs the usual Wayland/X11/OpenGL runtime libraries (libxkbcommon-x11-0 libgl1 libegl1 on Debian/Ubuntu). With Nix, nix develop provides the toolchain, those libraries and the dev tools. External functions need python3 and, on Linux, bubblewrap for the sandbox.

The Postgres extension is built separately with cargo-pgrx; see crates/pg_improv/README.md.

Quick start

Build the 3-D sample model (Time × Product × Region, with Price, UnitCost, Quantity, Revenue, COGS and Margin) and open it:

IMPROV=target/release/improv bash docs/demo/sample-model.sh sample.db
target/release/improv-gui sample.db     # or improv-tui sample.db

Or build a model by hand with the CLI:

improv init m.db
improv add-category m.db 1 Time;    improv add-item m.db 10 1 Q1
improv add-category m.db 2 Product; improv add-item m.db 20 2 Widget
improv add-measure m.db 100 Price    number input Product
improv add-measure m.db 101 Quantity number input Time Product
improv set m.db 100 10  --at Product=Widget
improv set m.db 101 100 --at Time=Q1,Product=Widget
improv define m.db 102 "Revenue = Price * Quantity"
improv eval m.db 102
# Revenue[Time=Q1, Product=Widget] = 1000

and stream new data through it:

printf 'Price 15 Product=Widget\n' | improv stream m.db Revenue
# Revenue Time=Q1, Product=Widget = 1500

The CLI also imports and exports CSV and SQLite, runs sandboxed Python functions (register-ext, then CALL(...)), and refreshes external measures on a schedule. Getting Started is the tutorial; docs/man/improv.1 lists every command.

In Postgres

-- postgresql.conf: shared_preload_libraries = 'pg_improv', wal_level = logical
CREATE EXTENSION pg_improv;
SELECT improv.create_model('plan');
SELECT improv.bind_category('plan', 'Time',    'sales', 'quarter');
SELECT improv.bind_category('plan', 'Product', 'sales', 'product');
SELECT improv.bind_measure ('plan', 'Price',    'sales', 'price');
SELECT improv.bind_measure ('plan', 'Quantity', 'sales', 'qty');
SELECT improv.define('plan', 'Revenue = Price * Quantity');
SELECT * FROM improv_plan.revenue;    -- kept current as sales changes

CREATE FUNCTION bonus(sales float8, target float8) RETURNS float8
  LANGUAGE improv AS $$ Ratio = sales / target
                        MAX2(Ratio - 1, 0) * 1000 $$;

By default a background worker applies committed changes incrementally (at 100k cells, a one-row UPDATE shows up about 50 ms after commit, and the writer pays nothing). Trigger mode updates the results inside the writer’s transaction, and improv.recalc() updates them when you call it. improv-gui 'postgres://me@host/sales?model=plan' opens the same model, and editing a cell there updates the row in sales. See Improv in Postgres. For a worked example (statement reconciliation, budgets, net worth and savings goals for a family with jobs, consulting income, several accounts, cards, crypto, foreign currency, a house and cars), see A family’s finances in Postgres.

In DuckDB

LOAD improv;
SELECT improv_calc('ROUND(PMT(rate/12, n, -pv), 2)',
                   json_object('rate', 0.06, 'n', 360, 'pv', 200000));  -- 1199.1
SELECT * FROM improv('sales', '
  category Time = year
  category Product = product
  input Price = price
  input Quantity = qty sum
  Revenue = Price * Quantity
  Share = Revenue / SUM(Revenue OVER Product)
  Growth = Revenue - PREV(Revenue, Time, 1, 0)');

improv_calc evaluates any formula on named inputs, improv_functions() lists the 681 built-in functions, and improv(source, model) runs a model over a table or query and returns one row per computed cell. One binary loads into DuckDB v1.5.6 and v2.0. See Improv in DuckDB and crates/duckdb_improv/README.md.

Keys

TUI: arrows move · e/Enter edit · Tab next measure · p pivot · [ ] page · Shift+arrows select a range (its sum in the status line) · f/F filter · S save view · v cycle views · I/E CSV import/export · R revert to saved · q quit and save.

GUI: drag a tile between strips to pivot (or right-click it) · double-click or Enter to edit · type Name = formula on the formula list’s last line · Shift+click or drag selects a range (its Sum in the status bar) · Ctrl+S save, Ctrl+Z undo, Ctrl+Y redo · File ▸ Revert to Saved after a save conflict.

Workspace

Crate Binary What it is
improv_core_model Categories, items, measures, coordinates, formulas, values, views
improv_engine Formula compiler and differential-dataflow evaluator; the live incremental engine
improv_nl_formula Controlled-English ⇄ formula
improv_storage_mentat A model as datoms in an embedded Mentat (SQLite) store
improv_storage_pg A model in a Postgres improv catalog, with columns bound to tables
improv_store Opens a file or a postgres:// URL
improv_storage_sql, improv_storage_csv, improv_data_source SQL and CSV import, export and refresh
improv_extfn Sandboxed external functions (Python, R, Julia, Pure, Wasm)
improv_cli improv Command-line tool and stream
improv_tui improv-tui Terminal UI
improv_gui improv-gui Desktop GUI (egui)
improv_server improv-server JSON HTTP API
improv_pg_model Postgres-free core of the extension
pg_improv (extension) The Postgres extension (pgrx, its own workspace)
improv_duckdb_model DuckDB-free core of the DuckDB extension
duckdb_improv (extension) The DuckDB extension improv (duckdb-rs, its own workspace)

Development

See CONTRIBUTING.md for the quality gate (fmt, clippy with -D warnings, tests, cargo deny, typos, rustdoc, the GUI screenshot golden and the ≥ 90% per-crate coverage gate). Design notes are in docs/reviews/ and .agent/steering/. The guide’s source is docs/src/ (mdbook build docs); man pages are in docs/man/.

License

Licensed under either of Apache-2.0 or MIT at your option.