pgcolumnar 1.0.0-alpha.4
- This Release
- pgcolumnar 1.0.0-alpha.4
- Date
- Status
- Unstable
- Other Releases
- Abstract
- Analytic column storage for PostgreSQL, built as a native table access method
- Description
- pgColumnar is a columnar storage table access method for PostgreSQL, written as a clean-room, MIT-licensed implementation. It reads and writes its own native format, PGCN v1, and supports chunk-group skipping from zone maps, bloom filters and star-schema join runtime filters, vectorized aggregation, projections, Z-order and Hilbert clustering, retention, online compaction and reclustering, parallel bulk ingest and export, Apache Arrow and Parquet import and export, Apache Iceberg, and object storage.
- Released By
- jd
- License
- MIT
- Resources
- Special Files
- Tags
Extensions
- pgcolumnar 1.0.0-alpha.4
- Analytic column storage for PostgreSQL, built as a native table access method
Documentation
- RELEASE_NOTES_1.0-alpha4
- pgColumnar 1.0-alpha4 release notes
- sql-reference
- SQL reference
- installation
- Installation
- roadmap
- Roadmap
- RELEASE_NOTES_1.0-alpha3
- pgColumnar 1.0-alpha3 release notes
- features
- Features
- NATIVE_FORMAT_AND_INTERFACE_SPEC
- pgColumnar native format and interface specification
- administration
- Administration
- README
- Running the pytest harness
- configuration
- Configuration reference
- testing
- Testing
- VACUITY_MODES
- The vacuity modes: what is refused, what is not, and what nobody attacked917’s pytest twin tested a function’s body and left its CALL SITE uncovered, so
- how-to
- How-to guides
- user-guide
- User guide
- requirements
- requirements
- limitations
- Limitations and compatibility
- suites_without_accounting
- suites_without_accounting
- ARCHITECTURE
- pgColumnar architecture
- ROADMAP
- pgColumnar roadmap1074 (a sub-margin FSST win is discarded and nothing tells the user), #1075
- check_ledger_budget
- check_ledger_budget
- README
- pgColumnar logo
- benchmarks
- Benchmarks426 for both the anchored and the infix case.682).** The Iceberg work is on the Iceberg read path. It does not touch the scan,598 gates the gating on projected payload width, a plan-time property the unreliable LIKE '%...%' estimate cannot corrupt.
- PROVENANCE
- Provenance421). ClickBench is licensed CC BY-NC-SA 4.0, not Apache-2.0, which an
- requirements-test
- requirements-test
- TESTS
- The pytest corpus: what each test asserts, and why it exists
- best-practices
- Best practices
- expected_tests
- expected_tests
- CHANGELOG
- Changelog1081 replaced a count of entry->fileOffset != rg->fileOffset with a per-field1078 repaired two arms in native_fetch_projection.sh that compared a whole-file875 projection-writer reset reddens both directions of the latch (got 105 want 116997 closed s3://u:p@bucket/key – userinfo in the URL the caller writes. The1024’s own report – two PRs each taking the next section number – is no longer1041 wrote 12 rows at 18 against 934 uniform ones, caught only by CI’s `suites1051 taught the extractor the recorders lib.sh shares. A suite may also define1010 inserted the majors a row claims as field 4, moving the last-red to field 5, and432’s definition of done, and it was reading the wrong argument. The python side was1023’s contents entry for section 30 landed after 31. Resolving this PR’s conflict in432’s parity the tool can certify, and it is a false red rather than a false green.1012’s test_join_vector_agg.py and #1020’s test_differential.py, which landed into the998 added the shell part and no pytest twin, against the owner’s rule that a test958 closed the datum one exploit used. The three scans that read such data922 replaced roughly 28 producer | grep -q PAT tests with964 snapshots the module’s bindings. Expect.num = a stub that still counts852.376 bound and the clustered-index and point-lookup guards in
README
Contents
Analytic column storage for PostgreSQL, built as a native table access method.
Read the documentation at commandprompt.github.io/pgcolumnar
A table created USING pgcolumnar stores its data by column, with per-column
compression, chunk-group skipping, and a vectorized aggregate path. It is for
analytic workloads: large scans, aggregates, and column projections over
append-mostly data.
pgColumnar builds from one source tree on PostgreSQL 15 through 18, with 19
validated against 19beta2, and is
licensed under the MIT License. It is pre-release; the version marker
is 1.0-alpha4, recorded in VERSION.
The latest published pre-release is v1.0-alpha4. A table USING pgcolumnar is stored in the
native on-disk format, PGCN v1.
Documentation
The full documentation is published at
commandprompt.github.io/pgcolumnar. The
same pages are in this repository under docs/:
| Features | What pgColumnar provides |
| Installation | Build, load, and create the extension |
| User guide | Create tables, load data, and query |
| Administration | Operate columnar tables in production |
| Configuration | Settings and per-table options |
| SQL reference | The pgcolumnar.* functions |
| Limitations | Compatibility and known constraints |
| Benchmarks | Size and latency numbers |
| Testing | The test suite and version matrix |
| Changelog | Notable changes |
| Architecture | Source map for developers |
Quick start
Build with PGXS against the target server, add the library to
shared_preload_libraries, and restart:
make PG_CONFIG=/path/to/pg_config
make install PG_CONFIG=/path/to/pg_config
shared_preload_libraries = 'pgcolumnar'
Then, in a database:
CREATE EXTENSION pgcolumnar;
CREATE TABLE events (id bigint, ts timestamptz, kind int, payload text)
USING pgcolumnar;
INSERT INTO events
SELECT g, now(), g % 8, 'p' || g
FROM generate_series(1, 1000000) g;
SELECT count(*), avg(kind) FROM events WHERE kind = 3;
See the installation guide for requirements and the user guide for loading and querying.
Independence
pgColumnar is an independent implementation. It is not derived from the source of any other columnar project. Its on-disk format, its metadata catalog and its SQL interface come from published column-store research and from the open Apache Arrow, Parquet and ORC specifications. They are recorded in design/NATIVE_FORMAT_AND_INTERFACE_SPEC.md. The implementation is built from that specification and the public PostgreSQL API, by the clean-room method described in PROVENANCE.md.
License
MIT. See LICENSE.