Skip to content

datarecord

Dimensioned attribute data with a declared schema.

A record holds components (named members of a type), connections between components and buses, attribute values over both, and the axes those values vary along. A schema declares what may exist; the data says what does.

Records stack: a layer is a partial record on top of a parent, resolved last-writer-wins, so a scenario variant costs the rows it changes rather than a copy of everything. On disk a record is a plain parquet directory that a tool knowing nothing about this package can read.

datarecord depends only on duckdb, narwhals and pydantic. It names no modelling framework — a framework consumes a record, a workflow engine produces one, and neither needs to know how the other works.

pip install datarecord           # core
pip install datarecord[pypsa]    # with the PyPSA tool
from datarecord import Record, connect

con = connect()
record = Record.at("s3://bucket/my-record/", con)

record.entity_types["Generator"].collect()  # wide member rows
record.attributes["p_max_pu"].collect()  # long value rows
record.flags("Generator")  # which axes each attribute uses

Where to go

How to read, edit, layer and write a record, and how a modelling framework consumes one.

What a record is and why it is that way — the authoritative design. The docstrings cite these pages rather than restating the argument.

Generated signatures and docstrings for every public symbol.

Contributing

See CONTRIBUTING.md for the workflow and conventions, and AGENTS.md for how AI-assisted contributions must be marked.

pixi run test    # the test suite
pixi run lint    # ruff, prettier, taplo, typos, zizmor, reuse, mypy
pixi run -e docs docs    # serve this site locally