DEP 24: categories are enum dimensions¶
| DEP | 24 |
|---|---|
| Title | categories are enum dimensions |
| Author | Bryn Pickering |
| Status | Implemented (2026-10-26) |
| Requires | DEP 12, DEP 13, DEP 18, DEP 23 |
| Created | 2026-10-21 |
| Version | Unreleased |
The design landed in the schema, the data model, the record format and the tools reference. Where they and this DEP differ, follow those pages.
Abstract¶
A str dim may list its labels with one_of, as mathspec#915 does for a mathspec dimension. A category is then a label of such a dim, and a categorising property is a property {one_of: <dim>} that refers to it. DEP 18 landed that, and dropped the Enum dtype. This DEP adds two things. A dim with one_of has its axis derived from the list, so the schema lists its labels and no layer writes them. A multi-dim may be categorised, so that connection.role categorises connection.
Motivation¶
A closed set of values was two constructs, an Enum dtype on a property and a reference to a dim, and only one categorised. DEP 18 made them one: ctype and role are dims that list their labels, and entity.ctype: {one_of: ctype} categorises entity.
A one_of dim's labels are still data. The list only bounds the axis. A layer writes dims/ctype.parquet whole, as DEP 20 requires, and may drop a label of the list from it. PyPSA writes the whole list on every export, so the file restates the schema. datarecord checks a category that a default_by names against the list, yet that category may not be a live label in a given record.
A multi-dim is still never categorised. connection.role is {one_of: role}, role lists input, output and attached, and PyPSA lists connection in partial. Yet datarecord refuses a default_by keyed on connection.role, because only a dim in partial that is not a multi-dim has categories. DEP 23 made the group a multi-dim and left open whether the rule extends to one.
Specification¶
What DEP 18 settled¶
| #81 proposed | DEP 18 landed |
|---|---|
enum: [...] beside a String dim's dtype |
one_of: [...] on a str dim, mathspec#915's final key |
a property into an enum dimension |
a property {one_of: <dim>} |
a categorising property is one into an enum dimension, of a dim in partial |
the same rule, spelt one_of |
{Enum: [...]} refused at load |
refused, naming one_of |
PyPSA's component and role as enum dimensions |
PyPSA's ctype and role dims |
categories checked at load in a default_by and a selector |
checked against the dim's one_of |
A categorising property of a multi-dim¶
A property {one_of: <d>} of a multi-dim in partial, where d lists its labels, categorises the multi-dim by the rule for a dim:
dimensions:
role: { one_of: [input, output, attached] }
connection: { over: { entity: entity, bus: bus } }
properties:
connection:
role: { one_of: role }
partial: [entity, bus, connection]
connection.rolecategorisesconnection, because a multi-dim is a dim since DEP 23.Schema.categorizing("connection")listsrole, anddefault_forandflagskey on it by plain name, as onentity.ctype.- A
default_bykeyed onconnection.roleapplies to a value ofconnection: an attribute over it, or a property of it. datarecord checks its keys at load againstrole'sone_of. - A selector keyword
role=on an attribute overconnection, or on a property of it, selects the tuples with that role.set("connection.note", v, role="output")works asset("entity.note", v, ctype="Generator")does.
A one_of dim's axis is the schema's¶
A dim with one_of: [...] has its axis derived from the list, in list order:
- No
dims/<dim>.parquetis written or read for it, andrecord.dims["ctype"]is the list. The writer refuses a layer with such a file. - A record written whole hands over the axis among its
dims. The writer drops the axis where it is the list, in the same order, and refuses any other, naming the dim and the difference. - datarecord refuses at load a property declared on it, because without a file no column can have one.
- datarecord refuses
addandremoveon it, naming the schema. - Otherwise it is a dim. An attribute may range over it and a property may refer to it. A value outside the list is refused as dangling.
| Before | Under this DEP |
|---|---|
the list bounds the axis, and a layer writes dims/ctype.parquet whole |
the axis is the list, with no layer file |
add refuses a label outside the list, and remove works |
both refused |
PyPSA writes the whole list as the ctype and role axes on export |
the tool writes nothing for them |
a property that refers to ctype is checked against the resolved axis |
checked against the list, with no dead labels and no cascade |
a one_of dim may have properties |
refused |
Rationale¶
Categories need a subject whose keys the owner map assigns per label, and partial makes a multi-dim such a subject, as it does a dim. Refusing them for a multi-dim alone would keep the one rule that set a group apart.
datarecord checks a category at load where it is named. That needs the vocabulary in the schema, and an axis that a layer may shrink lets a checked category be no live label. Deriving the axis from the list makes the axis and the list the same.
What it costs:
- A record with a
dims/file for aone_ofdim is re-exported without it. datarecord refuses a layer that writes one, and aRecordLikewhose axis of it is not the list. - datarecord refuses at load a property declared on a
one_ofdim. - datarecord refuses
addandremoveon aone_ofdim, so adding a PyPSA component type stays a schema change. - A schema that keyed a
default_byon a multi-dim's property was refused, and now loads.
Usage¶
schema.categorizing("connection") # ("role",)
schema.default_for("efficiency", role="input")
w.set("efficiency", 0.95, role="output") # selector: the category
record.flags(role="attached")
record.dims["ctype"] # its `one_of` labels, in order
w.add("ctype", ["Hydrogen"]) # refused: a `one_of` dim's labels are the schema's
Implementation notes¶
- A
one_ofdim's axis is derived from the schema on read. The writer refuses itsdims/file,add,removeand properties on it are refused, and the PyPSA tool stops writingctypeandrole. Schema.categorizingandSchema._categorized_byaccept a multi-dim inpartial, and the refusal of categories on a multi-dim goes.- Category selection in
set,flagsanddefault_forjoins on the multi-dim's struct column, as on a dim's column. - The dims, properties, format and PyPSA pages. The open question on a multi-dim's categories moves to settled.
Checks: a one_of dim has no layer file, and record.dims of it is its list. A default_by keyed on connection.role loads, and a misspelt role in it fails at load. default_for and flags agree per role on a PyPSA network.
Rejected ideas¶
Enumkept besideone_ofdims: two ways to state "one of these labels", one categorising and one not, by dtype. DEP 18 settled it.- A
one_ofdim that a layer may grow beyond its list: a category in adefault_bycould then be checked only against the folded record, not at load.
Open issues¶
- An axis that a layer writes, bounded by the list, the design before this DEP. A layer writes the axis,
addrefuses a label outside the list andremoveworks. A tool or layer then chooses which labels exist, and aone_ofdim may have properties. The cost is a file that restates the schema, and a category that may not be a live label. - Whether a
one_ofdim may grow in a layer. Its labels are the schema's, so a custom PyPSA component is a schema change, not a data edit. That is the cost of checking categories at load.