Tools¶
Tool
¶
Bases: Protocol
One modelling framework's view of a record.
Implementations are stateless (a module-level singleton is fine): every method takes the record or model it operates on. A structural type, for annotating code that takes any tool - not a dispatch table; tools are reached by importing them.
A Record rather than a record throughout, so a tool builds from a
directory as readily as from an overlay and has no reason to know layering
exists.
Notes
requires
¶
requires(record: RecordLike) -> Requirements
What this tool needs from record to build a model.
Record-dependent, not a constant: which attributes are required follows the categories the record's labels have and its declared schema.
Source code in src/datarecord/tools/base.py
verify
¶
verify(record: RecordLike) -> Requirements
build
¶
build(record: RecordLike) -> Any
to_datarecord
¶
to_datarecord(
model: Any,
con: DuckDBPyConnection,
*,
results_only: bool = False,
base: RecordLike | None = None,
) -> RecordLike
model presented as a layer write_layer can persist.
The inverse of build. Framework-specific: undoing a framework's own
shape is exactly what a tool knows and the record layer does not.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
con
|
DuckDBPyConnection
|
The connection the relations are planned on - in practice |
required |
results_only
|
bool
|
Present only what a solve produced - its result attributes and the
intermediates they need - as a thin layer to write as a child of
|
False
|
base
|
RecordLike | None
|
The record the layer is written over as a child: required with
|
None
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in src/datarecord/tools/base.py
Requirements
dataclass
¶
Requirements(
dims: frozenset[str] = frozenset(),
categories: frozenset[str] = frozenset(),
attributes: frozenset[tuple[str, str]] = frozenset(),
unsupported_keys: frozenset[
tuple[str, str]
] = frozenset(),
unsupported_dims: frozenset[str] = frozenset(),
unsupported_values: frozenset[
tuple[str, str]
] = frozenset(),
names: frozenset[str] = frozenset(),
ignored: frozenset[tuple[str, str]] = frozenset(),
)
What a tool needs from a record, and what a given record fails to supply.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dims
|
frozenset of str
|
Dims the tool cannot build a model without. |
frozenset()
|
categories
|
frozenset of str
|
Categories the tool requires the record to define labels of, or, in a
|
frozenset()
|
attributes
|
frozenset of tuple of (str, str)
|
|
frozenset()
|
unsupported_keys
|
frozenset of tuple of (str, str)
|
|
frozenset()
|
unsupported_dims
|
frozenset of str
|
Dims the record declares that this tool has no place for in the model it builds. |
frozenset()
|
unsupported_values
|
frozenset of tuple of (str, str)
|
|
frozenset()
|
names
|
frozenset of str
|
Labels the framework scopes per category and two of its categories claim, where a record has one axis row per label. |
frozenset()
|
ignored
|
frozenset of tuple of (str, str)
|
|
frozenset()
|
describe
¶
A one-line summary, for an error message or a log line.
Source code in src/datarecord/tools/base.py
Schema
dataclass
¶
Schema(attrs: dict[str, tuple[Attr, ...]] = dict())
A tool's attribute mapping against the record's vocabulary.
Only differing attributes need an entry; anything unlisted is read under the
same name, so an empty Schema is the identity. Attr.source names
attributes in the record's vocabulary.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
attrs
|
dict of str to tuple of Attr
|
Per entity type, the attributes that are renamed or computed. |
dict()
|
attr
¶
attr(ctype: str, name: str) -> Attr
ctype's mapping for name, or the identity when it has none.
sources
¶
Which record attribute(s) ctype's name needs; itself, if unmapped.
What verify checks resolvability against: a renamed or computed
attribute is satisfied by its sources, not by its own name.
Source code in src/datarecord/tools/base.py
resolve
¶
resolve(
record: RecordLike,
ctype: str,
name: str,
labels: Mapping[str, DuckDBPyRelation],
flags: Mapping[str, Flags],
*,
filter: bool = True,
) -> DuckDBPyRelation
ctype's name as a long relation over record, mapping applied and settled (Attr.resolve).
Source code in src/datarecord/tools/base.py
Attr
dataclass
¶
One tool attribute and the record attribute(s) it is built from.
The vocabulary seam: a rename, or a value computed from several,
declared rather than open-coded in a build.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
The tool's name for the attribute. |
required |
source
|
tuple of str
|
Record attribute name(s) it is read from. A rename has one. |
required |
compute
|
callable
|
Maps one settled long relation per |
None
|
Notes
resolve
¶
resolve(
record: RecordLike,
labels: Mapping[str, DuckDBPyRelation],
flags: Mapping[str, Flags],
*,
filter: bool = True,
) -> DuckDBPyRelation
This attribute's long relation, each source settled (settle) before compute.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
flags
|
Mapping[str, Flags]
|
Per source, its |
required |
filter
|
bool
|
As |
True
|
Source code in src/datarecord/tools/base.py
UnsupportedRecordError
¶
UnsupportedRecordError(tool: str, missing: Requirements)
Bases: ValueError
A record does not define everything the tool needs (Tool.verify).
Source code in src/datarecord/tools/base.py
PyPSA¶
The PyPSA tool needs the optional extra: pip install datarecord[pypsa].
PyPSATool
¶
Bases: Tool
Build a pypsa.Network from a record and read its results back.
component_types
¶
The component types PyPSA's type registry knows.
requires
¶
requires(record: RecordLike) -> Requirements
PyPSA's axes, the record's component types PyPSA knows, and their required attributes by source.
Source code in src/datarecord/tools/pypsa.py
verify
¶
verify(record: RecordLike) -> Requirements
What this record, a selection narrowed as build narrows it, fails to supply for a PyPSA build; falsy if it is usable.
Reports PyPSA's axes the schema lacks, keys and dims of its own this
tool cannot honour (and nothing else where it finds any), component
types PyPSA does not know, required attributes (by source) with no
row, property or declared default, a port's bus where a member has no
connection there, curves and values a network would hold one of
(_unheld), and in ignored stored attributes a type does not take,
which build leaves out.
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in src/datarecord/tools/pypsa.py
build
¶
build(record: RecordLike) -> Network
The resolved network.
A selection (Record.select) builds the network of its labels,
selected once more as PyPSA holds it (_narrowed): a squeezed
period is a single-period network without investment_period, a
squeezed scenario a deterministic one, and kept scenarios' weights
are renormalised. A record with a dim of its own is refused, as
PyPSACollection builds it, and a selection at one label of that dim
builds the member there.
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
UnsupportedRecordError
|
If |
Source code in src/datarecord/tools/pypsa.py
2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 | |
to_datarecord
¶
to_datarecord(
model: Network,
con: DuckDBPyConnection,
*,
results_only: bool = False,
base: RecordLike | None = None,
) -> RecordLike
Present a Network as the Record write_layer persists.
The inverse of build: static and series values become long rows on
con, port columns connection rows carrying a role, buses and
carriers the bus and carrier axes in PyPSA's order, "" NULL,
and each shape the shape_{type} property of the label it outlines.
Outputs are attributes like any other. A static value is left out
where it repeats what the record resolves to without it: over base,
base's value, else the default; over nothing, the default.
With results_only the record is a thin layer over base: the
outputs, at their default too, base's output rows at the scenarios,
buses, entities and connections the network holds that it does not
write again restated as NULL, and this solve's SubNetworks as the
sub_network axis, unless it holds fewer buses or entities than
base, whose SubNetworks it then leaves. A series is written at the
network's snapshots, base's standing at the others; over a
multi-period base, a value over no snapshot is written for the
entities active in the network's periods alone. base may be the
selection the network was built from (Record.select): the layer is
then over its record, each row at the labels it squeezes, one period
squeezed writing at that period. Over the selection at one label of
the record's dim of its own, the network is that member, written as
PyPSACollection writes a collection of it alone.
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
UnsupportedRecordError
|
If two entity types share a name. |
Source code in src/datarecord/tools/pypsa.py
2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 | |