contract.json¶
JsonParams
dataclass
¶
Json ¶
Json(
name: str,
*,
description: Optional[str] = None,
reader_params: TReaderParams = UnsetParams,
**kwargs,
)
Bases: DataStream[dict[str, str], JsonParams]
JSON file data stream provider.
A data stream implementation for reading single JSON objects from files.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
DataStream
|
Base class for data stream providers. |
required |
Examples:
from contraqctor.contract.json import Json, JsonParams
# Create and load a JSON stream
config_stream = Json(
"config",
reader_params=JsonParams(path="config/settings.json")
)
config_stream.load()
# Access the data
config = config_stream.data
api_key = config.get("api_key")
Source code in src/contraqctor/contract/base.py
103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | |
name
property
¶
name: str
Get the name of the data stream.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
Name identifier of the data stream. |
resolved_name
property
¶
resolved_name: str
Get the full hierarchical name of the data stream.
Generates a path-like name showing the stream's position in the hierarchy, using '::' as a separator between parent and child names.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
The fully resolved name including all parent names. |
description
property
¶
parent
property
¶
parent: Optional[DataStream]
Get the parent data stream.
Returns:
| Type | Description |
|---|---|
Optional[DataStream]
|
Optional[DataStream]: Parent data stream, or None if this is a root stream. |
is_collection
property
¶
is_collection: bool
Check if this data stream is a collection of other streams.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if this is a collection stream, False otherwise. |
reader_params
property
¶
reader_params: TReaderParams
Get the parameters for the data reader.
Returns:
| Name | Type | Description |
|---|---|---|
TReaderParams |
TReaderParams
|
Parameters for the data reader. |
at
property
¶
at: _AtProtocol
Get a child data stream by name.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
Name of the child data stream to retrieve. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
DataStream |
_AtProtocol
|
The child data stream with the given name. |
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
If the data stream does not support child access. |
Examples:
# Access stream in a collection
collection = data_collection.load()
temp_stream = collection.at("temperature")
# Or using dictionary-style syntax
humidity_stream = collection["humidity"]
has_data
property
¶
has_data: bool
Check if the data stream has loaded data.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if data has been loaded, False otherwise. |
has_error
property
¶
has_error: bool
Check if the data stream encountered an error during loading.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if an error occurred, False otherwise. |
data
property
¶
data: TData
Get the loaded data.
Returns:
| Name | Type | Description |
|---|---|---|
TData |
TData
|
The loaded data. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If data has not been loaded yet. |
set_parent ¶
set_parent(parent: DataStream) -> None
Set the parent data stream.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parent
|
DataStream
|
The parent data stream to set. |
required |
Source code in src/contraqctor/contract/base.py
164 165 166 167 168 169 170 | |
read ¶
read(
reader_params: Optional[TReaderParams] = None,
) -> TData
Read data using the configured reader.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
reader_params
|
Optional[TReaderParams]
|
Optional parameters to override the default reader parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
TData |
TData
|
Data read from the source. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If reader parameters are not set. |
Source code in src/contraqctor/contract/base.py
194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 | |
bind_reader_params ¶
bind_reader_params(params: TReaderParams) -> Self
Bind reader parameters to the data stream.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
params
|
TReaderParams
|
Parameters to bind to the data stream's reader. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If reader parameters have already been set. |
Source code in src/contraqctor/contract/base.py
211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 | |
clear ¶
clear() -> Self
Clear the loaded data from the data stream.
Resets the data to an unset state, allowing for reloading.
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Source code in src/contraqctor/contract/base.py
313 314 315 316 317 318 319 320 321 322 | |
load ¶
load() -> Self
Load data into the data stream.
Reads data from the source and stores it in the data stream.
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Examples:
from contraqctor.contract import csv
# Create and load a CSV stream
params = csv.CsvParams(path="data/measurements.csv")
csv_stream = csv.Csv("measurements", reader_params=params)
csv_stream.load()
# Access the data
df = csv_stream.data
print(f"Loaded {len(df)} rows")
Source code in src/contraqctor/contract/base.py
324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 | |
collect_errors ¶
collect_errors() -> List[ErrorOnLoad]
Collect all errors from this stream and its children.
Performs a depth-first traversal to gather all ErrorOnLoad instances.
Returns:
| Type | Description |
|---|---|
List[ErrorOnLoad]
|
List[ErrorOnLoad]: List of all errors raised on load encountered in the hierarchy. |
Source code in src/contraqctor/contract/base.py
379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 | |
load_all ¶
Recursively load this data stream and all child streams.
Performs depth-first traversal to load all streams in the hierarchy.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
strict
|
bool
|
If True, raises exceptions immediately; otherwise collects and returns them. |
False
|
Returns:
| Name | Type | Description |
|---|---|---|
list |
Self
|
List of tuples containing streams and exceptions that occurred during loading. |
Raises:
| Type | Description |
|---|---|
Exception
|
If strict is True and an exception occurs during loading. |
Examples:
# Load all streams and handle errors
errors = collection.load_all(strict=False)
if errors:
for stream, error in errors:
print(f"Error loading {stream.name}: {error}")
Source code in src/contraqctor/contract/base.py
396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 | |
MultiLineJson ¶
MultiLineJson(
name: str,
*,
description: Optional[str] = None,
reader_params: TReaderParams = UnsetParams,
**kwargs,
)
Bases: DataStream[list[dict[str, str]], JsonParams]
Multi-line JSON file data stream provider.
A data stream implementation for reading JSON files where each line contains a separate JSON object.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
DataStream
|
Base class for data stream providers. |
required |
Examples:
from contraqctor.contract.json import MultiLineJson, JsonParams
# Create and load a multi-line JSON stream
logs_stream = MultiLineJson(
"server_logs",
reader_params=JsonParams(path="logs/server_logs.jsonl")
)
logs_stream.load()
# Process log entries
for entry in logs_stream.data:
if entry.get("level") == "ERROR":
print(f"Error: {entry.get('message')}")
Source code in src/contraqctor/contract/base.py
103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | |
name
property
¶
name: str
Get the name of the data stream.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
Name identifier of the data stream. |
resolved_name
property
¶
resolved_name: str
Get the full hierarchical name of the data stream.
Generates a path-like name showing the stream's position in the hierarchy, using '::' as a separator between parent and child names.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
The fully resolved name including all parent names. |
description
property
¶
parent
property
¶
parent: Optional[DataStream]
Get the parent data stream.
Returns:
| Type | Description |
|---|---|
Optional[DataStream]
|
Optional[DataStream]: Parent data stream, or None if this is a root stream. |
is_collection
property
¶
is_collection: bool
Check if this data stream is a collection of other streams.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if this is a collection stream, False otherwise. |
reader_params
property
¶
reader_params: TReaderParams
Get the parameters for the data reader.
Returns:
| Name | Type | Description |
|---|---|---|
TReaderParams |
TReaderParams
|
Parameters for the data reader. |
at
property
¶
at: _AtProtocol
Get a child data stream by name.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
Name of the child data stream to retrieve. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
DataStream |
_AtProtocol
|
The child data stream with the given name. |
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
If the data stream does not support child access. |
Examples:
# Access stream in a collection
collection = data_collection.load()
temp_stream = collection.at("temperature")
# Or using dictionary-style syntax
humidity_stream = collection["humidity"]
has_data
property
¶
has_data: bool
Check if the data stream has loaded data.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if data has been loaded, False otherwise. |
has_error
property
¶
has_error: bool
Check if the data stream encountered an error during loading.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if an error occurred, False otherwise. |
data
property
¶
data: TData
Get the loaded data.
Returns:
| Name | Type | Description |
|---|---|---|
TData |
TData
|
The loaded data. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If data has not been loaded yet. |
set_parent ¶
set_parent(parent: DataStream) -> None
Set the parent data stream.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parent
|
DataStream
|
The parent data stream to set. |
required |
Source code in src/contraqctor/contract/base.py
164 165 166 167 168 169 170 | |
read ¶
read(
reader_params: Optional[TReaderParams] = None,
) -> TData
Read data using the configured reader.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
reader_params
|
Optional[TReaderParams]
|
Optional parameters to override the default reader parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
TData |
TData
|
Data read from the source. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If reader parameters are not set. |
Source code in src/contraqctor/contract/base.py
194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 | |
bind_reader_params ¶
bind_reader_params(params: TReaderParams) -> Self
Bind reader parameters to the data stream.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
params
|
TReaderParams
|
Parameters to bind to the data stream's reader. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If reader parameters have already been set. |
Source code in src/contraqctor/contract/base.py
211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 | |
clear ¶
clear() -> Self
Clear the loaded data from the data stream.
Resets the data to an unset state, allowing for reloading.
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Source code in src/contraqctor/contract/base.py
313 314 315 316 317 318 319 320 321 322 | |
load ¶
load() -> Self
Load data into the data stream.
Reads data from the source and stores it in the data stream.
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Examples:
from contraqctor.contract import csv
# Create and load a CSV stream
params = csv.CsvParams(path="data/measurements.csv")
csv_stream = csv.Csv("measurements", reader_params=params)
csv_stream.load()
# Access the data
df = csv_stream.data
print(f"Loaded {len(df)} rows")
Source code in src/contraqctor/contract/base.py
324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 | |
collect_errors ¶
collect_errors() -> List[ErrorOnLoad]
Collect all errors from this stream and its children.
Performs a depth-first traversal to gather all ErrorOnLoad instances.
Returns:
| Type | Description |
|---|---|
List[ErrorOnLoad]
|
List[ErrorOnLoad]: List of all errors raised on load encountered in the hierarchy. |
Source code in src/contraqctor/contract/base.py
379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 | |
load_all ¶
Recursively load this data stream and all child streams.
Performs depth-first traversal to load all streams in the hierarchy.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
strict
|
bool
|
If True, raises exceptions immediately; otherwise collects and returns them. |
False
|
Returns:
| Name | Type | Description |
|---|---|---|
list |
Self
|
List of tuples containing streams and exceptions that occurred during loading. |
Raises:
| Type | Description |
|---|---|
Exception
|
If strict is True and an exception occurs during loading. |
Examples:
# Load all streams and handle errors
errors = collection.load_all(strict=False)
if errors:
for stream, error in errors:
print(f"Error loading {stream.name}: {error}")
Source code in src/contraqctor/contract/base.py
396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 | |
PydanticModelParams
dataclass
¶
Bases: FilePathBaseParam, Generic[_TModel]
Parameters for Pydantic model-based JSON file processing.
Extends the base file path parameters with Pydantic model specification for parsing JSON into typed objects.
Attributes:
| Name | Type | Description |
|---|---|---|
model |
Type[_TModel]
|
Pydantic model class to use for parsing JSON data. |
encoding |
str
|
Character encoding for the JSON file. Defaults to UTF-8. |
Examples:
from pydantic import BaseModel
from contraqctor.contract.json import PydanticModelParams
class User(BaseModel):
user_id: str
name: str
active: bool = True
params = PydanticModelParams(path="users/profile.json", model=User)
PydanticModel ¶
PydanticModel(
name: str,
*,
description: Optional[str] = None,
reader_params: TReaderParams = UnsetParams,
**kwargs,
)
Bases: DataStream[_TModel, PydanticModelParams[_TModel]]
Pydantic model-based JSON data stream provider.
A data stream implementation for reading JSON files as Pydantic model instances.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
DataStream
|
Base class for data stream providers. |
required |
Examples:
from pydantic import BaseModel
from contraqctor.contract.json import PydanticModel, PydanticModelParams
class ServerConfig(BaseModel):
host: str
port: int
debug: bool = False
params = PydanticModelParams(path="config/server.json", model=ServerConfig)
config_stream = PydanticModel("server_config", reader_params=params).load()
server_config = config_stream.data
print(f"Server: {server_config.host}:{server_config.port}")
Source code in src/contraqctor/contract/base.py
103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | |
name
property
¶
name: str
Get the name of the data stream.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
Name identifier of the data stream. |
resolved_name
property
¶
resolved_name: str
Get the full hierarchical name of the data stream.
Generates a path-like name showing the stream's position in the hierarchy, using '::' as a separator between parent and child names.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
The fully resolved name including all parent names. |
description
property
¶
parent
property
¶
parent: Optional[DataStream]
Get the parent data stream.
Returns:
| Type | Description |
|---|---|
Optional[DataStream]
|
Optional[DataStream]: Parent data stream, or None if this is a root stream. |
is_collection
property
¶
is_collection: bool
Check if this data stream is a collection of other streams.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if this is a collection stream, False otherwise. |
reader_params
property
¶
reader_params: TReaderParams
Get the parameters for the data reader.
Returns:
| Name | Type | Description |
|---|---|---|
TReaderParams |
TReaderParams
|
Parameters for the data reader. |
at
property
¶
at: _AtProtocol
Get a child data stream by name.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
Name of the child data stream to retrieve. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
DataStream |
_AtProtocol
|
The child data stream with the given name. |
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
If the data stream does not support child access. |
Examples:
# Access stream in a collection
collection = data_collection.load()
temp_stream = collection.at("temperature")
# Or using dictionary-style syntax
humidity_stream = collection["humidity"]
has_data
property
¶
has_data: bool
Check if the data stream has loaded data.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if data has been loaded, False otherwise. |
has_error
property
¶
has_error: bool
Check if the data stream encountered an error during loading.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if an error occurred, False otherwise. |
data
property
¶
data: TData
Get the loaded data.
Returns:
| Name | Type | Description |
|---|---|---|
TData |
TData
|
The loaded data. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If data has not been loaded yet. |
set_parent ¶
set_parent(parent: DataStream) -> None
Set the parent data stream.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parent
|
DataStream
|
The parent data stream to set. |
required |
Source code in src/contraqctor/contract/base.py
164 165 166 167 168 169 170 | |
read ¶
read(
reader_params: Optional[TReaderParams] = None,
) -> TData
Read data using the configured reader.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
reader_params
|
Optional[TReaderParams]
|
Optional parameters to override the default reader parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
TData |
TData
|
Data read from the source. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If reader parameters are not set. |
Source code in src/contraqctor/contract/base.py
194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 | |
bind_reader_params ¶
bind_reader_params(params: TReaderParams) -> Self
Bind reader parameters to the data stream.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
params
|
TReaderParams
|
Parameters to bind to the data stream's reader. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If reader parameters have already been set. |
Source code in src/contraqctor/contract/base.py
211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 | |
clear ¶
clear() -> Self
Clear the loaded data from the data stream.
Resets the data to an unset state, allowing for reloading.
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Source code in src/contraqctor/contract/base.py
313 314 315 316 317 318 319 320 321 322 | |
load ¶
load() -> Self
Load data into the data stream.
Reads data from the source and stores it in the data stream.
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Examples:
from contraqctor.contract import csv
# Create and load a CSV stream
params = csv.CsvParams(path="data/measurements.csv")
csv_stream = csv.Csv("measurements", reader_params=params)
csv_stream.load()
# Access the data
df = csv_stream.data
print(f"Loaded {len(df)} rows")
Source code in src/contraqctor/contract/base.py
324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 | |
collect_errors ¶
collect_errors() -> List[ErrorOnLoad]
Collect all errors from this stream and its children.
Performs a depth-first traversal to gather all ErrorOnLoad instances.
Returns:
| Type | Description |
|---|---|
List[ErrorOnLoad]
|
List[ErrorOnLoad]: List of all errors raised on load encountered in the hierarchy. |
Source code in src/contraqctor/contract/base.py
379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 | |
load_all ¶
Recursively load this data stream and all child streams.
Performs depth-first traversal to load all streams in the hierarchy.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
strict
|
bool
|
If True, raises exceptions immediately; otherwise collects and returns them. |
False
|
Returns:
| Name | Type | Description |
|---|---|---|
list |
Self
|
List of tuples containing streams and exceptions that occurred during loading. |
Raises:
| Type | Description |
|---|---|
Exception
|
If strict is True and an exception occurs during loading. |
Examples:
# Load all streams and handle errors
errors = collection.load_all(strict=False)
if errors:
for stream, error in errors:
print(f"Error loading {stream.name}: {error}")
Source code in src/contraqctor/contract/base.py
396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 | |
ManyPydanticModelParams
dataclass
¶
ManyPydanticModelParams(
path: PathLike,
model: Type[_TModel],
encoding: str = "UTF-8",
index: Optional[str] = None,
column_names: Optional[dict[str, str]] = None,
)
Bases: FilePathBaseParam, Generic[_TModel]
Parameters for loading multiple Pydantic models from a file.
Extends the base file path parameters with Pydantic model specification and options for converting to a DataFrame.
Attributes:
| Name | Type | Description |
|---|---|---|
model |
Type[_TModel]
|
Pydantic model class to use for parsing JSON data. |
encoding |
str
|
Character encoding for the JSON file. Defaults to UTF-8. |
index |
Optional[str]
|
Optional column name to set as the DataFrame index. |
column_names |
Optional[dict[str, str]]
|
Optional dictionary mapping original column names to new names. |
Examples:
Defining parameters to load multiple models:
from pydantic import BaseModel
from contraqctor.contract.json import ManyPydanticModelParams
# Define a Pydantic model for log entries
class LogEntry(BaseModel):
timestamp: str
level: str
message: str
# Create parameters for loading log entries
params = ManyPydanticModelParams(
path="logs/server_logs.json",
model=LogEntry,
index="timestamp",
column_names={"level": "log_level", "message": "log_message"}
)
ManyPydanticModel ¶
ManyPydanticModel(
name: str,
*,
description: Optional[str] = None,
reader_params: TReaderParams = UnsetParams,
**kwargs,
)
Bases: DataStream[DataFrame, ManyPydanticModelParams[_TModel]]
Multi-model JSON data stream provider.
A data stream implementation for reading multiple JSON objects from a file, parsing them as Pydantic models, and returning them as a DataFrame.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
DataStream
|
Base class for data stream providers. |
required |
Examples:
Loading server logs into a DataFrame:
from contraqctor.contract.json import ManyPydanticModel, ManyPydanticModelParams
# Create and load the data stream
logs_stream = ManyPydanticModel(
"server_logs_df",
reader_params=params
)
logs_stream.load()
# Access the logs as a DataFrame
logs_df = logs_stream.data
# Analyze the logs
error_logs = logs_df[logs_df["log_level"] == "ERROR"]
Source code in src/contraqctor/contract/base.py
103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | |
name
property
¶
name: str
Get the name of the data stream.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
Name identifier of the data stream. |
resolved_name
property
¶
resolved_name: str
Get the full hierarchical name of the data stream.
Generates a path-like name showing the stream's position in the hierarchy, using '::' as a separator between parent and child names.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
The fully resolved name including all parent names. |
description
property
¶
parent
property
¶
parent: Optional[DataStream]
Get the parent data stream.
Returns:
| Type | Description |
|---|---|
Optional[DataStream]
|
Optional[DataStream]: Parent data stream, or None if this is a root stream. |
is_collection
property
¶
is_collection: bool
Check if this data stream is a collection of other streams.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if this is a collection stream, False otherwise. |
reader_params
property
¶
reader_params: TReaderParams
Get the parameters for the data reader.
Returns:
| Name | Type | Description |
|---|---|---|
TReaderParams |
TReaderParams
|
Parameters for the data reader. |
at
property
¶
at: _AtProtocol
Get a child data stream by name.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
Name of the child data stream to retrieve. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
DataStream |
_AtProtocol
|
The child data stream with the given name. |
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
If the data stream does not support child access. |
Examples:
# Access stream in a collection
collection = data_collection.load()
temp_stream = collection.at("temperature")
# Or using dictionary-style syntax
humidity_stream = collection["humidity"]
has_data
property
¶
has_data: bool
Check if the data stream has loaded data.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if data has been loaded, False otherwise. |
has_error
property
¶
has_error: bool
Check if the data stream encountered an error during loading.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if an error occurred, False otherwise. |
data
property
¶
data: TData
Get the loaded data.
Returns:
| Name | Type | Description |
|---|---|---|
TData |
TData
|
The loaded data. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If data has not been loaded yet. |
set_parent ¶
set_parent(parent: DataStream) -> None
Set the parent data stream.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parent
|
DataStream
|
The parent data stream to set. |
required |
Source code in src/contraqctor/contract/base.py
164 165 166 167 168 169 170 | |
read ¶
read(
reader_params: Optional[TReaderParams] = None,
) -> TData
Read data using the configured reader.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
reader_params
|
Optional[TReaderParams]
|
Optional parameters to override the default reader parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
TData |
TData
|
Data read from the source. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If reader parameters are not set. |
Source code in src/contraqctor/contract/base.py
194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 | |
bind_reader_params ¶
bind_reader_params(params: TReaderParams) -> Self
Bind reader parameters to the data stream.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
params
|
TReaderParams
|
Parameters to bind to the data stream's reader. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If reader parameters have already been set. |
Source code in src/contraqctor/contract/base.py
211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 | |
clear ¶
clear() -> Self
Clear the loaded data from the data stream.
Resets the data to an unset state, allowing for reloading.
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Source code in src/contraqctor/contract/base.py
313 314 315 316 317 318 319 320 321 322 | |
load ¶
load() -> Self
Load data into the data stream.
Reads data from the source and stores it in the data stream.
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Examples:
from contraqctor.contract import csv
# Create and load a CSV stream
params = csv.CsvParams(path="data/measurements.csv")
csv_stream = csv.Csv("measurements", reader_params=params)
csv_stream.load()
# Access the data
df = csv_stream.data
print(f"Loaded {len(df)} rows")
Source code in src/contraqctor/contract/base.py
324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 | |
collect_errors ¶
collect_errors() -> List[ErrorOnLoad]
Collect all errors from this stream and its children.
Performs a depth-first traversal to gather all ErrorOnLoad instances.
Returns:
| Type | Description |
|---|---|
List[ErrorOnLoad]
|
List[ErrorOnLoad]: List of all errors raised on load encountered in the hierarchy. |
Source code in src/contraqctor/contract/base.py
379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 | |
load_all ¶
Recursively load this data stream and all child streams.
Performs depth-first traversal to load all streams in the hierarchy.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
strict
|
bool
|
If True, raises exceptions immediately; otherwise collects and returns them. |
False
|
Returns:
| Name | Type | Description |
|---|---|---|
list |
Self
|
List of tuples containing streams and exceptions that occurred during loading. |
Raises:
| Type | Description |
|---|---|
Exception
|
If strict is True and an exception occurs during loading. |
Examples:
# Load all streams and handle errors
errors = collection.load_all(strict=False)
if errors:
for stream, error in errors:
print(f"Error loading {stream.name}: {error}")
Source code in src/contraqctor/contract/base.py
396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 | |
SoftwareEventsParams
dataclass
¶
SoftwareEventsParams(
path: PathLike,
model: Type[_TModel],
encoding: str = "UTF-8",
index: Optional[str] = "timestamp",
column_names: Optional[dict[str, str]] = None,
)
Bases: ManyPydanticModelParams
Parameters for software events file processing.
A specialized version of ManyPydanticModelParams that defaults to using the SoftwareEvent model from aind_behavior_services.
Attributes:
| Name | Type | Description |
|---|---|---|
model |
Type[SoftwareEvent]
|
Set to SoftwareEvent model and not modifiable after initialization. |
encoding |
str
|
Character encoding for the JSON file. Defaults to UTF-8. |
index |
Optional[str]
|
Optional column name to set as the DataFrame index. |
column_names |
Optional[dict[str, str]]
|
Optional dictionary mapping original column names to new names. |
Examples:
Defining parameters for loading software events:
from contraqctor.contract.json import SoftwareEventsParams
# Create parameters for software events
params = SoftwareEventsParams(
path="events/software_events.json",
index="event_id",
column_names={"timestamp": "event_time"}
)
SoftwareEvents ¶
SoftwareEvents(
name: str,
*,
description: Optional[str] = None,
reader_params: TReaderParams = UnsetParams,
**kwargs,
)
Bases: ManyPydanticModel[SoftwareEvent]
Software events data stream provider.
A specialized data stream for reading software event logs from JSON files using the SoftwareEvent model from aind_behavior_services.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
ManyPydanticModel
|
Base class for multi-model data stream providers. |
required |
Examples:
Analyzing software events data:
from contraqctor.contract.json import SoftwareEvents, SoftwareEventsParams
# Create parameters for software events
params = SoftwareEventsParams(
path="events/software_events.json",
index="event_id"
)
# Create and load the software events stream
events_stream = SoftwareEvents(
"software_events",
reader_params=params
)
events_stream.load()
# Access the events data
events_df = events_stream.data
# Perform analysis, e.g., count events by type
event_counts = events_df["event_type"].value_counts()
Source code in src/contraqctor/contract/base.py
103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | |
name
property
¶
name: str
Get the name of the data stream.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
Name identifier of the data stream. |
resolved_name
property
¶
resolved_name: str
Get the full hierarchical name of the data stream.
Generates a path-like name showing the stream's position in the hierarchy, using '::' as a separator between parent and child names.
Returns:
| Name | Type | Description |
|---|---|---|
str |
str
|
The fully resolved name including all parent names. |
description
property
¶
parent
property
¶
parent: Optional[DataStream]
Get the parent data stream.
Returns:
| Type | Description |
|---|---|
Optional[DataStream]
|
Optional[DataStream]: Parent data stream, or None if this is a root stream. |
is_collection
property
¶
is_collection: bool
Check if this data stream is a collection of other streams.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if this is a collection stream, False otherwise. |
reader_params
property
¶
reader_params: TReaderParams
Get the parameters for the data reader.
Returns:
| Name | Type | Description |
|---|---|---|
TReaderParams |
TReaderParams
|
Parameters for the data reader. |
at
property
¶
at: _AtProtocol
Get a child data stream by name.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
Name of the child data stream to retrieve. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
DataStream |
_AtProtocol
|
The child data stream with the given name. |
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
If the data stream does not support child access. |
Examples:
# Access stream in a collection
collection = data_collection.load()
temp_stream = collection.at("temperature")
# Or using dictionary-style syntax
humidity_stream = collection["humidity"]
has_data
property
¶
has_data: bool
Check if the data stream has loaded data.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if data has been loaded, False otherwise. |
has_error
property
¶
has_error: bool
Check if the data stream encountered an error during loading.
Returns:
| Name | Type | Description |
|---|---|---|
bool |
bool
|
True if an error occurred, False otherwise. |
data
property
¶
data: TData
Get the loaded data.
Returns:
| Name | Type | Description |
|---|---|---|
TData |
TData
|
The loaded data. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If data has not been loaded yet. |
set_parent ¶
set_parent(parent: DataStream) -> None
Set the parent data stream.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
parent
|
DataStream
|
The parent data stream to set. |
required |
Source code in src/contraqctor/contract/base.py
164 165 166 167 168 169 170 | |
read ¶
read(
reader_params: Optional[TReaderParams] = None,
) -> TData
Read data using the configured reader.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
reader_params
|
Optional[TReaderParams]
|
Optional parameters to override the default reader parameters. |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
TData |
TData
|
Data read from the source. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If reader parameters are not set. |
Source code in src/contraqctor/contract/base.py
194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 | |
bind_reader_params ¶
bind_reader_params(params: TReaderParams) -> Self
Bind reader parameters to the data stream.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
params
|
TReaderParams
|
Parameters to bind to the data stream's reader. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If reader parameters have already been set. |
Source code in src/contraqctor/contract/base.py
211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 | |
clear ¶
clear() -> Self
Clear the loaded data from the data stream.
Resets the data to an unset state, allowing for reloading.
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Source code in src/contraqctor/contract/base.py
313 314 315 316 317 318 319 320 321 322 | |
load ¶
load() -> Self
Load data into the data stream.
Reads data from the source and stores it in the data stream.
Returns:
| Name | Type | Description |
|---|---|---|
Self |
Self
|
The data stream instance for method chaining. |
Examples:
from contraqctor.contract import csv
# Create and load a CSV stream
params = csv.CsvParams(path="data/measurements.csv")
csv_stream = csv.Csv("measurements", reader_params=params)
csv_stream.load()
# Access the data
df = csv_stream.data
print(f"Loaded {len(df)} rows")
Source code in src/contraqctor/contract/base.py
324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 | |
collect_errors ¶
collect_errors() -> List[ErrorOnLoad]
Collect all errors from this stream and its children.
Performs a depth-first traversal to gather all ErrorOnLoad instances.
Returns:
| Type | Description |
|---|---|
List[ErrorOnLoad]
|
List[ErrorOnLoad]: List of all errors raised on load encountered in the hierarchy. |
Source code in src/contraqctor/contract/base.py
379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 | |
load_all ¶
Recursively load this data stream and all child streams.
Performs depth-first traversal to load all streams in the hierarchy.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
strict
|
bool
|
If True, raises exceptions immediately; otherwise collects and returns them. |
False
|
Returns:
| Name | Type | Description |
|---|---|---|
list |
Self
|
List of tuples containing streams and exceptions that occurred during loading. |
Raises:
| Type | Description |
|---|---|
Exception
|
If strict is True and an exception occurs during loading. |
Examples:
# Load all streams and handle errors
errors = collection.load_all(strict=False)
if errors:
for stream, error in errors:
print(f"Error loading {stream.name}: {error}")
Source code in src/contraqctor/contract/base.py
396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 | |