deepdrivemd.data.stream.adios_utils

Utility functions for ADIOS2.

Classes

AdiosStreamStepRW(connections, variables)

Read/Write step by step adios stream using Full API.
class deepdrivemd.data.stream.adios_utils.AdiosStreamStepRW(connections: Dict[int, Tuple[adios2.adios2.ADIOS, adios2.adios2.IO, adios2.adios2.Engine]], variables: Dict[str, Tuple[type, deepdrivemd.data.stream.enumerations.DataStructure]])

Read/Write step by step adios stream using Full API.

connections

dictionary of adios connections; key - integer, in aggregator it is simulation task id; value - a tuple of adios objects

Type

Dict[int, Tuple[adios2.adios2.ADIOS, adios2.adios2.IO, adios2.adios2.Engine]]

variables

dictionary describing variables; key - adios column name, value - a tuple of variable type and enumeration describing the structure type: scalar, numpy array, string; other class attributes are created on the fly using setattr: for each key two attributes are created: var_<key> - adios variable, d_<key> - data which stores the result of reading a particular variable key from a step of adios stream.

Type

Dict[str, Tuple[type, DataStructure]],

__init__(connections: Dict[int, Tuple[adios2.adios2.ADIOS, adios2.adios2.IO, adios2.adios2.Engine]], variables: Dict[str, Tuple[type, deepdrivemd.data.stream.enumerations.DataStructure]])

Initialize AdiosStreamStepRW object.

Parameters

  • connections (Dict[int, Tuple[adios2.adios2.ADIOS, adios2.adios2.IO, adios2.adios2.Engine]]) – dictionary of adios connections; key - integer, in aggregator it is simulation task id, value - a tuple of adios objects

  • variables (Dict[str, Tuple[type, DataStructure]]) – dictionary describing variables; key - adios column name, value - a tuple of variable type and enumeration describing the structure type: scalar, numpy array, string.

read_step(sim_task_id: int) → bool

Read the next step from adios stream given by connections[sim_task_id].

Parameters

sim_task_id (int) – is used as a key to get the corresponding adios objects from connections

Returns

bool – True if reading a step succeeded, False - otherwise.

write_step(wstream: adios2.adios2.Engine, variables: Dict[str, Tuple[type, deepdrivemd.data.stream.enumerations.DataStructure]], end_step: bool = False)

Write the next step from class d_… variables into wstream adios stream.

Parameters

  • wstream (adios2.adios2.Engine) – adios stream to which the data is written

  • variables (Dict[str, Tuple[type, DataStructure]]) – a dictionary indexed by adios column names, value is a tuple - data type, structure type; structure type can be scalar, array, string

  • end_step (bool, default = False) – if this is True, the write of the last variable would be marked by end_step = True meaning that the step writing is done; otherwise, terminating the step should be done outside of the method