dedalus.core.distributor
Classes for available data layouts and the paths between them.
Module Contents
- logger
- GROUP_TRANSFORMS
- TRANSPOSE_LIBRARY
- GROUP_TRANSPOSES
- SYNC_TRANSPOSES
- ALLTOALLV
- class Distributor(domain, comm=None, mesh=None)
Directs parallelized distribution and transformation of fields over a domain.
- Variables
comm (MPI communicator) – Global MPI communicator
rank (int) – Internal MPI process number
size (int) – Number of MPI processes
mesh (tuple of ints, optional) – Process mesh for parallelization (default: 1-D mesh of available processes)
comm_cart (MPI communicator) – Cartesian MPI communicator over mesh
coords (array of ints) – Coordinates in cartesian communicator (None if outside mesh)
layouts (list of layout objects) – Available layouts for domain
paths (list of path objects) – Transforms and transposes between layouts
Notes
Computations are parallelized by splitting D-dimensional data fields over an R-dimensional mesh of MPI processes, where R < D. In coefficient space, we take the first R dimensions of the data to be distributed over the mesh, leaving the last (D-R) dimensions local. To transform such a data cube to grid space, we loop backwards over the D dimensions, performing each transform if the corresponding dimension is local, and performing an MPI transpose with the next dimension otherwise. This effectively bubbles the first local dimension up from the (D-R)-th to the first dimension, transforming to grid space along the way. In grid space, then, the first dimensional is local, followed by R dimensions distributed over the mesh, and the last (D-R-1) dimensions local.
The distributor object for a given domain constructs layout objects describing each of the (D+R+1) layouts (sets of transform/distribution states) and the paths between them (D transforms and R transposes).
- get_layout_object(input)
Dereference layout identifiers.
- buffer_size(scales)
Compute necessary buffer size (bytes) for all layouts.
- class Layout(domain, mesh, coords, local, grid_space, dtype)
Object describing the data distribution for a given transform and distribution state.
- Variables
local (array of bools) – Axis locality flags (True/False for local/distributed)
grid_space (array of bools) – Axis grid-space flags (True/False for grid/coeff space)
dtype (numeric type) – Data type
scales. (All methods require a tuple of the current transform) –
- global_shape(scales)
Compute global data shape.
- blocks(scales)
Compute block sizes for data distribution.
- start(scales)
Compute starting coordinates for local data.
- local_shape(scales)
Compute local data shape.
- slices(scales)
Compute slices for selecting local portion of global data.
- buffer_size(scales)
Compute necessary buffer size (bytes).
- class Transform(layout0, layout1, axis, basis)
Directs transforms between two layouts.
- group_data(nfields, scales)
- increment_group(fields)
- decrement_group(fields)
- increment_single(field)
Backward transform.
- decrement_single(field)
Forward transform.
- increment(fields)
Backward transform.
- decrement(fields)
Forward transform.
- class Transpose(layout0, layout1, axis, comm_cart)
Directs transposes between two layouts.
- increment(fields)
Transpose from layout0 to layout1.
- decrement(fields)
Transpose from layout1 to layout0.
- increment_single(field)
Transpose field from layout0 to layout1.
- decrement_single(field)
Transpose field from layout1 to layout0.
- increment_group(*fields)
Transpose group from layout0 to layout1.
- decrement_group(*fields)
Transpose group from layout1 to layout0.