dedalus.core.distributor

Classes for available data layouts and the paths between them.

Module Contents

logger
GROUP_TRANSFORMS
TRANSPOSE_LIBRARY
GROUP_TRANSPOSES
SYNC_TRANSPOSES
ALLTOALLV
class Distributor(domain, comm=None, mesh=None)

Directs parallelized distribution and transformation of fields over a domain.

Variables
  • comm (MPI communicator) – Global MPI communicator

  • rank (int) – Internal MPI process number

  • size (int) – Number of MPI processes

  • mesh (tuple of ints, optional) – Process mesh for parallelization (default: 1-D mesh of available processes)

  • comm_cart (MPI communicator) – Cartesian MPI communicator over mesh

  • coords (array of ints) – Coordinates in cartesian communicator (None if outside mesh)

  • layouts (list of layout objects) – Available layouts for domain

  • paths (list of path objects) – Transforms and transposes between layouts

Notes

Computations are parallelized by splitting D-dimensional data fields over an R-dimensional mesh of MPI processes, where R < D. In coefficient space, we take the first R dimensions of the data to be distributed over the mesh, leaving the last (D-R) dimensions local. To transform such a data cube to grid space, we loop backwards over the D dimensions, performing each transform if the corresponding dimension is local, and performing an MPI transpose with the next dimension otherwise. This effectively bubbles the first local dimension up from the (D-R)-th to the first dimension, transforming to grid space along the way. In grid space, then, the first dimensional is local, followed by R dimensions distributed over the mesh, and the last (D-R-1) dimensions local.

The distributor object for a given domain constructs layout objects describing each of the (D+R+1) layouts (sets of transform/distribution states) and the paths between them (D transforms and R transposes).

get_layout_object(input)

Dereference layout identifiers.

buffer_size(scales)

Compute necessary buffer size (bytes) for all layouts.

class Layout(domain, mesh, coords, local, grid_space, dtype)

Object describing the data distribution for a given transform and distribution state.

Variables
  • local (array of bools) – Axis locality flags (True/False for local/distributed)

  • grid_space (array of bools) – Axis grid-space flags (True/False for grid/coeff space)

  • dtype (numeric type) – Data type

  • scales. (All methods require a tuple of the current transform) –

global_shape(scales)

Compute global data shape.

blocks(scales)

Compute block sizes for data distribution.

start(scales)

Compute starting coordinates for local data.

local_shape(scales)

Compute local data shape.

slices(scales)

Compute slices for selecting local portion of global data.

buffer_size(scales)

Compute necessary buffer size (bytes).

class Transform(layout0, layout1, axis, basis)

Directs transforms between two layouts.

group_data(nfields, scales)
increment_group(fields)
decrement_group(fields)
increment_single(field)

Backward transform.

decrement_single(field)

Forward transform.

increment(fields)

Backward transform.

decrement(fields)

Forward transform.

class Transpose(layout0, layout1, axis, comm_cart)

Directs transposes between two layouts.

increment(fields)

Transpose from layout0 to layout1.

decrement(fields)

Transpose from layout1 to layout0.

increment_single(field)

Transpose field from layout0 to layout1.

decrement_single(field)

Transpose field from layout1 to layout0.

increment_group(*fields)

Transpose group from layout0 to layout1.

decrement_group(*fields)

Transpose group from layout1 to layout0.