Activation Layers

Activation types and layers.

miopenActivationMode_t

enum miopenActivationMode_t

Activation layer modes

Values:

enumerator miopenActivationPASTHRU

No activation, pass through the data

enumerator miopenActivationLOGISTIC

Sigmoid function: \(1 / (1 + e^{-x})\)

enumerator miopenActivationTANH

Tanh activation \( \beta * tanh( \alpha * x) \)

enumerator miopenActivationRELU

Rectified Linear Unit \( max(0, x) \)

enumerator miopenActivationSOFTRELU

\(log(1 + e^x)\)

enumerator miopenActivationABS

Absolute value \(abs(x)\)

enumerator miopenActivationPOWER

Scaled and shifted power \((\alpha + \beta * x)^{gamma}\)

enumerator miopenActivationCLIPPEDRELU

Clipped Rectified Linear Unit \( min(\alpha, max(0,x)) \)

enumerator miopenActivationLEAKYRELU

Leaky Rectified Linear Unit \( \alpha * x | x <= 0; x | x > 0 \)

enumerator miopenActivationELU

Exponential Rectified Linear Unit \( \alpha * (e^{x} - 1) | x <= 0; x | x > 0 \)

miopenCreateActivationDescriptor

miopenStatus_t miopenCreateActivationDescriptor(miopenActivationDescriptor_t *activDesc)

Creates the Activation descriptor object.

Parameters

activDesc – Pointer to an activation tensor descriptor type

Returns

miopenStatus_t

miopenSetActivationDescriptor

miopenStatus_t miopenSetActivationDescriptor(const miopenActivationDescriptor_t activDesc, miopenActivationMode_t mode, double activAlpha, double activBeta, double activGamma)

Sets the activation layer descriptor details.

Sets all of the descriptor details for the activation layer

Parameters

  • activDesc – Pointer to a activation layer descriptor (output)

  • mode – Activation mode enum (input)

  • activAlpha – Alpha value for some activation modes (input)

  • activBeta – Beta value for some activation modes (input)

  • activGamma – Gamma value for some activation modes (input)

Returns

miopenStatus_t

miopenGetActivationDescriptor

miopenStatus_t miopenGetActivationDescriptor(const miopenActivationDescriptor_t activDesc, miopenActivationMode_t *mode, double *activAlpha, double *activBeta, double *activGamma)

Gets the activation layer descriptor details.

Retrieves all of the descriptor details for the activation layer.

Parameters

  • activDesc – Pointer to a activation layer descriptor (input)

  • mode – Activation mode enum (output)

  • activAlpha – Alpha value for some activation modes (output)

  • activBeta – Beta value for some activation modes (output)

  • activGamma – Gamma value for some activation modes (output)

Returns

miopenStatus_t

miopenActivationForward

miopenStatus_t miopenActivationForward(miopenHandle_t handle, const miopenActivationDescriptor_t activDesc, const void *alpha, const miopenTensorDescriptor_t xDesc, const void *x, const void *beta, const miopenTensorDescriptor_t yDesc, void *y)

Execute an activation forward layer.

Parameters

  • handle – MIOpen handle (input)

  • activDesc – Descriptor for activation layer (input)

  • alpha – Floating point scaling factor, allocated on the host (input)

  • xDesc – Tensor descriptor for data input tensor x (input)

  • x – Data tensor x (input)

  • beta – Floating point shift factor, allocated on the host (input)

  • yDesc – Tensor descriptor for output data tensor y (input)

  • y – Data tensor y (output)

Returns

miopenStatus_t

miopenActivationBackward

miopenStatus_t miopenActivationBackward(miopenHandle_t handle, const miopenActivationDescriptor_t activDesc, const void *alpha, const miopenTensorDescriptor_t yDesc, const void *y, const miopenTensorDescriptor_t dyDesc, const void *dy, const miopenTensorDescriptor_t xDesc, const void *x, const void *beta, const miopenTensorDescriptor_t dxDesc, void *dx)

Execute a activation backwards layer.

Parameters

  • handle – MIOpen handle (input)

  • activDesc – Descriptor for activation layer (input)

  • alpha – Floating point scaling factor, allocated on the host (input)

  • yDesc – Tensor descriptor for input data tensor y (input)

  • y – Data tensor y (input)

  • dyDesc – Tensor descriptor for input data tensor dy (input)

  • dy – Data delta tensor dy (input)

  • xDesc – Tensor descriptor for data input tensor x (input)

  • x – Data tensor x (input)

  • beta – Floating point shift factor, allocated on the host (input)

  • dxDesc – Tensor descriptor for data output tensor dx (input)

  • dx – Output data delta tensor dx (output)

Returns

miopenStatus_t

miopenDestroyActivationDescriptor

miopenStatus_t miopenDestroyActivationDescriptor(miopenActivationDescriptor_t activDesc)

Destroys the activation descriptor object.

Parameters

activDesc – Activation tensor descriptor type (input)

Returns

miopenStatus_t