Layer Fusion

GPU kernel fusion API reference

miopenFusionDirection_t

enum miopenFusionDirection_t

Kernel fusion direction in the network.

Values:

enumerator miopenVerticalFusion

fuses layers vertically, current the only supported mode

enumerator miopenHorizontalFusion

fuses layers horizontally, this is unimplemented

miopenCreateFusionPlan

miopenStatus_t miopenCreateFusionPlan(miopenFusionPlanDescriptor_t *fusePlanDesc, const miopenFusionDirection_t fuseDirection, const miopenTensorDescriptor_t inputDesc)

Creates the kenrel fusion plan descriptor object.

Parameters

  • fusePlanDesc – Pointer to a fusion plan (output)

  • fuseDirection – Horizontal or Vertical fusion (input)

  • inputDesc – Descriptor to tensor for the input (input)

Returns

miopenStatus_t

miopenDestroyFusionPlan

miopenStatus_t miopenDestroyFusionPlan(miopenFusionPlanDescriptor_t fusePlanDesc)

Destroy the fusion plan descriptor object.

Parameters

fusePlanDesc – A fusion plan descriptor type

Returns

miopenStatus_t

miopenCompileFusionPlan

miopenStatus_t miopenCompileFusionPlan(miopenHandle_t handle, miopenFusionPlanDescriptor_t fusePlanDesc)

Compiles the fusion plan.

Parameters

  • handle – MIOpen handle (input)

  • fusePlanDesc – A fusion plan descriptor (input)

Returns

miopenStatus_t

miopenFusionPlanGetOp

miopenStatus_t miopenFusionPlanGetOp(miopenFusionPlanDescriptor_t fusePlanDesc, const int op_idx, miopenFusionOpDescriptor_t *op)

Allows access to the operators in a fusion plan.

This api call does bounds checking on the supplied op_idx and would return miopenStatusError if the index is out of bounds

Parameters

  • fusePlanDesc – A fusion plan descriptor (input)

  • op_idx – Index of the required operator in the fusion plan, in the order of insertion

  • op – returned pointer to the operator

Returns

miopenStatus_t

miopenFusionPlanGetWorkSpaceSize

miopenStatus_t miopenFusionPlanGetWorkSpaceSize(miopenHandle_t handle, miopenFusionPlanDescriptor_t fusePlanDesc, size_t *workSpaceSize, miopenConvFwdAlgorithm_t algo)

Query the workspace size required for the fusion plan.

Parameters

  • fusePlanDesc – A fusion plan descriptor (input)

  • workSpaceSize – Pointer to memory to return size in bytes (output)

Returns

miopenStatus_t

miopenFusionPlanConvolutionGetAlgo

miopenStatus_t miopenFusionPlanConvolutionGetAlgo(miopenFusionPlanDescriptor_t fusePlanDesc, const int requestAlgoCount, int *returnedAlgoCount, miopenConvFwdAlgorithm_t *returnedAlgos)

Returns the supported algorithms for the convolution operator in the Fusion Plan.

A Convolution operator in a fusion plan may be implemented by different algorithms representing different tradeoffs of memory and performance. The returned list of algorithms is sorted in decreasing order of priority. Therefore, if the user does not request an algorithm to be set using the miopenFusionPlanConvolutionSetAlgo call, the first algorithm in the list would be used to execute the convolution in the fusion plan. Moreover this call must be immediately preceded by the miopenCreateOpConvForward call for the op in question.

Parameters

  • fusePlanDesc – A fusion plan descriptor (input)

  • requestAlgoCount – Number of algorithms to return (input)

  • returnedAlgoCount – The actual number of returned algorithms; always be less than equal to requestAlgoCount (output)

  • returnedAlgos – Pointer to the list of supported algorithms

Returns

miopenStatus_t

miopenCreateOpConvForward

miopenStatus_t miopenCreateOpConvForward(miopenFusionPlanDescriptor_t fusePlanDesc, miopenFusionOpDescriptor_t *convOp, miopenConvolutionDescriptor_t convDesc, const miopenTensorDescriptor_t wDesc)

Creates forward convolution operator.

Parameters

  • fusePlanDesc – A fusion plan descriptor (input)

  • convOp – Pointer to an operator type (output)

  • convDesc – Convolution layer descriptor (input)

  • wDesc – Descriptor for the weights tensor (input)

Returns

miopenStatus_t

miopenCreateOpActivationForward

miopenStatus_t miopenCreateOpActivationForward(miopenFusionPlanDescriptor_t fusePlanDesc, miopenFusionOpDescriptor_t *activFwdOp, miopenActivationMode_t mode)

Creates a forward activation operator.

Parameters

  • fusePlanDesc – A fusion plan descriptor (input)

  • activFwdOp – Pointer to an operator type (output)

  • mode – Activation version (input)

Returns

miopenStatus_t

miopenCreateOpBiasForward

miopenStatus_t miopenCreateOpBiasForward(miopenFusionPlanDescriptor_t fusePlanDesc, miopenFusionOpDescriptor_t *biasOp, const miopenTensorDescriptor_t bDesc)

Creates a forward bias operator.

Parameters

  • fusePlanDesc – A fusion plan descriptor (input)

  • biasOp – Pointer to an operator type (output)

  • bDesc – bias tensor descriptor (input)

Returns

miopenStatus_t

miopenCreateOpBatchNormInference

miopenStatus_t miopenCreateOpBatchNormInference(miopenFusionPlanDescriptor_t fusePlanDesc, miopenFusionOpDescriptor_t *bnOp, const miopenBatchNormMode_t bn_mode, const miopenTensorDescriptor_t bnScaleBiasMeanVarDesc)

Creates a forward inference batch normalization operator.

Parameters

  • fusePlanDesc – A fusion plan descriptor (input)

  • bnOp – Pointer to an operator type (output)

  • bn_mode – Batch normalization layer mode (input)

  • bnScaleBiasMeanVarDesc – Gamma, beta, mean, variance tensor descriptor (input)

Returns

miopenStatus_t

miopenCreateOperatorArgs

miopenStatus_t miopenCreateOperatorArgs(miopenOperatorArgs_t *args)

Creates an operator argument object.

Parameters

args – Pointer to an operator argument type (output)

Returns

miopenStatus_t

miopenDestroyOperatorArgs

miopenStatus_t miopenDestroyOperatorArgs(miopenOperatorArgs_t args)

Destroys an operator argument object.

Parameters

args – An operator argument type (output)

Returns

miopenStatus_t

miopenSetOpArgsConvForward

miopenStatus_t miopenSetOpArgsConvForward(miopenOperatorArgs_t args, const miopenFusionOpDescriptor_t convOp, const void *alpha, const void *beta, const void *w)

Sets the arguments for forward convolution op.

Parameters

  • args – An arguments object type (output)

  • convOp – Forward convolution operator (input)

  • alpha – Floating point scaling factor, allocated on the host (input)

  • beta – Floating point shift factor, allocated on the host (input)

  • w – Pointer to tensor memory (input)

Returns

miopenStatus_t

miopenSetOpArgsBatchNormInference

miopenStatus_t miopenSetOpArgsBatchNormInference(miopenOperatorArgs_t args, const miopenFusionOpDescriptor_t bnOp, const void *alpha, const void *beta, const void *bnScale, const void *bnBias, const void *estimatedMean, const void *estimatedVariance, double epsilon)

Sets the arguments for inference batch normalization op.

Parameters

  • args – An arguments object type (output)

  • bnOp – Batch normalization inference operator (input)

  • alpha – Floating point scaling factor, allocated on the host (input)

  • beta – Floating point shift factor, allocated on the host (input)

  • bnScale – Pointer to the gamma tensor memory (input)

  • bnBias – Pointer to the beta tensor memory (input)

  • estimatedMean – Pointer to population mean memory (input)

  • estimatedVariance – Pointer to population variance memory (input)

  • epsilon – Scalar value for numerical stability (input)

Returns

miopenStatus_t

miopenSetOpArgsBiasForward

miopenStatus_t miopenSetOpArgsBiasForward(miopenOperatorArgs_t args, const miopenFusionOpDescriptor_t biasOp, const void *alpha, const void *beta, const void *bias)

Sets the arguments for forward bias op.

Parameters

  • args – An arguments object type (output)

  • biasOp – Forward bias operator (input)

  • alpha – Floating point scaling factor, allocated on the host (input)

  • beta – Floating point shift factor, allocated on the host (input)

  • bias – Pointer to the forward bias input tensor memory (input)

Returns

miopenStatus_t

miopenExecuteFusionPlan

miopenStatus_t miopenExecuteFusionPlan(const miopenHandle_t handle, const miopenFusionPlanDescriptor_t fusePlanDesc, const miopenTensorDescriptor_t inputDesc, const void *input, const miopenTensorDescriptor_t outputDesc, void *output, miopenOperatorArgs_t args)

Executes the fusion plan.

Parameters

  • handle – MIOpen handle (input)

  • fusePlanDesc – fused plan descriptor (input)

  • inputDesc – Descriptor of the input tensor (input)

  • input – Source data tensor (input)

  • outputDesc – Decriptor of the output tensor (input)

  • output – Destination data tensor (output)

  • args – An argument object of the fused kernel (input)

Returns

miopenStatus_t