Simd Library Documentation.

Home | Release Notes | Download | Documentation | Forum | SourceForge | GitHub
Neural Network

Artificial Neural Network's functions for accelerating of Simd::Neural. More...

Functions

SIMD_API void SimdNeuralConvert (const uint8_t *src, size_t stride, size_t width, size_t height, float *dst, int inversion)
 Converts a 8-bit gray image to the 32-bit float array. More...
 
SIMD_API void SimdNeuralSigmoid (const float *src, size_t size, const float *slope, float *dst)
 Calculates sigmoid for 32-bit float array. More...
 
SIMD_API void SimdNeuralRoughSigmoid (const float *src, size_t size, const float *slope, float *dst)
 Calculates rough sigmoid for 32-bit float array. More...
 
SIMD_API void SimdNeuralRoughSigmoid2 (const float *src, size_t size, const float *slope, float *dst)
 Calculates rough sigmoid for 32-bit float array. More...
 
SIMD_API void SimdNeuralDerivativeSigmoid (const float *src, size_t size, const float *slope, float *dst)
 Multiplies output 32-bit float array by derivative of sigmoid from input 32-bit float array. More...
 
SIMD_API void SimdNeuralTanh (const float *src, size_t size, const float *slope, float *dst)
 Calculates hyperbolic tangent for 32-bit float array. More...
 
SIMD_API void SimdNeuralRoughTanh (const float *src, size_t size, const float *slope, float *dst)
 Calculates rough hyperbolic tangent for 32-bit float array. More...
 
SIMD_API void SimdNeuralDerivativeTanh (const float *src, size_t size, const float *slope, float *dst)
 Multiplies output 32-bit float array by derivative of hyperbolic tangent from input 32-bit float array. More...
 
SIMD_API void SimdNeuralRelu (const float *src, size_t size, const float *slope, float *dst)
 Calculates Relu (rectified linear unit) function for 32-bit float array. More...
 
SIMD_API void SimdNeuralDerivativeRelu (const float *src, size_t size, const float *slope, float *dst)
 Multiplies output 32-bit float array by derivative of Relu (rectified linear unit) from input 32-bit float array. More...
 
SIMD_API void SimdNeuralProductSum (const float *a, const float *b, size_t size, float *sum)
 Calculates sum of products for two 32-bit float arrays. More...
 
SIMD_API void SimdNeuralAddVectorMultipliedByValue (const float *src, size_t size, const float *value, float *dst)
 Adds the product of a vector and a scalar to given vector. More...
 
SIMD_API void SimdNeuralUpdateWeights (const float *x, size_t size, const float *a, const float *b, float *d, float *w)
 Updates ANN weights. More...
 
SIMD_API void SimdNeuralAdaptiveGradientUpdate (const float *delta, size_t size, size_t batch, const float *alpha, const float *epsilon, float *gradient, float *weight)
 Updates neural network weights with using of adaptive gradients method. More...
 
SIMD_API void SimdNeuralAddConvolution3x3 (const float *src, size_t srcStride, size_t width, size_t height, const float *weights, float *dst, size_t dstStride)
 Adds 3x3 convolution of 32-bit float image. More...
 
SIMD_API void SimdNeuralAddConvolution5x5 (const float *src, size_t srcStride, size_t width, size_t height, const float *weights, float *dst, size_t dstStride)
 Adds 5x5 convolution of 32-bit float image (forward propagation). More...
 
SIMD_API void SimdNeuralAddConvolution3x3Back (const float *src, size_t srcStride, size_t width, size_t height, const float *weights, float *dst, size_t dstStride)
 Adds 3x3 convolution of 32-bit float image (backward propagation). More...
 
SIMD_API void SimdNeuralAddConvolution5x5Back (const float *src, size_t srcStride, size_t width, size_t height, const float *weights, float *dst, size_t dstStride)
 Adds 5x5 convolution of 32-bit float image (backward propagation). More...
 
SIMD_API void SimdNeuralAddConvolution3x3Sum (const float *src, size_t srcStride, const float *dst, size_t dstStride, size_t width, size_t height, float *sums)
 Accumulates changes of weights for 3x3 convolution of 32-bit float image during backward propagation. More...
 
SIMD_API void SimdNeuralAddConvolution5x5Sum (const float *src, size_t srcStride, const float *dst, size_t dstStride, size_t width, size_t height, float *sums)
 Accumulates changes of weights for 5x5 convolution of 32-bit float image during backward propagation. More...
 
SIMD_API void SimdNeuralMax2x2 (const float *src, size_t srcStride, size_t width, size_t height, float *dst, size_t dstStride)
 Reduces input 32-bit float image in two times (takes maximum value in 2x2 window and copies to the output image). More...
 
template<template< class > class A>
SIMD_INLINE void NeuralConvert (const View< A > &src, float *dst, bool inversion)
 Converts a 8-bit gray image to the 32-bit float array. More...
 

Detailed Description

Artificial Neural Network's functions for accelerating of Simd::Neural.

Function Documentation

void SimdNeuralConvert ( const uint8_t *  src,
size_t  stride,
size_t  width,
size_t  height,
float *  dst,
int  inversion 
)

Converts a 8-bit gray image to the 32-bit float array.

The length of output array must be equal to the area of input image.

For every point:

dst[i] = inversion ? (255 - src[col]) / 255 : src[i]/255;
Note
This function has a C++ wrapper Simd::NeuralConvert(const View<A>& src, float * dst, bool inversion).
Parameters
[in]src- a pointer to pixels data of input image.
[in]stride- a row size of the image.
[in]width- an image width.
[in]height- an image height.
[out]dst- a pointer to output array.
[in]inversion- a flag of color inversion.
void SimdNeuralSigmoid ( const float *  src,
size_t  size,
const float *  slope,
float *  dst 
)

Calculates sigmoid for 32-bit float array.

All arrays must have the same size.

For every element:

dst[i] = 1/(1 + exp(-slope*src[i]));
Parameters
[in]src- a pointer to the input array.
[in]size- a size of arrays.
[in]slope- a pointer to the slope parameter.
[out]dst- a pointer to output array.
void SimdNeuralRoughSigmoid ( const float *  src,
size_t  size,
const float *  slope,
float *  dst 
)

Calculates rough sigmoid for 32-bit float array.

All arrays must have the same size.

For every element:

x = ::abs(src[i]*slope);
e = 1 + x + x*x*0.5417 + x*x*x*x*0.1460;
dst[i] = 1 / (1 + (src[i] > 0 ? 1 / e : e));

It is approximate way (maximal absolute error is 0.002294 (~0.23%) ) of sigmoid function (SimdNeuralSigmoid) calculation:

dst[i] = 1/(1 + exp(-slope*src[i]));
Note
This function is used in Simd::Neural::Function.
Parameters
[in]src- a pointer to the input array.
[in]size- a size of arrays.
[in]slope- a pointer to the slope parameter.
[out]dst- a pointer to output array.
void SimdNeuralRoughSigmoid2 ( const float *  src,
size_t  size,
const float *  slope,
float *  dst 
)

Calculates rough sigmoid for 32-bit float array.

All arrays must have the same size.

For every element:

x = -src[i]*slope;
e = max(1 + x/128, 0.5)^128;
dst[i] = 1 / (1 + e);

It is approximate way (maximal absolute error is 0.001721 (~0.17%) ) of sigmoid function (SimdNeuralSigmoid) calculation:

dst[i] = 1/(1 + exp(-slope*src[i]));
Note
This function is used in Simd::Neural::Function.
Parameters
[in]src- a pointer to the input array.
[in]size- a size of arrays.
[in]slope- a pointer to the slope parameter.
[out]dst- a pointer to output array.
void SimdNeuralDerivativeSigmoid ( const float *  src,
size_t  size,
const float *  slope,
float *  dst 
)

Multiplies output 32-bit float array by derivative of sigmoid from input 32-bit float array.

All arrays must have the same size.

For every element:

dst[i] *= slope*(1 - src[i])*src[i];
Note
This function is used in Simd::Neural::Function.
Parameters
[in]src- a pointer to the input array.
[in]size- a size of arrays.
[in]slope- a pointer to the slope parameter.
[in,out]dst- a pointer to output array.
void SimdNeuralTanh ( const float *  src,
size_t  size,
const float *  slope,
float *  dst 
)

Calculates hyperbolic tangent for 32-bit float array.

All arrays must have the same size.

For every element:

x = slope*src[i];
dst[i] = (exp(x) - exp(-x))/(exp(x) + exp(-x));
Parameters
[in]src- a pointer to the input array.
[in]size- a size of arrays.
[in]slope- a pointer to the slope parameter.
[out]dst- a pointer to output array.
void SimdNeuralRoughTanh ( const float *  src,
size_t  size,
const float *  slope,
float *  dst 
)

Calculates rough hyperbolic tangent for 32-bit float array.

All arrays must have the same size.

For every element:

x = ::abs(src[i]*slope);
e = 1 + x + x*x*0.5658 + x*x*x*x*0.1430;
dst[i] = (src[i] > 0 ? 1 : -1)*(e - 1/e)/(e + 1/e);

It is approximate way (maximal absolute error is 0.001514 (~0.15%) ) of hyperbolic tangent (SimdNeuralTanh) function calculation:

x = slope*src[i];
dst[i] = (exp(x) - exp(-x))/(exp(x) + exp(-x));
Note
This function is used in Simd::Neural::Function.
Parameters
[in]src- a pointer to the input array.
[in]size- a size of arrays.
[in]slope- a pointer to the slope parameter.
[out]dst- a pointer to output array.
void SimdNeuralDerivativeTanh ( const float *  src,
size_t  size,
const float *  slope,
float *  dst 
)

Multiplies output 32-bit float array by derivative of hyperbolic tangent from input 32-bit float array.

All arrays must have the same size.

For every element:

    dst[i] *= slope*(1 - src[i]*src[i]);
Note
This function is used in Simd::Neural::Function.
Parameters
[in]src- a pointer to the input array.
[in]size- a size of arrays.
[in]slope- a pointer to the slope parameter.
[in,out]dst- a pointer to output array.
void SimdNeuralRelu ( const float *  src,
size_t  size,
const float *  slope,
float *  dst 
)

Calculates Relu (rectified linear unit) function for 32-bit float array.

All arrays must have the same size.

For every element:

dst[i] =  src[i] > 0 ? src[i] : slope*src[i];
Note
This function is used in Simd::Neural::Function.
Parameters
[in]src- a pointer to the input array.
[in]size- a size of arrays.
[in]slope- a pointer to the slope parameter.
[out]dst- a pointer to output array.
void SimdNeuralDerivativeRelu ( const float *  src,
size_t  size,
const float *  slope,
float *  dst 
)

Multiplies output 32-bit float array by derivative of Relu (rectified linear unit) from input 32-bit float array.

All arrays must have the same size.

For every element:

dst[i] *=  src[i] > 0 ? 1 : slope;
Note
This function is used in Simd::Neural::Function.
Parameters
[in]src- a pointer to the input array.
[in]size- a size of arrays.
[in]slope- a pointer to the slope parameter.
[in,out]dst- a pointer to output array.
void SimdNeuralProductSum ( const float *  a,
const float *  b,
size_t  size,
float *  sum 
)

Calculates sum of products for two 32-bit float arrays.

All arrays must have the same size.

For every element:

sum += a[i]*b[i];
Note
This function is used in Simd::Neural.
Parameters
[in]a- a pointer to the first 32-bit float array.
[in]b- a pointer to the second 32-bit float array.
[in]size- a size of arrays.
[out]sum- a pointer to 32-bit float sum of products.
void SimdNeuralAddVectorMultipliedByValue ( const float *  src,
size_t  size,
const float *  value,
float *  dst 
)

Adds the product of a vector and a scalar to given vector.

All arrays must have the same size.

For every element:

dst[i] += src[i]*value[0];
Note
This function is used in Simd::Neural.
Parameters
[in]src- a pointer to the input 32-bit float array.
[in]size- a size of arrays.
[in]value- a pointer to the scalar 32-bit float value.
[in,out]dst- a pointer to cumulative 32-bit float array.
void SimdNeuralUpdateWeights ( const float *  x,
size_t  size,
const float *  a,
const float *  b,
float *  d,
float *  w 
)

Updates ANN weights.

All arrays must have the same size.

The algorithm performs:

for (size_t k = 0; k < size; ++k)
{
    d[k] = a[0]*d[k] + b[0]*x[k];
    w[k] += d[k];
}
Parameters
[in]x- a pointer to the X array.
[in]size- a size of arrays.
[in]a- a pointer to the first parameter.
[in]b- a pointer to the second parameter.
[in,out]d- a pointer to the D array.
[in,out]w- a pointer to the W array.
void SimdNeuralAdaptiveGradientUpdate ( const float *  delta,
size_t  size,
size_t  batch,
const float *  alpha,
const float *  epsilon,
float *  gradient,
float *  weight 
)

Updates neural network weights with using of adaptive gradients method.

Adaptive gradients method. J Duchi, E Hazan and Y Singer, "Adaptive subgradient methods for online learning and stochastic optimization" The Journal of Machine Learning Research, pages 2121-2159, 2011.

The algorithm performs:

for (i = 0; i < size; ++i)
{
    d = delta[i]/batch;
    gradient[i] += d*d;
    weight[i] -= alpha * d / sqrt(gradient[i] + epsilon);
}
Note
All arrays must have the same size. This function is used in Simd::Neural.
Parameters
[in]delta- a pointer to the array with error (delta).
[in]size- a size of arrays.
[in]batch- a batch size.
[in]alpha- a pointer to alpha parameter (update speed).
[in]epsilon- a pointer to epsilon parameter (a small number used to avoid division by zero).
[in,out]gradient- a pointer to the array with gradients.
[in,out]weight- a pointer to the array with weights.
void SimdNeuralAddConvolution3x3 ( const float *  src,
size_t  srcStride,
size_t  width,
size_t  height,
const float *  weights,
float *  dst,
size_t  dstStride 
)

Adds 3x3 convolution of 32-bit float image.

Note
This function is used in Simd::Neural.
Parameters
[in]src- a pointer to the input 32-bit float image.
[in]srcStride- a row size of the input image (in 32-float values).
[in]width- a width of the output image (input image width must be equal to output image width + 2).
[in]height- a height of the output image (input image height must be equal to output image height + 2).
[in]weights- a pointer to the array with weights (its size must be at least 9).
[in,out]dst- a pointer to the output 32-bit float image.
[in]dstStride- a row size of the output image (in 32-float values).
void SimdNeuralAddConvolution5x5 ( const float *  src,
size_t  srcStride,
size_t  width,
size_t  height,
const float *  weights,
float *  dst,
size_t  dstStride 
)

Adds 5x5 convolution of 32-bit float image (forward propagation).

Note
This function is used in Simd::Neural.
Parameters
[in]src- a pointer to the input 32-bit float image.
[in]srcStride- a row size of the input image (in 32-float values).
[in]width- a width of the output image (input image width must be equal to output image width + 4).
[in]height- a height of the output image (input image height must be equal to output image height + 4).
[in]weights- a pointer to the array with weights (its size must be at least 25).
[in,out]dst- a pointer to the output 32-bit float image.
[in]dstStride- a row size of the output image (in 32-float values).
void SimdNeuralAddConvolution3x3Back ( const float *  src,
size_t  srcStride,
size_t  width,
size_t  height,
const float *  weights,
float *  dst,
size_t  dstStride 
)

Adds 3x3 convolution of 32-bit float image (backward propagation).

Note
This function is used in Simd::Neural.
Parameters
[in]src- a pointer to the input 32-bit float image.
[in]srcStride- a row size of the input image (in 32-float values).
[in]width- a width of the input image (output image width must be equal to input image width + 2).
[in]height- a height of the input image (output image height must be equal to input image height + 2).
[in]weights- a pointer to the array with weights (its size must be at least 9).
[in,out]dst- a pointer to the output 32-bit float image.
[in]dstStride- a row size of the output image (in 32-float values).
void SimdNeuralAddConvolution5x5Back ( const float *  src,
size_t  srcStride,
size_t  width,
size_t  height,
const float *  weights,
float *  dst,
size_t  dstStride 
)

Adds 5x5 convolution of 32-bit float image (backward propagation).

Note
This function is used in Simd::Neural.
Parameters
[in]src- a pointer to the input 32-bit float image.
[in]srcStride- a row size of the input image (in 32-float values).
[in]width- a width of the input image (output image width must be equal to input image width + 4).
[in]height- a height of the input image (output image height must be equal to input image height + 4).
[in]weights- a pointer to the array with weights (its size must be at least 25).
[in,out]dst- a pointer to the output 32-bit float image.
[in]dstStride- a row size of the output image (in 32-float values).
void SimdNeuralAddConvolution3x3Sum ( const float *  src,
size_t  srcStride,
const float *  dst,
size_t  dstStride,
size_t  width,
size_t  height,
float *  sums 
)

Accumulates changes of weights for 3x3 convolution of 32-bit float image during backward propagation.

Note
This function is used in Simd::Neural.
Parameters
[in]src- a pointer to the input 32-bit float image.
[in]srcStride- a row size of the input image (in 32-float values).
[in]dst- a pointer to the output 32-bit float image.
[in]dstStride- a row size of the output image (in 32-float values).
[in]width- a width of the output image (input image width must be equal to output image width + 2).
[in]height- a height of the output image (input image height must be equal to output image height + 2).
[in,out]sums- a pointer to the array with changes of weights (its size must be at least 9).
void SimdNeuralAddConvolution5x5Sum ( const float *  src,
size_t  srcStride,
const float *  dst,
size_t  dstStride,
size_t  width,
size_t  height,
float *  sums 
)

Accumulates changes of weights for 5x5 convolution of 32-bit float image during backward propagation.

Note
This function is used in Simd::Neural.
Parameters
[in]src- a pointer to the input 32-bit float image.
[in]srcStride- a row size of the input image (in 32-float values).
[in]dst- a pointer to the output 32-bit float image.
[in]dstStride- a row size of the output image (in 32-float values).
[in]width- a width of the output image (input image width must be equal to output image width + 4).
[in]height- a height of the output image (input image height must be equal to output image height + 4).
[in,out]sums- a pointer to the array with changes of weights (its size must be at least 25).
void SimdNeuralMax2x2 ( const float *  src,
size_t  srcStride,
size_t  width,
size_t  height,
float *  dst,
size_t  dstStride 
)

Reduces input 32-bit float image in two times (takes maximum value in 2x2 window and copies to the output image).

Note
This function is used in Simd::Neural.
Parameters
[in]src- a pointer to the input 32-bit float image.
[in]srcStride- a row size of the input image (in 32-float values).
[in]width- a width of the input image (output image width must be lesser in 2 times).
[in]height- a height of the input image (output image height must be lesser in 2 times).
[in,out]dst- a pointer to the output 32-bit float image.
[in]dstStride- a row size of the output image (in 32-float values).
void NeuralConvert ( const View< A > &  src,
float *  dst,
bool  inversion 
)

Converts a 8-bit gray image to the 32-bit float array.

The length of output array must be equal to the area of input image.

For every point:

dst[i] = inversion ? (255 - src[col]) / 255 : src[i]/255;
Note
This function is a C++ wrapper for function SimdNeuralConvert.
Parameters
[in]src- an input image.
[out]dst- a pointer to output array.
[in]inversion- a flag of color inversion.