Skip to content

Is s a pooling layer? #3

@dhy2000

Description

@dhy2000

In main.cu, apart from the input layer l_input, there are 3 layers defined: c1, s1, f.

CUDA-CNN/main.cu

Lines 13 to 17 in 95c9f33

// Define layers of CNN
static Layer l_input = Layer(0, 0, 28*28);
static Layer l_c1 = Layer(5*5, 6, 24*24*6);
static Layer l_s1 = Layer(4*4, 1, 6*6*6);
static Layer l_f = Layer(6*6*6, 10, 10);

In generalized definition of CNN, it consists of convolutional layer, pooling layer and fully connected layer. As c and f corresponds to convolutional and fully-connected, there should be a pooling layer (perhaps named p ).

According to LeNet, pooling layer is also named sub-sampling layer, which may explain the naming of s.

As the Pooling Layer chapter says:

Introduces zero parameters since it computes a fixed function of the input.

however, according to the implementation of fp_preact_s1, it calculated a weighted sum to per 4x4 slice:

CUDA-CNN/layer.cu

Lines 151 to 168 in 95c9f33

__global__ void fp_preact_s1(float input[6][24][24], float preact[6][6][6], float weight[1][4][4])
{
const int pos = blockIdx.x * blockDim.x + threadIdx.x;
const int size = blockDim.x * gridDim.x;
const int N = 4*4*6*6*6;
for (int n = N * pos / size; n < N * (pos+1) / size; ++n) {
int idx = n;
const int i1 = ((idx /= 1 ) % 4);
const int i2 = ((idx /= 4 ) % 4);
const int i3 = ((idx /= 4 ) % 6);
const int i4 = ((idx /= 6 ) % 6);
const int i5 = ((idx /= 6 ) % 6);
atomicAdd(&preact[i3][i4][i5], weight[0][i1][i2] * input[i3][i4 * 4 + i1][i5 * 4 + i2]);
}
}

there is still weight parameters to train, which seems violating the 'zero parameters' trait of pooling layer.

So is the s layer a pooling (sub-sampling) layer? If it is, how can it have parameters to train; or if not, is the whole network still CNN?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions