-
Notifications
You must be signed in to change notification settings - Fork 21
Description
In main.cu, apart from the input layer l_input, there are 3 layers defined: c1, s1, f.
Lines 13 to 17 in 95c9f33
| // Define layers of CNN | |
| static Layer l_input = Layer(0, 0, 28*28); | |
| static Layer l_c1 = Layer(5*5, 6, 24*24*6); | |
| static Layer l_s1 = Layer(4*4, 1, 6*6*6); | |
| static Layer l_f = Layer(6*6*6, 10, 10); |
In generalized definition of CNN, it consists of convolutional layer, pooling layer and fully connected layer. As c and f corresponds to convolutional and fully-connected, there should be a pooling layer (perhaps named p ).
According to LeNet, pooling layer is also named sub-sampling layer, which may explain the naming of s.
As the Pooling Layer chapter says:
Introduces zero parameters since it computes a fixed function of the input.
however, according to the implementation of fp_preact_s1, it calculated a weighted sum to per 4x4 slice:
Lines 151 to 168 in 95c9f33
| __global__ void fp_preact_s1(float input[6][24][24], float preact[6][6][6], float weight[1][4][4]) | |
| { | |
| const int pos = blockIdx.x * blockDim.x + threadIdx.x; | |
| const int size = blockDim.x * gridDim.x; | |
| const int N = 4*4*6*6*6; | |
| for (int n = N * pos / size; n < N * (pos+1) / size; ++n) { | |
| int idx = n; | |
| const int i1 = ((idx /= 1 ) % 4); | |
| const int i2 = ((idx /= 4 ) % 4); | |
| const int i3 = ((idx /= 4 ) % 6); | |
| const int i4 = ((idx /= 6 ) % 6); | |
| const int i5 = ((idx /= 6 ) % 6); | |
| atomicAdd(&preact[i3][i4][i5], weight[0][i1][i2] * input[i3][i4 * 4 + i1][i5 * 4 + i2]); | |
| } | |
| } |
there is still weight parameters to train, which seems violating the 'zero parameters' trait of pooling layer.
So is the s layer a pooling (sub-sampling) layer? If it is, how can it have parameters to train; or if not, is the whole network still CNN?