Convolutional Neural Network

Imports

import torch
import torchvision.datasets as dsets
import torchvision.transforms as transforms
import torch.nn.init

device = 'cuda' if torch.cuda.is_available() else 'cpu'

# for reproducibility
torch.manual_seed(777)
if device == 'cuda':
    torch.cuda.manual_seed_all(777)

<torch._C.Generator at 0x7fc3da535b90>

1 CNN model

• Input (input structure)
  • Input type: torch.Tensor
  • Input shape: (N x C x H x W)
    • N: Batch size, C: # of channels, H: height, W: width
  • (?, 1, 28, 28)
    • Assume multiple grayscale images of size 28x28
• Layer design
  • Layer 1
    • Conv2d >> C: 32, Kernel size (filter size): 3, Stride: 1, Padding: 1
    • ReLU
    • MaxPool >> Kernel size: 2, Stride: 2
    • Input–output (?, 1, 28, 28) >> (?, 32, 14, 14)
  • Layer 2
    • Conv2d >> C: 64, Kernel size (filter size): 3, Stride: 1, Padding: 1
    • ReLU
    • MaxPool >> Kernel size: 2, Stride: 2
    • Input–output (?, 32, 14, 14) >> (?, 64, 7, 7)
  • Layer 3
    • Linear >> input: 7x7x64 output: 10
    • Softmax

class CNN(torch.nn.Module):

    def __init__(self):
        super(CNN, self).__init__()
        # L1 Input shape=(?, 1, 28, 28)
        #    Conv     -> (?, 32, 28, 28)
        #    Pool     -> (?, 32, 14, 14)
        self.layer1 = torch.nn.Sequential(
            torch.nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1),
            torch.nn.ReLU(),
            torch.nn.MaxPool2d(kernel_size=2, stride=2))
        # L2 Input shape=(?, 32, 14, 14)
        #    Conv      ->(?, 64, 14, 14)
        #    Pool      ->(?, 64, 7, 7)
        self.layer2 = torch.nn.Sequential(
            torch.nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1),
            torch.nn.ReLU(),
            torch.nn.MaxPool2d(kernel_size=2, stride=2))
        # L3 Linear 64x7x7 inputs -> 10 outputs
        self.fc = torch.nn.Linear(64*7*7, 10, bias=True)
        torch.nn.init.xavier_uniform_(self.fc.weight)

    def forward(self, x):
        out = self.layer1(x)
        out = self.layer2(out)
        out = out.view(out.size(0), -1)   # Flatten them for FC
        out = self.fc(out)
        return out

2 Assignment

Build the following CNN model

• Input
  • Input type: torch.Tensor
  • Input shape: (?, 1, 28, 28)
    • Assume multiple grayscale images of size 28x28
• Layers
  • Layer 1
    • Conv2d >> C: 32, Kernel size (filter size): 3, Stride: 1, Padding: 1
    • ReLU
    • MaxPool >> Kernel size: 2, Stride: 2
    • Input–output (?, 1, 28, 28) >> (?, 32, 14, 14)
  • Layer 2
    • Conv2d >> C: 64, Kernel size (filter size): 3, Stride: 1, Padding: 1
    • ReLU
    • MaxPool >> Kernel size: 2, Stride: 2
    • Input–output (?, 32, 14, 14) >> (?, 64, 7, 7)
  • Layer 3
    • Conv2d >> C: 128, Kernel size (filter size): 3, Stride: 1, Padding: 1
    • ReLU
    • MaxPool >> Kernel size: 2, Stride: 2, Padding: 1
    • Input–output (?, 64, 7, 7) >> (?, 128, 4, 4)
  • Layer 4
    • Linear >> input: 4x4x128 output: 625
    • ReLU
    • Dropout
    • Input–output (4x4x128) >> (625)
  • Layer 5
    • Linear >> input: 625 output: 10
    • Softmax (account for using PyTorch's Cross Entropy Loss function)