A collection of deep learning architectures (a PyTorch implementation).
- python >= 3.6
- PyTorch > 1.3
- torchvision
- visdom
If you have more nvidia cards & cores available, adjust the batch size (BSZ), number of GPUs (gpus), & number of threads (cpus) accordingly in the ./ImageNet.sh. Next, select an available architecture and update your train & validation folder location (trainDataPath and testDataPath). Finally, run ./ImageNet.sh.
To replicate Hinton's paper on MNIST, run the following:
python Capsule.py -A capsule -B 256 -E 500 --optimizer adam --gpus 2 --cpus 6 --trainDataPath ./data --testDataPath ./data --replicate_paper
Ignore the replicate_paper argument to create a deep architecture (with few residual blocks before primary capsule). You can essentially add any block available in NeuralLayers to create a deeper architecture, which is followed by a primary capsule and secondary capsule. However, do consider two things 1. if you do reconstruction, update the reconstruction network relative to tensor_size, 2. capsule nets do require a good amount of gpu ram.
Trained on CIFAR10 (pggan-cifar10.py) -- requires more training (more gpus)!
- Maxout Networks
- SWISH: A SELF-GATED ACTIVATION FUNCTION
- Mish: Self Regularized Non-Monotonic Activation Function
- Auto-Encoding Variational Bayes
- Progressive Growing of GANs
- Self-Attention Generative Adversarial Networks
- Aggregated Residual Transformations for Deep Neural Networks
- Deep Residual Learning for Image Recognition
- Densely Connected Convolutional Networks
- Dynamic Routing Between Capsules
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- MobileNetV2: Inverted Residuals and Linear Bottlenecks
- Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
- Squeeze-and-Excitation Networks
- AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy
- ContextNet: Exploring Context and Detail for Semantic Segmentation in Real-time
- U-Net: Convolutional Networks for Biomedical Image Segmentation
- AN EXPLORATION OF SOFTMAX ALTERNATIVES BELONGING TO THE SPHERICAL LOSS FAMILY
- Analyzing and Improving Representations with the Soft Nearest Neighbor Loss
- ArcFace: Additive Angular Margin Loss for Deep Face Recognition
- CosFace: Large Margin Cosine Loss for Deep Face Recognition
- Rethinking Feature Distribution for Loss Functions in Image Classification
- Tversky loss function for image segmentation using 3D fully convolutional deep networks
- SSD: Single Shot MultiBox Detector (MobileNetV2SSD320 and TinySSD320)