Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
103 views

A Brief Introduction To Pytorch: (A Deep Learning Library)

This document introduces PyTorch, a popular deep learning library. It discusses why deep learning libraries are necessary, highlighting that they allow for easily building computational graphs and efficiently running models on GPUs. PyTorch is introduced as a dynamic graph library that supports Python and has a friendly API for research. The major components of PyTorch like Tensor, autograd, nn and optim are described at a high level. Finally, the document provides examples of using PyTorch to build and train neural networks on MNIST, including MLPs, RNNs and CNNs.

Uploaded by

nuts
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
103 views

A Brief Introduction To Pytorch: (A Deep Learning Library)

This document introduces PyTorch, a popular deep learning library. It discusses why deep learning libraries are necessary, highlighting that they allow for easily building computational graphs and efficiently running models on GPUs. PyTorch is introduced as a dynamic graph library that supports Python and has a friendly API for research. The major components of PyTorch like Tensor, autograd, nn and optim are described at a high level. Finally, the document provides examples of using PyTorch to build and train neural networks on MNIST, including MLPs, RNNs and CNNs.

Uploaded by

nuts
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 32

A Brief Introduction to PyTorch

(a deep learning library)

Huaipeng Zhao

pytorch.org
Outline
 Why Deep Learning Libraries and Why PyTorch ?
 A Brief Introduction to PyTorch
 PyTorch in action
Outline
 Why Deep Learning Libraries and Why PyTorch ?
 A Brief Introduction to PyTorch
 PyTorch in action
Why a DL Library is Necessary?
• Complicated DL architectures
• Easily build big computational graphs
• Easily compute gradients
Why a DL Library is Necessary?
• Run it all efficiently on GPU
Popular Deep Learning Libraries
Why

Life is short, you need


Why

• Support Python
• Dynamic Graph
• Friendly API for research
• Easily debug
• Love all things [Pythonic]
Outline
 Why Deep Learning Libraries and Why PyTorch ?
 A Brief Introduction to PyTorch
 PyTorch in action
Installing
Three Levels of Abstraction
• Tensor: Imperative ndarray
• Variable: Node in a computational graph (data,grad)
• Module: A neural network layer
A Toy Neural Network

Forward
A Toy Neural Network

Forward

Tensor
A Toy Neural Network

Backward
A Toy Neural Network
Major Components of PyTorch

Package Description
torch a Tensor library like NumPy, with strong GPU support

torch.autograd a tape based automatic differentiation library that supports all


differentiable Tensor operations in torch

torch.nn a neural networks library deeply integrated with autograd designed for
maximum flexibility

torch.optim an optimization package to be used with torch.nn with standard


optimization methods such as SGD, RMSProp, LBFGS, Adam etc.

torch.utils DataLoader, Dataset and other utility functions for convenience


Torch Module:PyTorch as A Tensor Library
• Tensor operations: slicing, indexing, math operations, linear algebra, r
eductions
• CPU & GPU
• Fast! (comparison on speed of matrix multiplication)
𝑴∗𝑴∗𝑴
   𝑀 ∈ ℝ1000 ×1000

Numpy

PyTorch
Torch.nn : a neural networks library

Containers Module, Sequential,ModuleList,ParameterList


Convolution Layers Conv1d,Conv2d,Conv3d…
Recurrent Layers RNN,LSTM,GRU,RNNCell…
Linear Layers Linear, Bilinear
Non-linear Activations ReLU,Sigmoid,Tanh,LeakyReLU…
Loss Functions NLLLoss,BCELoss,CrossEntropyLoss…
Torch.nn.functional

Torch.nn

Torch.nn.functional
How to load dataset?
How to build a model?
How to train a model?
Outline
 Why Deep Learning Libraries and Why PyTorch ?
 A Brief Introduction to PyTorch
 PyTorch in action
MNIST Dataset
MLP for MNIST (0-d features)

10

64

28*28
MLP for MNIST

Last Test Acc: 95.8%


RNN for MNIST (1-d features)

GRU
GRU GRU
GRU … GRU
GRU

28

…
28
RNN for MNIST

Last Test Acc: 97.7%


CNN for MNIST (2-d features)
CNN for MNIST

Last Test Acc: 99.2%


Assignments
1. Try more models (Bi-LSTM,ResNet…)
2. L2 Regularization
3. Dropout
4. Data Argumentation
5. Batch Normalization
6. Try other optimization algorithms(SGD…)
7. Try more activation function(Tanh,Sigmoid…)
Resources
• Official resources
• Documentation http://pytorch.org/docs/master/
• Tutorials http://pytorch.org/tutorials/index.html
• Example projects https://github.com/pytorch/examples
• Github code
• fairseq-py
• OpenNMT-py
• Open courses and blogs
• Stanford CS231N 2017, Lecture 8 “Deep Learning Software”

You might also like