Pytorch Tutorial For Beginner: Department of Computer Science & Engineering University of Washington
Pytorch Tutorial For Beginner: Department of Computer Science & Engineering University of Washington
Pytorch Tutorial For Beginner: Department of Computer Science & Engineering University of Washington
CSE446
February 2018
PyTorch
From TensorFlow-Examples
See the Difference: Linear Regression
# Apply gradients
for param in fc.parameters():
param.data.add_(-learning_rate * param.grad.data)
From pytorch/examples
Even Better
PyTorch: Create optimizer while feeding data
import torch.optim as optim
# Define linear regression model (a function)
Yhat = torch.nn.Linear(W.size(0), 1)
opt = optim.SGD(my_model.parameters(), lr=learning_rate)
for epoch in range(training_epochs):
batch_x, batch_y = get_batch() # Get data
Yhat.zero_grad() # Reset gradients
# Forward pass
output = F.mse_loss(Yhat(batch_x), batch_y)
loss = output.data[0]
output.backward() # Backward pass
# Updates parameters!
opt.step()
From pytorch/examples
Essentials
import torch.nn.functional as F
import torch.nn as nn
import torch.nn.init as init
Building Block: Tensor
I Multi-dimensional matrix. (Float/Byte/Long)
I Can initialize from and convert to numpy arrays.
# torch.Tensor = torch.FloatTensor
t1 = torch.Tensor(4, 6)
t1.size() # Returns torch.Size([4, 6])
t2 = torch.Tensor([3.2, 4.3, 5.5])
t3 = torch.Tensor(np.array([[3.2], [4.3], [5.5]]))
t4 = torch.rand(4, 6)
t5 = t1 + t2 # addition
t6 = t2 * t3 # entry-wise product
t7 = t2 @ t3 # matrix multiplication
t8 = t1.view(2,12) # reshapes t1 to be 2 by 12
t8 = t1.view(2,-1) # same as above
t9 = t1[:, -1] # last column from the left
z_sum = torch.sum(z)
Thanks Benjamin Evans for code examples and Kaiyu Zheng for
drafting the slides.