5 Pytorch Operations frequently used

I have explored a few PyTorch functions:

These 5 functions can be found in:

https://jovian.ml/sudhakarmlal/01-tensor-operations

An short introduction about PyTorch and about these functions are as follows

Function 1 — torch.zeros

Creates a tensor with all values 0

In [20]:

# Example 1 - create an zero 5*3 tensor with values are 0
x = torch.zeros(5, 3, dtype=torch.long)
print(x)

tensor([[0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0], [0, 0, 0]])

The above created a torch tensor of 5 * 3 with all values are 0

In [21]:

# Example 2 - create an zero 2*2*3  3 Dimensional tensor with values 0
x = torch.zeros(2, 2,3, dtype=torch.float)
print(x)

tensor([[[0., 0., 0.], [0., 0., 0.]], [[0., 0., 0.], [0., 0., 0.]]])

The above created a tensor with 2 2 3 with dtype as float

In [22]:

# Example 3 - breaking 
x = torch.zeros(0, dtype=torch.double)
print(x)

tensor([], dtype=torch.float64)

In the above example a 0*0 tensor is created with no values

Summary: The tensor.zeros function is created to create a tensor with all values 0’s of any dimension(e.g 2D or 3D).But if 0 is specified as an argument for the matrix size it doesn’t create tensor with 0 values

Function 2 — torch.view

Resizes the tensor to a specific dimension

In [23]:

# Example 1 - workingx = torch.randn(2, 2)
y = x.view(4)
print(x.size(),y.size())

torch.Size([2, 2]) torch.Size([4])

Resizes tensor of dimension 2*2 to 4 (ie form a two-dimension to 1 demension)

In [24]:

# Example 2 - workingt1 = torch.randn(4, 4)
t2= t1.view(-1,8)
print(t1.size(),t2.size())

torch.Size([4, 4]) torch.Size([2, 8])

Resizes tensor of dimension 44(2D) to (28) again (2d)

In [25]:

# Example 3 - breakingt3 = torch.randn(4, 4)
t4= t3.view(-1,7)
print(t3.size(),t4.size())

--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-25-de608c467a83> in <module> 2 3 t3 = torch.randn(4, 4) ----> 4 t4= t3.view(-1,7) 5 print(t3.size(),t4.size()) RuntimeError: shape '[-1, 7]' is invalid for input of size 16

The above resize will not work as 16 is not divisible by 7 to convert a 4*4(16) elements to multiples of 7

torch.view can resize the tensors e.g from 2D to 1D,or 2D to 2D,2D to 3D.But if the dimensions (provided as parameters to view) doesn’t match it will fail

Function torch.add

Add two tensors

In [26]:

# Example 1 x=torch.tensor([[1, 2], [3, 4]])
y =torch.tensor([[9, 7], [8, 4]])
z=torch.add(x,y) t
print(z)

File "<ipython-input-26-e85cd502325f>", line 6 z=torch.add(x,y) t ^ SyntaxError: invalid syntax

Added two matrixes x and y and stores the resultant matrix in z

In [27]:

# Example 2t1 = torch.tensor([
[[1, 2, 3],
[3, 4, 5]],
[[5, 6, 7],
[7, 8, 9]]])
t2 =torch.tensor([
[[4, 5, 6],
[6, 7, 8]],
[[9, 8, 6],
[3, 4, 5]]])
t3 = torch.add(t1,t2)
print(t3)

tensor([[[ 5, 7, 9], [ 9, 11, 13]], [[14, 14, 13], [10, 12, 14]]])

Add two three dimensional tensors t1 and t2 and stores it in t3

In [28]:

# Example 3 - breakinga=torch.tensor([[1, 2], [3, 4]])
b =torch.tensor([[9, 7], [8, 4]])
c =torch.tensor([[10, 15], [6, 12]])
d =torch.add(a,b,c)

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-28-6dac08e987a8> in <module> 5 c =torch.tensor([[10, 15], [6, 12]]) 6 ----> 7 d =torch.add(a,b,c) TypeError: add() takes 2 positional arguments but 3 were given

The add function can add only two tensors and not three

torch.add can be used to add two tensors of any dimension.It cannot be used to add more than 2 tensors

Function 4 — torch.copy_

Copies one tensor to the other

In [29]:

# Example 1 - working
a=torch.tensor([[1, 2], [3, 4]])
b = torch.empty_like(x).copy_(a)
print(b)

tensor([[1., 2.], [3., 4.]])

Copies a two dimensional tensor to another tensor

In [30]:

# Example 2 - working
t1 = torch.tensor([
[[1, 2, 3],
[3, 4, 5]],
[[5, 6, 7],
[7, 8, 9]]])
t2 = torch.empty_like(t1).copy_(t1)
print(t2)

tensor([[[1, 2, 3], [3, 4, 5]], [[5, 6, 7], [7, 8, 9]]])

Copies a three dimensional tensor to another tensor

In [31]:

# Example 3 - breaking 
x=torch.tensor([[1, 2], [3, 4]])
x.size()
#y = torch.randn(2, 2)
#y.copy_(x)
z = torch.randn(4, 4)
z.copy_(x)

--------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) <ipython-input-31-632355b42210> in <module> 8 9 z = torch.randn(4, 4) ---> 10 z.copy_(x) RuntimeError: The size of tensor a (4) must match the size of tensor b (2) at non-singleton dimension 1

In the above example the size of x is (2,2) which we are trying to copy to z of (4,4) hence it failed

The torch.copy_ works for copying one tensor to other given both are of same dimension.If it doesn’t it will fail.

Function 5 — torch.to

Add some explanations

In [32]:

# Example 1 - workingimport numpy as np
a = np.ones(5)
b = torch.from_numpy(a)
device = torch.device("cpu")
b.to(device)

Out[32]:

tensor([1., 1., 1., 1., 1.], dtype=torch.float64)

Explanation about example

In [33]:

# Example 2 - working
import numpy as np
x = np.ones(6)
y = torch.from_numpy(x)
z=y.to("cpu", torch.int64)
print(z)

tensor([1, 1, 1, 1, 1, 1])

In Example1 I have passed device as an argument to torch.to function.Here Iam specifying dtype as int and directly passing “cpu” rather than the device

In [34]:

# Example 3 - breaking import numpy as np
t1 = np.ones(7)
t2 = torch.from_numpy(t1)
t3=t2.to("cuda", torch.float64)
print(t3)

--------------------------------------------------------------------------- AssertionError Traceback (most recent call last) <ipython-input-34-96522b9c7b5d> in <module> 5 t2 = torch.from_numpy(t1) 6 ----> 7 t3=t2.to("cuda", torch.float64) 8 print(t3) /srv/conda/envs/notebook/lib/python3.7/site-packages/torch/cuda/__init__.py in _lazy_init() 147 raise RuntimeError( 148 "Cannot re-initialize CUDA in forked subprocess. " + msg) --> 149 _check_driver() 150 if _cudart is None: 151 raise AssertionError( /srv/conda/envs/notebook/lib/python3.7/site-packages/torch/cuda/__init__.py in _check_driver() 52 Found no NVIDIA driver on your system. Please check that you 53 have an NVIDIA GPU and installed a driver from ---> 54 http://www.nvidia.com/Download/index.aspx""") 55 else: 56 # TODO: directly link to the alternative bin that needs install AssertionError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

The above example throws “Found no NVIDIA driver on your system” as I have set the device to “cuda” which doesn’t allow me to run on GPU as we don’t have GPU setup for the same

Conclusion

The blog covers the following functions:

These are very important functions used in pytorch. We can further use this functions in the pytorch models which we would develop in the near future.