For example, I have 1D vector with dimension (5). I would like to reshape it into 2D matrix (1,5).
Here is how I do it with numpy
>>> import numpy as np
>>> a = np.array([1,2,3,4,5])
>>> a.shape
(5,)
>>> a = np.reshape(a, (1,5))
>>> a.shape
(1, 5)
>>> a
array([[1, 2, 3, 4, 5]])
>>>
But how can I do that with Pytorch Tensor (and Variable). I don't want to switch back to numpy and switch to Torch variable again, because it will loss backpropagation information.
Here is what I have in Pytorch
>>> import torch
>>> from torch.autograd import Variable
>>> a = torch.Tensor([1,2,3,4,5])
>>> a
1
2
3
4
5
[torch.FloatTensor of size 5]
>>> a.size()
(5L,)
>>> a_var = variable(a)
>>> a_var = Variable(a)
>>> a_var.size()
(5L,)
.....do some calculation in forward function
>>> a_var.size()
(5L,)
Now I want it size to be (1, 5). How can I resize or reshape the dimension of pytorch tensor in Variable without loss grad information. (because I will feed into another model before backward)
11 Answers
Use torch.unsqueeze(input, dim, out=None)
>>> import torch
>>> a = torch.Tensor([1,2,3,4,5])
>>> a
1
2
3
4
5
[torch.FloatTensor of size 5]
>>> a = a.unsqueeze(0)
>>> a
1 2 3 4 5
[torch.FloatTensor of size 1x5]
you might use
a.view(1,5)
Out:
1 2 3 4 5
[torch.FloatTensor of size 1x5]
There are multiple ways of reshaping a PyTorch tensor. You can apply these methods on a tensor of any dimensionality.
Let's start with a 2-dimensional 2 x 3 tensor:
x = torch.Tensor(2, 3)
print(x.shape)
# torch.Size([2, 3])
To add some robustness to this problem, let's reshape the 2 x 3 tensor by adding a new dimension at the front and another dimension in the middle, producing a 1 x 2 x 1 x 3 tensor.
Approach 1: add dimension with None
Use NumPy-style insertion of None (aka np.newaxis) to add dimensions anywhere you want. See here.
print(x.shape)
# torch.Size([2, 3])
y = x[None, :, None, :] # Add new dimensions at positions 0 and 2.
print(y.shape)
# torch.Size([1, 2, 1, 3])
