How to understand tensors with multiple dimensions?

Question

I've been recently a bit confused about tensors. Say if we have a tensor with shape (3,2,3,4), we mean that in the first dimension there are 3 groups of numbers? OR it just means that there are exactly just 3 numbers in the first dimension?

Then here comes the second question, with the tensor A that has the shape (3,2), why is that the output of torch.max(A,0) returns a group of max values that contains 2 max values instead of 3,considering the fact that there are 3 numbers in the first dimension.

>>>a = torch.randn(3,2)
a tensor([[-1.1254, -0.1549],
        [-0.5308,  1.0427],
        [-0.1268,  1.0866]])
>>>torch.max(a,0)
torch.return_types.max(
values=tensor([-0.1268,  1.0866]),
indices=tensor([2, 2]))

I mean why doesn't it return a list of 3 max values?

Then the third question, if we have two tensors with shape(3,3,10,2) and (2,4,10,1), can we just concatenate these two tensors on the third dimension considering they have the same size on that dimension? If it is feasible, what is the reason behind it?

I'll be much appreciated if you help me understand this!

Answer 1

Tensors are just high level dimensions of vectors. You should start with a vector first. Ex: 4 elements vector A = [1,2,3,4]. A 6-vectors A together is B = [[1,2,3,4],[1,2,3,4],...,[1,2,3,4]]. This is called matrix (2 dimensions), the shape is 6x4. Now if you take 2 matrices B, it will form a tensor C, shape is (2,6,4). Extend C for a list of 3 C, you can get a 4-dimensions tensor (3,2,6,4) and so on. You can take this picture for better illustration.

For the torch max, torch.max(input, dim, keepdim=False, out=None) when you choose the dim=0 (the dimension to reduce to this mean you will find the max along the 0 axis (aka the first shape), it will follow through this axis to find the maximum number corresponding to the rest tensor (according to your tensor, you only have to check in the 2 elements vectors), which is left. If you extend this for higher dimension, like (3,4,5) shape, the max will be (4,5).

For the last question, there is a no for that tensor. Because when you look at the illustration figure, there is no way for you to just base on 1 shape to concatenate 2 different size matching tensor. You have to maintain the same shape for all dimensions except the on you are going to concatenate.

>>> b = np.random.randint(0,2,size=(2,3,2,3))
>>> a = np.random.randint(0,2,size=(2,3,1,3))
>>> np.concatenate([a,b],axis=2)

Else, the only way for you to do it is flattening both of the 2 tensors so that you only need a vector represent for both a and b.

Answer 2

About you first answer : think about images. that way it will be simple to visualize. Or in your example, x = np.zeros((3,2,3,4)) 3 number of 3d tensors having (2,3,4) dimensions each are stacked.

for second question you should have at least n-1 same dimensions to concatenate in n-th dimension. below picture will help to understand.

image : https://ibb.co/q5WnMD5

third question you mentioned max(a,0), where 0 represent axis, basically column and it is finding maximum values for each column.