λ vector on Tensor CP Decomposition with Alternating Least Square

Question

I am trying to understand the procedure of tensor cp decomposition with alternating least squares based on this paper. At page 464 is referred that "It is often useful to assume that the columns of A, B, and C are normalized to length one with the weights absorbed into the vector λ " In addition, at page 471 line 7 of psedo code is "

normalize columns of A(n) (storing norms as λ)

"

I don't understand what values will be stored on vector λ and on matrix Λ.

What i understand is that we do normalization to ever column of the factor matrices and we store norms on a new vector λ For example for a 3x3x3 tensor with rank=3, will have three 3x3 factors A,B and C and after normalize to unit length every column of all these matrices, i will end up with 9 norms. These norms will be the values of the diagonal matrix Λ ?

Am i missing something? Thank you

Tensor CP Decomposition with Alternating Least Square.

Answer 1

I'm not familiar with the paper, but it looks like this is probably what they mean:

Suppose we're looking at the 3x3 case, since that's easy to draw. We have fixed, non-normalized matrices $A,B,C$ and want matrices $a_r, b_r, c_r$ that are column normalized, and some matrix $\lambda$ s.t. $A B C = \lambda a_r b_r c_r$

\lambda A = 
/ a1 b1 c1 \  / x1 x2 x1 \     / a1 x1 + b1 y1 + c1 z1; a1 x2 + b1 y2 + c1 z2;  ... \
| a2 b2 c2 |  | y1 y2 y2 |  =  | a2 x1 + b2 y1 + c2 z1; ...                         |
\ a3 b3 c3 /  \ z1 z2 z3 /     \ a3 x1 + b3 y1 + c3 z1; ...                         /

Then solve the following system of linear equations for $(a b c)$ to enforce the column-normalization of $\lambda A$:

/1\    / (a1+a2+a3) x1 + (b1+b2+b3) y1 + (c1+c2+c3) z1 \
|1| =  | (a1+a2+a3) x2 + (b1+b2+b3) y2 + (c1+c2+c3) z2 |
\1/    \ (a1+a2+a3) x3 + (b1+b2+b3) z2 + (c1+c2+c3) z3 /

We've technically got too many free variables here, but since we only care about the sum of, e.g., $a1+a2+a3$, and not what any individual a is, we'll just enforce the constraint that $\lambda$ be a diagonal matrix, at which point we have:

/1\    / a1 x1 + b2 y1 + c3 z1 \
|1| =  | a1 x2 + b2 y2 + c3 z2 |
\1/    \ a1 x3 + b2 z2 + c3 z3 /

We plug that into a linear equation solver to get values for $a1,b2,c3$. We then do the same thing for B and C and multiply all the lambdas together (taking advantage of the associativity of matrix multiplication to pull them all to the left), at which point $A B C = \lambda a_r b_r c_r$, and all the $foo_r$ matrices have normalized columns.