## SIAGA: Tensors

### Seven pictures from Applied Algebra and Geometry: Picture #6

The Society for Industrial and Applied Mathematics, SIAM, has recently released a journal of Applied Algebra and Geometry called SIAGA. See here for more information on the new journal. They will start taking online submissions on March 23rd.

The poster for the journal features seven pictures. In this penultimate blog post I will talk about the sixth picture, on the subject of Tensors. In the first section of this post, “The Context”, I’ll set the mathematical scene. In the second section, “The Picture”, I’ll talk about this particular image.

# The Context

Tensors are the higher-dimensional analogues of matrices. They are data arrays with three or more dimensions, and are represented by an array of size $n_1 \times \cdots \times n_d$, where $n_k$ is the number of ‘rows’ in the $k$th direction of the array. The entries of the tensor $A$ are denoted by $A_{i_1 \ldots i_d}$ where $i_k \in \{ 1, \ldots, n_k \}$ tells you which row in the $k$th direction you are looking at. Just as for a matrix, the entries of a tensor are elements in some field, for example real or complex numbers.

Tensors occur naturally when it makes sense to organize data by more than two indices. For example, if we have a function that depends on three or more discretized inputs $f(x,y,z)$ where $x \in \{ x_1, \ldots, x_{n_1} \}$, $y \in \{ y_1, \ldots, y_{n_2} \}$ and $z \in \{ z_1, \ldots, z_{n_3} \}$, then we can organize the values $A_{ijk} = f(x_i,y_j,z_k)$ into a tensor of size $n_1 \times n_2 \times n_3$. Tensors are increasingly widely used in many applications, especially signal processing, where the uniqueness of a tensor’s decomposition allows the different signals comprising a mixture to be found. They have also been used in machine learning, genomics, geometric complexity theory and statistics.

Our data analysis techniques are currently limited to a matrix-centric perspective. To overcome this, there has been tremendous effort to extend the well-understood properties of matrices to the higher-dimensional world of tensors. A greater understanding of tensors paves the way for very exciting new developments that can cater to the natural structure of tensor-based data, for example in experimental design or confounding factor analysis. This analysis and understanding uses interesting and complicated geometry.

One requirement for computability of a tensor is to have a good low rank approximation. Tensors of size $n_1 \times \cdots \times n_d$ have $n_1 \ldots n_d$ entries and, for applications, this quickly becomes unreasonably large. Matrices are analyzable via their singular value decomposition, and the best low rank approximation is obtainable directly from this by truncating at the $r$th largest singular value. We can extend many of the useful notions from linear algebra to tensors: we have eigenvectors and singular vectors of tensors, and a higher order version of the singular value decomposition.

# The Picture

As well as being a picture of the well-known Rubik’s cube, this picture describes a cartoon of a tensor of size $3 \times 3 \times 3$. Such a tensor consists of 27 values.

To understand the structure contained in a tensor, we use its natural symmetry group to find a presentation of it that is simple and structurally transparent. This motivation also underlies the Rubik’s puzzle although the symmetries can be quite different: a change of basis transformation for the tensor case, and a permutation of pieces in the case of the puzzle.

Despite being small, a $3 \times 3 \times 3$ tensor has interesting geometry. It is known that a generic tensor of size $3 \times 3 \times 3$ has seven eigenvectors in $\mathbb{P}^2$. In the paper “Eigenconfigurations of Tensors” by Abo, Seigal and Sturmfels, we show that any configuration of seven eigenvectors can arise, provided no six of the seven points lie on a conic.