Three correlations and a samosa

Let’s imagine a hypothetical situation. There’s an infection going round, and we want to predict the future severity of someone’s illness.

There is a test that offers a good prediction. Let’s say the outcome of the test has a correlation of 0.78 with the patient’s severity of infection. The problem with the test is that it is expensive and time-consuming. But there’s an alternative test, which is much cheaper and faster. We don’t know how well the cheap test correlates with the severity of infection, but we know the correlation between the cheap test and the expensive test is quite high, at 0.89.

We have three related correlations – two known, and one unknown.

What can we say about how well the cheap test correlates with the severity of infection?

We might expect to be able to say something about the unknown correlation. For example, if the expensive test had a correlation of 1 with the severity of infection, and that the cheap test also had a correlation of 1 with the expensive test, then everything is perfectly correlated and the cheap test must also have a correlation of 1 with the severity of infection.

But, let’s assume the expensive test only has a correlation of 0.5 with the severity of infection, and that the two tests are also only correlated with correlation 0.5. Now it isn’t clear whether we can say anything about the correlation of the cheap test with the severity of infection.

Let’s go back to the numbers in the original example.

We can organise our correlations into a matrix. We have three variables:

1. severity of infection
2. expensive test outcome
3. cheap test outcome

And we build the matrix:

The entry of the matrix gives the correlation between variable and . Each variable has a perfect correlation with itself, so the diagonal entries of the matrix are equal to 1. In addition, the correlation between and is the same as the correlation between and , so the matrix is symmetric.

A matrix of correlations has to be positive semi-definite. This condition will allow us to find the range of possible values for .

But first, let’s consider the case where all three correlations are unknown. We then have correlation matrix

The region of possible values for the correlations are the values for which the matrix is positive semi-definite. This is a region of 3-dimensional space that looks like this:

It is called the elliptope, a name that dates back at least as far as 1996 (see here) and is also called the samosa, a name that dates back at least as far as 2011 (see here).

We can now fix values for our two known correlations, to see the possible values for the third correlation. The possibilities are all values that keep the triple of correlations inside the samosa.

If we know two correlations are and , the range of possibilities for the third correlation is given by the black line

We can find the upper and lower limit of the third correlation by seeing where the black line intersects the boundary of the samosa. The boundary is defined by setting the determinant of the matrix

to be zero. We get a quadratic polynomial in with roots at approximately 0.41 and 0.98. So the third correlation has to lie in the range 0.41 to 0.98.

So for the infection example, although the cheap test has a high correlation with the expensive test, in the worst case it only offers a correlation of 0.41 with the severity of infection.

If the two known correlations had both been 0.5, a similar computation shows that the third correlation has to lie in the range -0.5 to 1. The third correlation could even be negative!

If two correlations take values 0.5, the third correlation could range from -0.5 to 1, the values along the black line

This is the first part of a small series of “correlated” posts that I hope to write about correlations – stay tuned for more!

Diagram algebras

As Anna said, time has flown since we regularly wrote here, so let’s get going again!

An application I wrote recently included a summary for ‘the general public’. Here’s an excerpt, about some diagram algebras I have been studying.

Consider 2n holes in the ground, and n moles which have to pop out of one hole and make their way to a free hole. If we consider the paths these moles create we get a diagram belonging to some of the diagram algebras I will study: allowing the moles to cross each others’ paths gives us the Brauer algebra, and if we don’t allow them to cross we get the Temperley-Lieb algebra. Diagram algebras have strong connections with physics: replacing our moles with particles, gives a physicists connection diagram.

Let’s unpack this a little, with some pictures. We have 2n holes, and n moles, who each have a start and an end hole, and create a path. Let’s take n=3.

Here are some moles expertly drawn. They have to move forwards (they can’t move around the back of holes) and make their way to a free one.

in the Brauer case, these paths are allowed to cross:

But in the Temperley-Lieb case, they aren’t:

(There are some more interesting looking paths for this, but our n=3 case doesn’t have so many!)

Of course, (disclaimer) these are only the basic building blocks of these algebras and explained in a very non-mathematical way! (Moles rhymes with holes so why not!)

In maths, we package this information a little differently, with the basic building block (or basis) elements given by matchings (for the Brauer algebra) or planar matchings (for the Temperley-Lieb algebra) of the union of the sets

{-n,…,-1} and {1,…n}.

Which essentially means we line n of our 2n ‘holes’ up on one side, and n on the other. This allows for operations in the algebra, but that’s a different story. Here are some elements of the Brauer and Temperley-Leib algebras drawn in this way:

An element of the Brauer algebra Br₅

An element of the Temperley-Lieb algebra TL₅

There are a many areas of mathematics these algebras, and other digram algebras, are well-used in. For instance they are very important in representation theory, knot theory, topological quantum field theory, and as I mentioned before they describe connection diagrams in physics.

Algorithms to find rectangles, for humans and computers

Cell blocks (or Shikaku) is a puzzle that looks like this:

It has a grid with some numbered squares. The goal is to divide the grid into rectangles, so that each rectangle contains only one numbered square, and the number on the square is the area of the rectangle.

From the locations and values of the numbers, we see where some of the rectangles should go: the “14” has to go in a rectangle of area 14, which (to avoid other numbered squares) has to be the 2×7 rectangle on rows 2 and 3. This cuts off row 1 from the rest, so we can find the rectangles on that row too. For other cell blocks puzzles it can be more difficult to work out where to start:

Finding rectangles in a grid may not seem particularly useful, but it can sometimes help us to analyse data. However, while cell blocks puzzles are designed to be solved (or even enjoyed) by a human, finding rectangles in other grids requires some computer assistance.

In a recent(-ish) project, we designed an algorithm to divide a grid into rectangles in accordance with certain rules. We used the rectangles that the algorithm found to analyse some biological data.

Here is an example of how it works. We have a grid that summarises some biological data.

The rows of the grid give information about different breast cancer cell lines, which we can think of as different sub-types of breast cancer. The columns of the grid are labelled by experimental conditions. Each cell line has been exposed to each experimental condition, to try to disentangle similarities and differences between them.

The colours on the grid give the type of response. For example, if a location on the grid is yellow, we think of this as saying that the cell line has a “high response” to a particular experimental condition. Green is “medium response” and blue is “low response”.

It is difficult to summarise the locations of the grid with each of the three different responeses, which complicates the biological interpretation. To get around this, we run our algorithm to find rectangles on the grid that are “as close as possible” to the above picture, in the sense that as few locations as possible have changed colour. We obtain this picture:

The algorithm works by re-casting the search for rectangles as an integer optimisation problem, and then using the branch and cut method. The (matlab) code to find the constraints on the optimisation is given here (on pages 155-8). This enabled the computer to solve our challenging cell blocks style puzzle.

Simpson’s paradox as a semi-algebraic set

This post is about the second part of the talk I gave at It all adds up in January. It’s about some of the maths behind the idea of making decisions based on partial observations, something we do a lot. One important example of this is deciding medical treatment options based on results from a clinical trial.

Let’s say you are deciding between two medicines: whether to take the red pill or the blue pill.

You decide to take the option that gives you the greatest chance of being “happy”. You’re given some data to help you decide: you’re told that 62% of people who take the blue pill are happy, compared with 57% of people who choose the red pill.

But that’s not all, you’re also told the percentages broken down into older people and younger people

• for young people, 63% of those who take the blue pill are happy, but 80% of those who take the red pill are happy
• for older people, 50% of those who take the blue pill are happy, and 56% of those who take the red pill are happy

The data seem incompatible. If you don’t take your age into account, you choose the blue pill to have the higher chance of happiness. If you take age into account, regardless of whether you are old or young, then you should take the red pill for the higher chance of happiness.

But the data are not inconsistent. Here is an example of a survey the percentages could have come from.

And here is the aggregated data across both age groups:

It seems difficult to make a decision based on this data.

This is an example of a phenomenon is called Simpson’s paradox.

It can be described in terms of inequalities. We can turn each of the numbers in the tables above into probabilities, by dividing by the total number of respondents. The phenomenon occurs when the following inequalities hold:

Such examples are rare, but they do happen. We can sample the space of probabilities to compute that it happens around approximately 1.7% of the time, when we have three binary variables.

This 1.7% is the volume of a semi-algebraic set: a region of space where the three inequalities hold. Here is a 3D picture of the algebraic conditions. The region between the three surfaces, which the arrow is pointing to, is the zone of Simpson’s paradox.

What can we learn from this? One thing is the importance of randomised trials for comparing medicines. The above data are not a well-designed trial, because almost all the young people took the blue pill, and almost all the older people took the red pill. More generally, as interpreting statistics from the news becomes an increasing part of our daily lives, this example also shows how easy it is, unfortunately, for such statistics to be misleading.

It all adds up

Time has flown since our last post on this blog, and I wanted to re-kindle things here with a post about It All Adds Up, a conference for girls in years 9 to 12 that takes place each year at the Mathematical Institute in Oxford.

Back in January, I gave the plenary talk to the two hundred or so year 10-11s. It was a great experience, and all the more memorable now, when a room of that many people seems a like a world away.

One of the topics I spoke to them about was

how to detect an object from its shadows

Often an object of mathematical interest is high-dimensional and, when visualising or understanding it, we can find a “shadow” that captures important stucture. For some examples inspired by statistics, see this post and this one too.

Let’s imagine we have a mystery object in a box. We can shine a light on the box in the three different directions and observe the shadow. Based on the shadows, we want to understand the object in the box.

For example, if each of the three shadows is a filled-in square, then one possibility for the object is a filled-in cube, but there are other options too.

Are there combinations of three shadows that can’t come from any object?

For example, does there exist an object that has its three shadows given by the three shapes shown: a square, a triangle, and a circle?

It turns out that such an object does exist! It looks like this: (see here for more angles of it and for the files we used to make the 3D print.)

We could also try to design an object that has three shadows given by these three letters (or your initials instead of mine):

How could we show that three shadows could not have come from a particular object? One tool is the Loomis-Whitney inequality, which relates the volume of an object to the area of its three shadows. It says that the areas of the shadows cannot be too small relative to the volume of the object:

We are often interested in higher-dimensional examples. For example, does there exist a four dimensional object that has these shadows?

This picture is from here, where it arises in the context of marginals of quantum systems.

For an even more artistic example, here is a sculture by Matthieu Robert-Ortis, where one projection of a sculpture gives an elephant, while another gives two giraffes:

Thanks to Vicky Neale and Mareli Grady for organising It All Adds Up, and for inviting me to speak there. And thanks to Derek Moulton and Alain Goriely for help with the 3D printing.

Knotty adjectives

First of all let me apologise for my lack of recent posts, finishing a PhD and moving countries is exhausting!

I recently gave a learning seminar on knots, and I would like to share some of my discoveries with you in this post.

What is a knot?

We all create knots in our daily lives, by tying our shoelaces or tangling our headphones. But mathematically a knot is the following:

“A knot is an oriented, locally flat embedding of into “.

we can think of this as taking a string, tying some sort of knot and then gluing the two ends of the string together! We then add an arrow to ‘orient’ the knot: imagine the string flowing in one direction.

The ‘locally flat’ roughly means that zooming in to any section of our knot we see what looks like a straight line travelling through space, and the fact we are mapping the circle into the three-sphere can be ignored for now: we can just imagine that is the Euclidean space .

Knot diagrams

We can view a knot in the form of a knot diagram $ latex D $. This is the projection of the knot onto a plane in , where we symbolise which part of our string is behind another part by using under and over crossings. Here are some examples of well known knots and their diagrams:

 

In 1932 Reidemeister proved that any two diagrams that represent equivalent knots are related by a sequence of Reidemeister moves along side smooth deformations that preserve the arcs and crossings. These moves are as follows:

 

Knot adjectives!

When we define a new type of mathematical object, such as a group, topological space, or knot, it helps to be able to describe specific subsets of those objects and so we define characteristics that a group/space/knot may have. For instance a group could be cyclic or abelian, a space could be connected or compact and a knot could be…. below we will introduce some adjectives!

Prime

A prime knot is one that cannot be written as the connect sum of two or more knots, which aren’t the unknot. The connect sum operation takes two knots, cuts them, and then glues them to each other as below. We have to make sure that we do this in a way which agrees with the orientations.

The unknot and the trefoil are both prime knots, whereas the granny knot is the connect sum of a trefoil with itself and the reef knot is the connect sum of the trefoil with its reflection!

Ribbon

A knot is called ribbon  if it bounds a self intersecting disc with only ribbon singularities. A ribbon singularity is shown below, where the knot is in red and the disk that it bounds is in blue.

The reef knot is an example of a ribbon knot:

 

Alternating

An alternating knot is one such that, when you travel along the knot diagram you encounter an ‘over then under then over then under’ pattern in the crossings. The trefoil is an example of an alternating knot: try placing your finger on the trefoil and flowing along the string!

That’s all the knotty adjectives I have time for now: I’ll be back soon to explore the relationship between knots and braids!

Visualizing statistical models – it’s child’s play

Before you ask a mathematician if they can visualize the fourth dimension, ask them if they can truly visualize a three-dimensional object, like the boundary of a four-dimensional football. If they tell you it’s easy, and their name isn’t Maryna Viazovska, they’re probably lying.

Making an accurate picture of an object from a high dimensional space is very challenging. In this blog post we’ll see a surprising case where it turns out to be possible. We’ll visualize an interesting seven-dimensional object, which comes from a question in statistics.

Let’s consider the probability that each of the teams in the quarter-finals of the Men’s FIFA 2018 World Cup would win. The teams were (Uruguay, France, Brazil, Belgium, Russia, Croatia, Sweden, England). Today we know the probabilities of the teams winning, in that order, are , because France has already won. Back on 3rd July the probabilities (according to FiveThirtyEight) were , and on 7th July the probabilities were .

In a recent project we were studying which probability distributions lie in a particular statistical model. We found out that our statistical model is given by inequalities that the eight probabilities need to satisfy. If we call the probabilities , the inequalities are:

The probabilities have to sum to 1, so . We want to visualize the part of seven-dimensional space in which the inequalities hold. How can we do it?

The first step is to notice that some combinations of letters do not affect whether the inequalities hold or not. They are:

So we can apply a change of coordinates that removes these three directions, leaving something four-dimensional. Finally, to get something three-dimensional we can assume that the four remaining coordinates lie on the sphere.

We end up with a picture that looks like this:

The part of space that lies inside the statistical model are the points outside either the blue blob, the green blob, or the yellow blob.

These days, we have an even better way to visualize the statistical model, truly in 3D. It even doubles-up as a handmade toy for children.

Order yours here

 

We can’t help but wonder – which other children’s toys are really statistical models in disguise?

A duality of pictures

Duality relates objects, which seem different at first but turn out to be similar. The concept of duality occurs almost everywhere in maths. If two objects seem different but are actually the same, we can view each object in a “usual” way, and in a “dual” way – the new vantage point is helpful for new understanding of the object.  In this blog post we’ll see a pictorial example of a mathematical duality.

How are these two graphs related?

In the first graph, we have five vertices, the five black dots, and six green edges which connect them. For example, the five vertices could represent cities (San Francisco, Oakland, Sausalito etc. ) and the edges could be bridges between them.

In the second graph, the role of the cities and the bridges has swapped. Now the bridges are the vertices, and the edges (or hyperedges) are the cities. For example, we can imagine that the cities are large metropolises and the green vertices are the bridge tolls between one city and the next.

Apart from swapping the role of the vertices and the edges, the information in the two graphs is the same. If we shrink each city down to a dot in the second graph, and grow each bridge toll into a full bridge, we get the first graph. We will see that the graphs are dual to each other.

We represent each graph by a labeled matrix: we label the rows by the vertices and the columns by the edges, and we put a in the matrix whenever the vertex is in the edge. For example, the entry for vertex and edge is , because edge contains vertex . The matrix on the left is for the first graph, and the one on the right is for the second graph.

We can see that the information in the two graphs is the same from looking at the two matrices – they are the same matrix, transposed (or flipped). The matrix of a hypergraph is the transpose of the matrix of the dual hypergraph.

Mathematicians are always on the look-out for hidden dualities between seemingly different objects, and we are happy when we find them. For example, in a recent project we studied the connection between graphical models, from statistics, and tensor networks, from physics. We showed that the two constructions are the duals of each other, using the hypergraph duality we saw in this example.

Flattening a cube

If you conduct a survey, among some friends, consisting of three YES/NO questions, how can you summarize the responses?

I conducted a survey recently at a conference. The three questions were:

• Is it your first time at the Mathematisches Forschungsinstitut Oberwolfach?
• Do you like the weather?
• Have you played any games?

There are eight options for how someone could respond to three YES/NO questions. Taking YES=1, and NO=0, the eight options are labelled by the binary strings: 000, 001, 010, 100, 011, 101, 110, 111.

We can think of 0 and 1 as coordinates in space, and arrange the eight numbers into a cube:

This 3D arrangement reflects the fact that there are three questions in the survey. Since our dataset is small, there’s not much need for further analysis to compress or visualize the data. But for a larger survey, we will summarize the structural information in the data using principal components.

The first step of principal component analysis is to restructure the 3D cube of data into a 2D matrix. This is called “flattening” the cube. We combine two YES/NO questions from the survey into a single question with four possible responses. There are three choices for which questions to combine, so there are three possible ways to flatten the cube into a matrix:

Our analysis of the data depends on which flattening we choose! Generally speaking, it’s bad news if an arbitrary decision has an impact on the conclusions of an analysis.

So we need to understand…

How do the principal components depend on the choice of flattening?

This picture give an answer to that question:

All points inside the star-shaped surface correspond to valid combinations of principal components from the three flattenings, while points outside are the invalid combinations. More details can be found here.

Understanding the brain using topology: the Blue Brain project

ALERT ALERT! Applied topology has taken the world has by storm once more. This time techniques from algebraic topology are being applied to model networks of neurons in the brain, in particular with respect to the brain processing information when exposed to a stimulus. Ran Levi, one of the ‘co-senior authors’ of the recent paper published in Frontiers in Computational Neuroscience is based in Aberdeen and he was kind enough to let me show off their pictures in this post. The paper can be found here.

So what are they studying?

When a brain is exposed to a stimulus, neurons fire seemingly at random. We can detect this firing and create a ‘movie’ to study. The firing rate increases towards peak activity, after which it rapidly decreases. In the case of chemical synapses, synaptic communication flows from one neuron to another and you can view this information by drawing a picture with neurons as dots and possible flows between neurons as lines, as shown below. In this image more recent flows show up as brighter.

Image credit: Blue Brain project. This image shows a depiction of neurons and synaptic connections between them. The more recently a synaptic communication has been fired, the brighter it is depicted in the image.

Numerous studies have been conducted to better understand the pattern of this build up and rapid decrease in neuron spikes and this study contains significant new findings as to how neural networks are built up and decay throughout the process, both at a local and global scale. This new approach could provide substantial insights into how the brain processes and transfers information. The brain is one of the main mysteries of medical science so this is huge! For me the most exciting part of this is that the researchers build their theory through the lens of Algebraic Topology and I will try to explain the main players in their game here.

Topological players: cliques and cavities

The study used a digitally constructed model of a rats brain, which reproduced neuron activity from experiments in which the rats were exposed to stimuli. From this model ‘movies’ of neural activity could be extracted and analysed. The study then compared their findings to real data and found that the same phenomenon occurred.

Neural networks have been previously studied using graphs, in which the neurons are represented by vertices and possible synaptic connections between neurons by edges. This throws away quite a lot of information since during chemical synapses the synaptic communication flows, over a miniscule time period, from one neuron to another. The study takes this into account and uses directed graphs, in which an edge has a direction emulating the synaptic flow. This is the structural graph of the network that they study. They also study functional graphs, which are subgraphs of the structural graph. These contain only the connections that fire within a certain ‘time bin’. You can think of these as synaptic connections that occur in a ‘scene’ of the whole ‘movie’. There is one graph for each scene and this research studies how these graphs change throughout the movie.

The main structural objects discovered and consequentially studied in these movies are subgraphs called directed cliques. These are graphs for which every vertex is connected to every other vertex. There is a source neuron from which all edges are directed away, and a sink neuron for which all edges are directed towards. In this sense the flow of information has a natural direction. Directed cliques consisting of n neurons are called simplices of dimension (n-1). Certain sub-simplices of a directed clique for their own directed cliques, when the vertices in the sub-simplices contain their own source and sink neuron, called sub-cliques. Below are some examples of the directed clique simplices.

Image credit: EPFL. This image shows examples of directed cliques.

And the images below show these simplices occurring naturally in the neural network.

Image credit: Frontiers in Computational Neuroscience, ‘Cliques of Neurons Bound into Cavities Provide a Missing Link between Structure and Function’, Figure 1A. This image shows a reconstructed microcircuit produced using the model of neural activity. A 5-neuron clique is shown in red.

Image credit: Frontiers in Computational Neuroscience, ‘Cliques of Neurons Bound into Cavities Provide a Missing Link between Structure and Function’, Figure 1B3. This image shows a zoomed in depiction of the 5 neuron clique in the image above, with its corresponding simplex on the right.

 

Image credit: Frontiers in Computational Neuroscience, ‘Cliques of Neurons Bound into Cavities Provide a Missing Link between Structure and Function’, Adaptation of Figure 2C. This image shows a 6-simplex (a directed clique with 7 vertices) on the left and a 7-simplex on the right, with representations of how these cliques appear in the neural network shown in the centre.

 

The researchers found that over time, simplices of higher and higher dimension were born in abundance, as synaptic communication increased and information flowed between neurons. Then suddenly all cliques vanished, the brain had finished processing the new information. This relates the neural activity to an underlying structure which we can now study in more detail. It is a very local structure, simplices of up to 7 dimensions were detected, a clique of 8 neurons in a microcircuit containing tens of thousands. It was the pure abundance of this local structure that made it significant, where in this setting local means concerning a small number of vertices in the structural graph.

As well as considering this local structure, the researchers also identified a global structure in the form of cavities. Cavities are formed when cliques share neurons, but not enough neurons to form a larger clique. An example of this sharing is shown below, though please note that this is not yet an example of a cavity. When many cliques together bound a hollow space, this forms a cavity. Cavities represent homology classes, and you can read my post on introducing homology here. An example of a 2 dimensional cavity is also shown below.

An example of simplices sharing neurons.

Image credit: Frontiers in Computational Neuroscience, ‘Cliques of Neurons Bound into Cavities Provide a Missing Link between Structure and Function’, Figure 5A. This image shows an example of a two dimensional cavity. It is bounded by 2 simplicies (triangles) which are directed cliques with 3 neurons.

 

The graph below shows the formation of cavities over time. The x-axis corresponds to the first Betti number, which gives an indication of the number of 1 dimensional cavities, and the y-axis similarly gives an indication of the number of 3 dimensional cavities, via the third Betti number. The spiral is drawn out over time as indicated by the text specifying milliseconds on the curve. We see that at the beginning there is an increase in the first Betti number, before an increase in the third alongside a decrease in the first, and finally a sharp decrease to no cavities at all. Considering the neural movie, we view this as an initial appearance of many 1 dimensional simplices, creating 1 dimensional cavities. Over time, the number of 2 and 3 dimensional simplices increases, by filling in extra connections between 1 dimensional simplices, so the lower dimensional cavities are replaced with higher dimensional ones. When the number of higher dimensional cavities is maximal, the whole thing collapses. The brain has finished processing the information!

Image credit: Frontiers in Computational Neuroscience, ‘Cliques of Neurons Bound into Cavities Provide a Missing Link between Structure and Function’, Figure 6B

 

The time dependent formation of the cliques and cavities in this model was interpreted to try and measure both local information flow, influenced by the cliques, and global flow across the whole network, influenced by cavities.

So why is topology important?

These topological players provide a strong mathematical framework for measuring the activity of a neural network, and the process a brain undergoes when exposed to stimuli. The framework works without parameters (for example there is no measurement of distance between neurons in the model) and one can study the local structure by considering cliques, or how they bind together to form a global structure with cavities. By continuing to study the topological properties of these emerging and disappearing structures alongside neuroscientists we could come closer to understanding our own brains! I will leave you with a beautiful artistic impression of what is happening.

Image credit: Blue Brain project. This image shows an artists depiction of their interpretation of the results, projected into 3 dimensions. The simplices are represented by the clique-like small structures and the centre is the artists depiction of a cavity.

 

There is a great video of Kathryn Hess (EPFL) speaking about the project, watch it here.

For those of you who want to read more, check out the following blog and news articles (I’m sure there will be more to come and I will try to update the list)

Frontiers blog

Wired article

Newsweek article