“The anchovies were nowhere near the sardines and the tuna. That’s because they were near the pizza toppings.

But it was only a problem because this was a three-dimensional grocery store. If it had been a thirty-dimensional grocery store they could have been near the pizza and the sardines.”

I’ve been putting off a post about phase space for nearly a year now, and watching this talk made me remember that I’ll soon have to do it.

Geoff Hinton talks here about a number of different spaces that are not the physical space that math was initially developed on.

The bag of words model of a document takes each word in a text to be a dimension of the document, with like 80,000 possibilities each or however many words there are in English. (The 80,000 possibilities are the underlying corpus.)

Latent semantic analysis of the bag-of-words type is how Google now does its search rankings. (PageRank only constitutes something like 30% of SERP ranking anymore, because [a] it’s too easy to game and [b] it’s inspecific to what’s being searched on. Domain authority and LSI comprise the rest. <—separate article)

Seeing a pixel-by-pixel representation of a 2-D image as a list vector is problematic because the first pixel in a 200×300 Facebook profile image is next to the second pixel and also next to the 201st pixel. I.e. one needs a 2-array.

Hinton talks about abstract feature space and energies — equivalently evolutionary fitness or economic utility — and ravines and mountains upon this manifold.

The number of dimensions here is like the number of parameters (same as free parameters or degrees of freedom or arbitrary parameters in stats class) and in a neural net each “synapse” or graph edge is a lever you can pull.

The same metaphor — and this is a metaphor in a grand sense which I hope to cover before the year is up — applies to the equalizer on your uncle’s home stereo, i.e. the number of terms in the Fourier decomposition.