“The anchovies were nowhere near the sardines and the tuna. That’s because they were near the pizza toppings.

But it was only a problem because this was a three-dimensional grocery store. If it had been a thirty-dimensional grocery store they could have been near the pizza and the sardines.”

I’ve been putting off a post about phase space for nearly a year now, and watching this talk made me remember that I’ll soon have to do it.

Geoff Hinton talks here about a number of different spaces that are not the physical space that math was initially developed on.

  • The bag of words model of a document takes each word in a text to be a dimension of the document, with like 80,000 possibilities each or however many words there are in English. (The 80,000 possibilities are the underlying corpus.)
  • Latent semantic analysis of the bag-of-words type is how Google now does its search rankings. (PageRank only constitutes something like 30% of SERP ranking anymore, because [a] it’s too easy to game and [b] it’s inspecific to what’s being searched on. Domain authority and LSI comprise the rest. <—separate article)
  • Seeing a pixel-by-pixel representation of a 2-D image as a list vector is problematic because the first pixel in a 200×300 Facebook profile image is next to the second pixel and also next to the 201st pixel. I.e. one needs a 2-array.
  • Hinton talks about abstract feature space and energies — equivalently evolutionary fitness or economic utility and ravines and mountains upon this manifold.
  • The number of dimensions here is like the number of parameters (same as free parameters or degrees of freedom or arbitrary parameters in stats class) and in a neural net each “synapse” or graph edge is a lever you can pull.
  • The same metaphor — and this is a metaphor in a grand sense which I hope to cover before the year is up — applies to the equalizer on your uncle’s home stereo, i.e. the number of terms in the Fourier decomposition.

About isomorphismes

Argonaut: someone engaged in a dangerous but potentially rewarding adventure.
Video | This entry was posted in Uncategorized and tagged , , , , , , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s