Lately I’ve been exploring the idea of a vector space with a large number of dimensions (but few degrees of freedom). A model was presented with five degrees of freedom in 500 dimensions (neurons, as it happens).

The question is, given the axes are bit-level, does normal vector manipulation semantics make sense. My contention is it has severe problems.

Continue reading →