Where to start if you are new to the area?

We recommend to start the exploration of the area by reading the introductory papers below:

Please also check the page with videos. There are many recordings of great introductory lectures by P. Kanerva. Video is a high-level overview of HD/VSA implemented in hardware. 

If you would like to look for papers within a specific direction of HD/VSA research consider searching the collection of publications.

Is there a survey of the area?

Yes, there is a comprehensive two-part survey that extensively covers most of the aspects of HD/VSA:

What type of data structures can be represented by HD/VSA?

HD/VSA can represent and query an impressive range of data structures. Examples of such structures are key-value pairs also known as role-filler bindings, sets, histograms, sequences, graphs, trees, stacks, state automata, and others. There is a comprehensive write-up, which introduces transformation of a plethora of data structures into high-dimensional vectors. Informally, we refer to to this write-up as “cookbook”: 

What is the information capacity of HD/VSA representations?

It is a very common practice to ask about the information capacity of distributed representations used in HD/VSA representations. Some early results in this direction were already obtained in [1] when studying Holographic Reduced Representations model. Later, some results for Matrix Binding of Additive Terms and Binary Spatter Codes models were presented in [2] and [3], respectively. The most general and comprehensive analysis of the information capacity of different HD/VSA models (and also some classes of recurrent neural networks) has been provided in [4] and exnteded further in [5]. So we highly recommend studying the theory in [4] to get a strong grasp on information capacity of HD/VSA models.

What is the "Blessing of Dimensionality"?

This is the term, which is often used to name phenomena opposite to the well-known Curse of Dimensionality. Different areas of mathematics and computer science assign different meanings to the Blessing of Dimensionality.

In HD/VSA, the Blessing of Dimensionality manifistates itself in the increased capacity [1] of distributed representations  for vectors of higher dimensionality.  

In Stochastic computing, the Blessing of Dimensionality manifistates itself in the progressive precision [2] of computations for vectors of   higher dimensionality.  

In Compressed sensing, the Blessing of Dimensionality manifistates itself in the improved quality of the reconstructed signal [3] for the projection matrices with larger number of dimensions. 

In Concentration of measure, the Blessing of Dimensionality manifistates itself in stochastic separation theorems [4], which suggest that  “If the dimension n of the underlying topological vector space is large, then random finite but exponentially large in n samples are linearly separable, with high probability, for a range of practically relevant classes of distributions”.  

Literature: