Key-Value Query-Response associative memory model

[SergeyGubanov]

Пара формул про связь с так называемыми "Random Features":

$$ \sum_{n = 1}^{N} \exp \left( \langle \vec{K}_{n}, \vec{Q} \rangle + \langle \vec{V}_{n}, \vec{R} \rangle \right) = \sum_{n = 1}^{N} \exp \left( \langle \vec{K}_{n}, \vec{Q} \rangle \right) \, \exp \left( \langle \vec{V}_{n}, \vec{R} \rangle \right) = \langle {\bf K}(Q), {\bf V}(R) \rangle, $$

$$ {\bf K}(Q) \in \mathbb{R}^{N}, \quad {\bf V}(R) \in \mathbb{R}^{N}, $$

$$ {\bf K}(Q) = \left\{ \exp \left( \langle \vec{K}_{1}, \vec{Q} \rangle \right), \,\exp \left( \langle \vec{K}_{2}, \vec{Q} \rangle \right), \ldots, \,\exp \left( \langle \vec{K}_{N}, \vec{Q} \rangle \right) \right\}, $$

$$ {\bf V}(R) = \left\{ \exp \left( \langle \vec{V}_{1}, \vec{R} \rangle \right), \,\exp \left( \langle \vec{V}_{2}, \vec{R} \rangle \right), \ldots, \,\exp \left( \langle \vec{V}_{N}, \vec{R} \rangle \right) \right\}. $$