Sparser Johnson-Lindenstrauss Transforms

Abstract

We give two different Johnson-Lindenstrauss distributions, each with column sparsity $s = \Theta(\epsilon^{−1} log(1/\delta))$ and embedding into optimal dimension $k = O(\epsilon^{−2} log(1/\delta))$ to achieve distortion $1\pm \epsilon$ with probability $1−\delta$. That is, only an $O(\epsilon)$-fraction of entries are non-zero in each embedding matrix in the supports of our distributions. These are the first distributions to provide o(k) sparsity for all values of $\epsilon$, $\delta$. Previously the best known construction obtained $s = \overset \sim \Theta (\epsilon^{-1} log^2(1/\delta))^1$ [Dasgupta-Kumar-Sarlós, STOC 2010]. In addition, one of our distributions can be sampled from a seed of $O(log(1/\delta) log d)$ uniform random bits. Some applications that use Johnson-Lindenstrauss embeddings as a black box, such as those in approximate numerical linear algebra ([Sarlós, FOCS 2006], [Clarkson-Woodruff, STOC 2009]), require exponentially small $\delta$. Our linear dependence on $log(1/\delta)$ in the sparsity is thus crucial in these applications to obtain speedup.Engineering and Applied Science