Abstract
The supervised learning problem to determine a neural network approximation R-d (sic) x (sic)Sigma(K)(k=1) (beta) over cap (k)e(i omega k.x) with one hidden layer is studied as a random Fourier features algorithm. The Fourier features, i.e., the frequencies omega(k) is an element of R-d, are sampled using an adaptive Metropolis sampler. The Metropolis test accepts proposal frequencies omega'(k), having corresponding amplitudes (beta) over cap'(k), with the probability min {1, (vertical bar(beta) over cap 'k vertical bar/vertical bar(beta) over cap (k)vertical bar gamma}, for a certain positive parameter gamma, determined by minimizing the approximation error for given computational work. This adaptive, non-parametric stochastic method leads asymptotically, as K -> infinity, to equidistributed amplitudes vertical bar(beta) over cap (k)vertical bar, analogous to deterministic adaptive algorithms for differential equations. The equidistributed amplitudes are shown to asymptotically correspond to the optimal density for independent samples in random Fourier features methods. Numerical evidence is provided in order to demonstrate the approximation properties and efficiency of the proposed algorithm. The algorithm is tested both on synthetic data and a real-world high-dimensional benchmark.