Skip to main content

Research Repository

Advanced Search

High-Dimensional Function Approximation with Neural Networks for Large Volumes of Data

High-Dimensional Function Approximation with Neural Networks for Large Volumes of Data Thumbnail


Abstract

Approximation of high-dimensional functions is a challenge for neural networks due to the curse of dimensionality. Often the data for which the approximated function is defined resides on a low-dimensional manifold and in principle the approximation of the function over this manifold should improve the approximation performance. It has been show that projecting the data manifold into a lower dimensional space, followed by the neural network approximation of the function over this space, provides a more precise approximation of the function than the approximation of the function with neural networks in the original data space. However, if the data volume is very large, the projection into the low-dimensional space has to be based on a limited sample of the data. Here, we investigate the nature of the approximation error of neural networks trained over the projection space. We show that such neural networks should have better approximation performance than neural networks trained on high-dimensional data even if the projection is based on a relatively sparse sample of the data manifold. We also find that it is preferable to use a uniformly distributed sparse sample of the data for the purpose of the generation of the low-dimensional projection. We illustrate these results considering the practical neural network approximation of a set of functions defined on high-dimensional data including real world data as well.

Acceptance Date Jan 7, 2017
Publication Date Feb 1, 2018
Journal IEEE Transactions on Neural Networks and Learning Systems
Print ISSN 2162-237X
Publisher Institute of Electrical and Electronics Engineers (IEEE)
Pages 500-508
DOI https://doi.org/10.1109/TNNLS.2017.2651985
Keywords Big data, function approximation, high-dimensional data, manifold mapping, neural networks
Publisher URL http://dx.doi.org/10.1109/TNNLS.2017.2651985

Files







Downloadable Citations