Deep stacked stochastic configuration networks for lifelong learning of non-stationary data streams

Abstract

The concept of SCN offers a fast framework with universal approximation guarantee for lifelong learning of non-stationary data streams. Its adaptive scope selection property enables for proper random generation of hidden unit parameters advancing conventional randomized approaches constrained with a fixed scope of random parameters. This paper proposes deep stacked stochastic configuration network (DSSCN) for continual learning of non-stationary data streams which contributes two major aspects: 1) DSSCN features a self-constructing methodology of deep stacked network structure where hidden unit and hidden layer are extracted automatically from continuously generated data streams; 2) the concept of SCN is developed to randomly assign inverse covariance matrix of multivariate Gaussian function in the hidden node addition step bypassing its computationally prohibitive tuning phase. Numerical evaluation and comparison with prominent data stream algorithms under two procedures: periodic hold-out and prequential test-then-train processes demonstrate the advantage of proposed methodology.Ministry of Education (MOE)Nanyang Technological UniversityThis work is fully supported by Ministry of Education, Republic of Singapore, Tier 1 Research Grant and NTU Start-up Grant. The authors also thank Singapore Institute of Manufacturing Technology (SIMTech), Singapore for providing RFID dataset and acknowledge the assistance of Mr. MD. Meftahul Ferdaus for Latex typesetting of this paper