Repository landing page
High-Performance Kernel Machines With Implicit Distributed Optimization and Randomization
Abstract
<p>We propose a framework for massive-scale training of kernel-based statistical models, based on combining distributed convex optimization with randomization techniques. Our approach is based on a block-splitting variant of the alternating directions method of multipliers, carefully reconfigured to handle very large random feature matrices under memory constraints, while exploiting hybrid parallelism typically found in modern clusters of multicore machines. Our high-performance implementation supports a variety of statistical learning tasks by enabling several loss functions, regularization schemes, kernels, and layers of randomized approximations for both dense and sparse datasets, in an extensible framework. We evaluate our implementation on large-scale model construction tasks and provide a comparison against existing sequential and parallel libraries. Supplementary materials for this article are available online.</p- Dataset
- Dataset
- Medicine
- Sociology
- Space Science
- Biological Sciences not elsewhere classified
- Mathematical Sciences not elsewhere classified
- Information Systems not elsewhere classified
- kernel
- multicore machines
- dataset
- cluster
- Randomization
- randomized approximations
- loss functions
- variety
- implementation
- extensible framework
- model construction tasks
- Supplementary materials
- approach
- Implicit
- reconfigured
- sequential
- variant
- Optimization
- directions method
- feature matrices
- multiplier
- article
- Kernel
- training
- memory constraints
- regularization schemes
- optimization
- randomization techniques
- parallelism