Foundations and Trends® in Machine Learning
2 total works
Over the last decade, the long-running endeavour to automate high-level processes in machine learning (ML) has risen to mainstream prominence. Beyond this, an even loftier goal is the pursuit of autonomy, which describes the capability of the system to independently adjust an ML solution over a lifetime of changing contexts. This monograph provides an expansive perspective on what constitutes an automated/autonomous ML system. In doing so, the authors survey developments in hyperparameter optimisation, multicomponent models, neural architecture search, automated feature engineering, meta-learning, multi-level ensembling, dynamic adaptation, multi-objective evaluation, resource constraints, flexible user involvement, and the principles of generalisation. Furthermore, they develop a conceptual framework throughout to illustrate one possible way of fusing high-level mechanisms into an autonomous ML system. This monograph lays the groundwork for students and researchers to understand the factors limiting architectural integration, without which the field of automated ML risks stifling both its technical advantages and general uptake.
Automated Deep Learning
by Xuanyi Dong, David Jacob Kedziora, Katarzyna Musial, and Bogdan Gabrys
Published 27 February 2024
Deep learning (DL) has proven to be a highly effective approach for developing models in diverse contexts, including visual perception, speech recognition, and machine translation. Automated deep learning (AutoDL) endeavours to minimize the need for human involvement and is best known for its achievements in neural architecture search (NAS).
In this monograph, the authors examine research efforts into automation across the entirety of an archetypal DL workflow. In so doing, they propose a comprehensive set of ten criteria by which to assess existing work in both individual publications and broader research areas, namely novelty, solution quality, efficiency, stability, interpretability, reproducibility, engineering quality, scalability, generalizability, and eco-friendliness.
Aimed at students and researchers, this monograph provides an evaluative overview of AutoDL in the early 2020s, identifying where future opportunities for progress may exist.
In this monograph, the authors examine research efforts into automation across the entirety of an archetypal DL workflow. In so doing, they propose a comprehensive set of ten criteria by which to assess existing work in both individual publications and broader research areas, namely novelty, solution quality, efficiency, stability, interpretability, reproducibility, engineering quality, scalability, generalizability, and eco-friendliness.
Aimed at students and researchers, this monograph provides an evaluative overview of AutoDL in the early 2020s, identifying where future opportunities for progress may exist.