SAPIENTML: Synthesizing Machine Learning Pipelines by Learning from Human-Written Solutions

5Citations
Citations of this article
20Readers
Mendeley users who have this article in their library.
Get full text

Abstract

Automatic machine learning, or AutoML, holds the promise of truly democratizing the use of machine learning (ML), by substantially automating the work of data scientists. However, the huge combinatorial search space of candidate pipelines means that current AutoML techniques, generate sub-optimal pipelines, or none at all, especially on large, complex datasets. In this work we propose an AutoML technique SapientML, that can learn from a corpus of existing datasets and their human-written pipelines, and efficiently generate a high-quality pipeline for a predictive task on a new dataset. To combat the search space explosion of AutoML, SapientML employs a novel divide-and-conquer strategy realized as a three-stage program synthesis approach, that reasons on successively smaller search spaces. The first stage uses meta-learning to predict a set of plausible ML components to constitute a pipeline. In the second stage, this is then refined into a small pool of viable concrete pipelines using a pipeline dataflow model derived from the corpus. Dynamically evaluating these few pipelines, in the third stage, provides the best solution. We instantiate SapientML as part of a fully automated tool-chain that creates a cleaned, labeled learning corpus by mining Kaggle, learns from it, and uses the learned models to then synthesize pipelines for new predictive tasks. We have created a training corpus of 1,094 pipelines spanning 170 datasets, and evaluated SapientML on a set of 41 benchmark datasets, including 10 new, large, real-world datasets from Kaggle, and against 3 state-of-the-art AutoML tools and 4 baselines. Our evaluation shows that SapientML produces the best or comparable accuracy on 27 of the benchmarks while the second best tool fails to even produce a pipeline on 9 of the instances. This difference is amplified on the 10 most challenging benchmarks, where SapientML wins on 9 instances with the other tools failing to produce pipelines on 4 or more benchmarks.

References Powered by Scopus

No free lunch theorems for optimization

10752Citations
N/AReaders
Get full text

A survey on Image Data Augmentation for Deep Learning

8303Citations
N/AReaders
Get full text

Sequential model-based optimization for general algorithm configuration

1764Citations
N/AReaders
Get full text

Cited by Powered by Scopus

AutoML from Software Engineering Perspective: Landscapes and Challenges

9Citations
N/AReaders
Get full text

On taking advantage of opportunistic meta-knowledge to reduce configuration spaces for automated machine learning

2Citations
N/AReaders
Get full text

Enhancing Program Synthesis with Large Language Models Using Many-Objective Grammar-Guided Genetic Programming

1Citations
N/AReaders
Get full text

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Cite

CITATION STYLE

APA

Saha, R. K., Ura, A., Mahajan, S., Zhu, C., Li, L., Hu, Y., … Prasad, M. R. (2022). SAPIENTML: Synthesizing Machine Learning Pipelines by Learning from Human-Written Solutions. In Proceedings - International Conference on Software Engineering (Vol. 2022-May, pp. 1932–1944). IEEE Computer Society. https://doi.org/10.1145/3510003.3510226

Readers over time

‘22‘23‘24‘2505101520

Readers' Seniority

Tooltip

PhD / Post grad / Masters / Doc 7

70%

Professor / Associate Prof. 1

10%

Lecturer / Post doc 1

10%

Researcher 1

10%

Readers' Discipline

Tooltip

Computer Science 10

83%

Engineering 1

8%

Arts and Humanities 1

8%

Save time finding and organizing research with Mendeley

Sign up for free
0