SODA: An optimizing scheduler for large-scale stream-based distributed computer systems

83Citations
Citations of this article
66Readers
Mendeley users who have this article in their library.

This article is free to access.

Abstract

This paper describes the SODA scheduler for System S, a highly scalable distributed stream processing system. Unlike traditional batch applications, streaming applications are open-ended. The system cannot typically delay the processing of the data. The scheduler must be able to shift resource allocation dynamically in response to changes to resource availability, job arrivals and departures, incoming data rates and so on. The design assumptions of System S, in particular, pose additional scheduling challenges. SODA must deal with a highly complex optimization problem, which must be solved in real-time while maintaining scalability. SODA relies on a careful problem decomposition, and intelligent use of both heuristic and exact algorithms. We describe the design and functionality of SODA, outline the mathematical components, and describe experiments to show the performance of the scheduler. © 2008 Springer Berlin Heidelberg.

Cite

CITATION STYLE

APA

Wolf, J., Bansal, N., Hildrum, K., Parekh, S., Rajan, D., Wagle, R., … Fleischer, L. (2008). SODA: An optimizing scheduler for large-scale stream-based distributed computer systems. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 5346 LNCS, pp. 306–325). https://doi.org/10.1007/978-3-540-89856-6_16

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free