University Of Tasmania

File(s) under permanent embargo

Resource scheduling and provisioning for processing of dynamic stream workflows under latency constraints

journal contribution
posted on 2023-05-21, 14:39 authored by Alexander Brown, Saurabh GargSaurabh Garg, Erin MontgomeryErin Montgomery, Ujjwal K C
There has been an increasing demand for large-scale processing of stream workflows. Many streaming data sources, such as video and audio, differ in terms of the type and density of information they contain over time, which require dynamic workflow structures where paths change in response to data variations. Additionally, information is often only inferred from examining discrete chunks and hence, cannot be arbitrarily parallelised like other streaming data sources. Stream processes using these data sources often operate as dynamic workflows, where different paths are selected based on prior processing information, relating to differences in the nature of the data itself. To better enable low latency analysis in such scenarios, this work investigates strategies to schedule and provision streaming dynamic workflow scenarios of various complexities and degrees of unpredictability. It is found that, as scenarios become increasingly uncertain and unpredictable, it is better to use simpler strategies which do not make assumptions about workflow structures or completion times.


Publication title

Future Generation Computer Systems








School of Information and Communication Technology


Elsevier Science Bv

Place of publication

Po Box 211, Amsterdam, Netherlands, 1000 Ae

Rights statement

© 2022 Elsevier B.V. All rights reserved.

Repository Status

  • Restricted

Socio-economic Objectives

Rehabilitation or conservation of terrestrial environments; Applied computing; Native forests