BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260522T150115Z
LOCATION:D220
DTSTART;TZID=America/Chicago:20181112T160000
DTEND;TZID=America/Chicago:20181112T163000
UID:submissions.supercomputing.org_SC18_sess172_ws_phpsc106@linklings.com
SUMMARY:Balsam: Automated Scheduling and Execution of Dynamic, Data-Intens
 ive HPC Workflows
DESCRIPTION:Michael A. Salim, Thomas D. Uram, J. Taylor Childers, Venkatra
 m Vishwanath, Michael E. Papka, and Prasanna Balaprakash (Argonne National
  Laboratory)\n\nWe introduce the Balsam service to manage high-throughput 
 task scheduling and execution on supercomputing systems. Balsam allows use
 rs to populate a task database with a variety of tasks ranging from simple
  independent tasks to dynamic multi-task workflows. With abstractions for 
 the local resource scheduler and MPI environment, Balsam dynamically packa
 ges tasks into ensemble jobs and manages their scheduling lifecycle. The e
 nsembles execute in a pilot "launcher'' which (i) ensures concurrent, load
 -balanced execution of arbitrary serial and parallel programs with heterog
 eneous processor requirements, (ii) requires no modification of user appli
 cations, (iii) is tolerant of task-level faults and provides several optio
 ns for error recovery, (iv) stores provenance data (e.g task history, erro
 r logs) in the database, (v) supports dynamic workflows, in which tasks ar
 e created or killed at runtime. Here, we present the design and Python imp
 lementation of the Balsam service and launcher. The efficacy of this syste
 m is illustrated using two case studies: hyperparameter optimization of de
 ep neural networks, and high-throughput single-point quantum chemistry cal
 culations. We find that the unique combination of flexible job-packing and
  automated scheduling with dynamic (pilot-managed) execution facilitates e
 xcellent resource utilization. The scripting overheads typically needed to
  manage resources and launch workflows on supercomputers are substantially
  reduced, accelerating workflow development and execution.\n\nTag: Paralle
 l Application Frameworks, Reproducibility, Scientific Computing\n\nRegistr
 ation Category: Workshop Reg Pass\n\n
END:VEVENT
END:VCALENDAR