BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260522T150117Z
LOCATION:D173
DTSTART;TZID=America/Chicago:20181111T115500
DTEND;TZID=America/Chicago:20181111T122000
UID:submissions.supercomputing.org_SC18_sess163_ws_works111@linklings.com
SUMMARY:A Practical Roadmap for Provenance Capture and Data Analysis in Sp
 ark-Based Scientific Workflows
DESCRIPTION:Thaylon Guedes (Fluminense Federal University, Fluminense Fede
 ral University, Brazil); Vitor Silva and Marta Mattoso (Federal University
  of Rio de Janeiro); and Marcos Bedo and Daniel Oliveira (Fluminense Feder
 al University, Fluminense Federal University, Brazil)\n\nWhenever high-per
 formance computing applications meet data-intensive scalable systems, an a
 ttractive approach is the use of Apache Spark for the management of scient
 ific workflows. Spark provides several advantages such as being widely sup
 ported and granting efficient in-memory data management for large-scale ap
 plications. However, Spark still lacks support for data tracking and workf
 low provenance. Additionally, Spark’s memory management requires accessing
  all data movements between the workflow activities. Therefore, the runnin
 g of legacy programs on Spark is interpreted as a “black-box” activity, wh
 ich prevents the capture and analysis of implicit data movements. Here, we
  present SAMbA, an Apache Spark extension for the gathering of prospective
  and retrospective provenance and domain data within distributed scientifi
 c workflows. Our approach relies on enveloping both RDD structure and data
  contents at runtime so that (i) RDD-enclosure consumed and produced data 
 are captured and registered by SAMbA in a structured way, and (ii) provena
 nce data can be queried during and after the execution of scientific workf
 lows. By following the W3C PROV representation, we model the roles of RDD 
 regarding prospective and retrospective provenance data. Our solution prov
 ides mechanisms for the capture and storage of provenance data without jeo
 pardizing Spark’s performance. The provenance retrieval capabilities of ou
 r proposal are evaluated in a practical case study, in which data analytic
 s are provided by several SAMbA parameterizations.\n\nTag: Reproducibility
 , Scientific Computing, Scientific Workflows, Workflows, HPC, Data Intensi
 ve\n\nRegistration Category: Workshop Reg Pass\n\n
END:VEVENT
END:VCALENDAR