BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260522T150116Z
LOCATION:C146
DTSTART;TZID=America/Chicago:20181113T113000
DTEND;TZID=America/Chicago:20181113T120000
UID:submissions.supercomputing.org_SC18_sess179_pap450@linklings.com
SUMMARY:Scaling Embedded In Situ Indexing with DeltaFS
DESCRIPTION:Qing Zheng, Charles D. Cranor, Danhao Guo, Gregory R. Ganger, 
 George Amvrosiadis, and Garth A. Gibson (Carnegie Mellon University) and B
 radley W. Settlemyer, Gary Grider, and Fan Guo (Los Alamos National Labora
 tory)\n\nAnalysis of large-scale simulation output is a core element of sc
 ientific inquiry, but analysis queries may experience significant I/O over
 head when the data is not structured for efficient retrieval. While in-sit
 u processing allows for improved time-to-insight for many applications, sc
 aling in-situ frameworks to hundreds of thousands of cores can be difficul
 t in practice. The DeltaFS in-situ indexing is a new approach for in-situ 
 processing of massive amounts of data to achieve efficient point and small
 -range queries. This paper describes the challenges and lessons learned wh
 en scaling this in-situ processing function to hundreds of thousands of co
 res. We propose techniques for scalable all-to-all communication that is m
 emory and bandwidth efficient, concurrent indexing, and specialized LSM-Tr
 ee formats. Combining these techniques allows DeltaFS to control the cost 
 of in-situ processing while maintaining 3 orders of magnitude query speedu
 p when scaling alongside the popular VPIC particle-in-cell code to 131,072
  cores.\n\nTag: Clouds and Distributed Computing, File Systems, I/O, Stora
 ge\n\nRegistration Category: Tech Program Reg Pass\n\nSession Chair: Denni
 s Gannon (Indiana University, The eScience Cloud)\n\n
END:VEVENT
END:VCALENDAR