BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260522T150116Z
LOCATION:C140/142
DTSTART;TZID=America/Chicago:20181115T133000
DTEND;TZID=America/Chicago:20181115T140000
UID:submissions.supercomputing.org_SC18_sess190_pap425@linklings.com
SUMMARY:Exploring Flexible Communications for Streamlining DNN Ensemble Tr
 aining Pipelines
DESCRIPTION:Randall Pittman, Hui Guan, and Xipeng Shen (North Carolina Sta
 te University) and Seung-Hwan Lim and Robert M. Patton (Oak Ridge National
  Laboratory)\n\nParallel training of a Deep Neural Network (DNN) ensemble 
 on a cluster of nodes is a common practice to train multiple models in ord
 er to construct a model with a higher prediction accuracy. Existing ensemb
 le training pipelines can perform a great deal of redundant operations, re
 sulting in unnecessary CPU usage, or even poor pipeline performance.  In o
 rder to remove these redundancies, we need pipelines with more communicati
 on flexibility than existing DNN frameworks provide.\n\nThis project inves
 tigates a series of designs to improve pipeline flexibility and adaptivity
 , while also increasing performance. We implement our designs using Tensor
 flow with Horovod, and test it using several large DNNs. Our results show 
 that the CPU time spent during training is reduced by 2-11X. Furthermore, 
 our implementation can achieve up to 10X speedups when CPU core limits are
  imposed. Our best pipeline also reduces the average power draw of the ens
 emble training process by 5-16%.\n\nTag: Applications, Cosmology, Data Ana
 lytics, Deep Learning, Machine Learning, Programming Systems, Storage, Vis
 ualization\n\nRegistration Category: Tech Program Reg Pass\n\nSession Chai
 r: Tal Ben-Nun (Lawrence Livermore National Laboratory (LLNL))\n\n
END:VEVENT
END:VCALENDAR