BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260522T150115Z
LOCATION:D167/174
DTSTART;TZID=America/Chicago:20181112T103000
DTEND;TZID=America/Chicago:20181112T110000
UID:submissions.supercomputing.org_SC18_sess151_ws_mlhpce110@linklings.com
SUMMARY:Communication-Efficient Parallelization Strategy for Deep Convolut
 ional Neural Network Training
DESCRIPTION:Sunwoo Lee and Ankit Agrawal (Northwestern University), Prasan
 na Balaprakash (Argonne National Laboratory), and Alok Choudhary and Wei-k
 eng Liao (Northwestern University)\n\nTraining modern Convolutional Neural
  Network (CNN) models is extremely time-consuming, and the efficiency of i
 ts parallelization plays a key role in finishing the training in a reasona
 ble amount of time. The well-known parallel synchronous Stochastic Gradien
 t Descent (SGD) algorithm suffers from high costs of inter-process communi
 cation and synchronization. To address such problems, the asynchronous SGD
  algorithm employs a master-slave model for parameter update. However, it 
 can result in a poor convergence rate due to the staleness of gradient. In
  addition, the master-slave model is not scalable when running on a large 
 number of compute nodes. In this paper, we present a communication-efficie
 nt gradient averaging algorithm for synchronous SGD, which adopts a few de
 sign strategies to maximize the degree of overlap between computation and 
 communication. The time complexity analysis shows our algorithm outperform
 s the traditional algorithms that use MPI allreduce-based communication. B
 y training the two popular deep CNN models, VGG-16 and ResNet-50, on Image
 Net dataset, our experiments performed on Cori Phase-I, a Cray XC40 superc
 omputer at NERSC show that our algorithm can achieve up to 2516.36x speedu
 p for VGG-16 and 2734.25x speedup for ResNet-50 when running on up to 8192
  cores.\n\nTag: Deep Learning, Machine Learning\n\nRegistration Category: 
 Workshop Reg Pass\n\n
END:VEVENT
END:VCALENDAR