BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260522T150116Z
LOCATION:C2/3/4 Ballroom
DTSTART;TZID=America/Chicago:20181113T083000
DTEND;TZID=America/Chicago:20181113T170000
UID:submissions.supercomputing.org_SC18_sess322_post140@linklings.com
SUMMARY:Large-Message Size Allreduce at Wire Speed for Distributed Deep Le
 arning
DESCRIPTION:Kenji Tanaka, Yuki Arikawa, Kenji Kawai, Junichi Kato, Tsuyosh
 i Ito, Huy Cu Ngo, Kazutaka Morita, Fumiaki Miura, Takeshi Sakamoto, and S
 atoshi Shigematsu (Japan Telegraph and Telephone Corporation)\n\nIn large-
 scale distributed deep learning, the Allreduce operation for large message
 s (100 KB or more) is critical for gathering gradients from multiple worke
 r nodes and broadcasting the sum of the gradients to them. When the messag
 e is large, the latency in Allreduce operation would make it difficult to 
 take advantage of large-scale distributed deep learning. To reduce the lat
 ency, we devised a dataflow architecture with an Allreduce-specific hardwa
 re accelerator that performs data aggregation and reduction while data is 
 being transferred. The accelerator is designed to immediately start Allred
 uce operation before an entire message is recived. Furthermore, Allreduce 
 can be operated at wire speed by vectorizing the gradients and summing the
 m in parallel. Experimental results reveal that the proposed architecture 
 performs Allreduce at 96% of wire speed for a large message. Moreover, the
  latency of Allreduce is reduced by 65% compared with a state-of-the-art A
 llreduce method when applied for ResNet-50.\n\nRegistration Category: Tech
  Program Reg Pass, Exhibits Reg Pass\n\n
END:VEVENT
END:VCALENDAR