BEGIN:VCALENDAR
VERSION:2.0
PRODID:Linklings LLC
BEGIN:VTIMEZONE
TZID:America/Chicago
X-LIC-LOCATION:America/Chicago
BEGIN:DAYLIGHT
TZOFFSETFROM:-0600
TZOFFSETTO:-0500
TZNAME:CDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0500
TZOFFSETTO:-0600
TZNAME:CST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260522T150116Z
LOCATION:D167/174
DTSTART;TZID=America/Chicago:20181112T170000
DTEND;TZID=America/Chicago:20181112T173000
UID:submissions.supercomputing.org_SC18_sess151_ws_mlhpce123@linklings.com
SUMMARY:Large Minibatch Training on Supercomputers with Improved Accuracy 
 and Reduced Time to Train
DESCRIPTION:Valeriu Codreanu and Damian Podareanu (SURFsara) and Vikram Sa
 letore (Intel Corporation)\n\nFor the past 6 years, the ILSVRC competition
  and the ImageNet dataset have attracted a lot of interest from the Comput
 er Vision community, allowing for state-of-the-art accuracy to grow tremen
 dously. This should be credited to the use of deep artificial neural netwo
 rk designs. As these became more complex, the storage, bandwidth, and comp
 ute requirements increased. This means that with a non-distributed approac
 h, even when using the most high-density server available, the training pr
 ocess may take weeks, making it prohibitive. Furthermore, as datasets grow
 , the representation learning potential of deep networks grows as well by 
 using more complex models. This synchronicity triggers a sharp increase in
  the computational requirements and motivates us to explore the scaling be
 haviour on petaflop scale supercomputers. In this paper we describe the ch
 allenges and novel solutions needed in order to train ResNet-50 in a large
  scale environment. We demonstrate above 90 percent scaling efficiency and
  a training time of 28 minutes using up to 104K x86 cores. This is support
 ed by software tools from Intel's ecosystem. Moreover, we show that with r
 egular 90 - 120 epoch train runs we can achieve a top-1 accuracy as high a
 s 77 percent for the unmodified ResNet-50 topology. We also introduce the 
 novel Collapsed Ensemble technique that allows us to obtain a 77.5 percent
  top-1 accuracy, similar to that of a ResNet-152, while training a unmodif
 ied ResNet-50 topology for the same fixed training budget.\n\nTag: Deep Le
 arning, Machine Learning\n\nRegistration Category: Workshop Reg Pass\n\n
END:VEVENT
END:VCALENDAR