Trainieren Grosser Dl-Modelle über Serverlose Architektur unter Verwendung eines Kommunikationskanals auf Cloud-Speicherdienstbasis

Details

Veröffentlichungs-Nr.

EP4293578

Aktenzeichen

EP23170342

Anmeldetag

27. April 2023

Veröffentlichung

20. Dezember 2023

Rechtsraum

EP

IPC

G06N3/063G06N3/045G06N3/098G06F9/50

Abstract

State of the art methods require size of DL model, or its gradients be less than maximum data item size of storage used as a communication channel for model training with serverless platform. Embodiments of the present disclosure provide a method and system for training large DL models via serverless architecture using communication channel when the gradients are larger than maximum size of one data item allowed by the channel. Gradients that are generated by each worker during current training instance, are chunked into segments and stored in the communication channel. Corresponding segments of each worker are aggregated by aggregators and stored back. Each of the aggregated corresponding segments are read by each worker to generate an aggregated model to be used during successive training instance. Optimization techniques are used for reading-from and writing-to the channel resulting in significant improvement in the performance and the cost of training.

Anmelder

Firma

Tata Consultancy Services Limited

Land

🇮🇳 Indien

Fachgebiete