Verfahren und System zum Trainieren eines Maschinenlernmodells mit Transformatorarchitektur

Details

Veröffentlichungs-Nr.

EP4800604

Anmeldetag

28. Februar 2025

Veröffentlichung

2. September 2026

Rechtsraum

EP

IPC

G06N3/0455, G06N3/0895

Abstract

The present invention relates to a method for training a machine learning model (200) having a transformer architecture (202), the method comprising: providing (S1) time series training data (203) to the machine learning model (200) as input data (204), the machine learning model (200) processing the time series training data (203) via at least one attention layer (206) of at least one encoder layer (208) of the transformer architecture (202); applying (S2) a dynamic region masking (210) of the time series training data (203) based on attention weights of the attention layer (206) to derive masked training data(212); providing (S3) the masked training data (212) to the machine learning model (200) as input data (204); performing (S4) semi-supervised contrastive learning (214) of a latent space representation (216) of the time series training data (203) and the masked training data (212) derived from the at least one encoder layer (208); and providing (S5) the thereby trained machine learning model (218) for performing a downstream task.

Anmelder

Firma

Bayerische Motoren Werke Aktiengesellschaft

Land

🇩🇪 Deutschland

Fachgebiete