System und Verfahren zur Effizienten Aufmerksamkeitsschichtimplementierung für Spärliche Vektorquantizierte Tiefe Neuronale Netzwerke

Details

Veröffentlichungs-Nr.

EP4734000

Anmeldetag

21. Oktober 2025

Veröffentlichung

29. April 2026

Rechtsraum

EP

IPC

G06N3/063, G06N3/08

Abstract

A method for generating vector-quantized deep neural networks that includes creating binary masks for the query vector and key vector, wherein creating the binary masks includes replacing each non-zero element with a value of 1, calculating a logical AND operator of the binary masks of the query and key vector, identifying the position of non-zero bits utilizing results from the logical AND operator of the binary masks of the query and key vector to obtain one or more non-zero indices, and for only the non-zero indices of the logical AND operator, outputting a dot-product value, creating binary masks of the value vector, wherein the binary masks are associated with non-zero values, and for only the non-zero indices of the value vector, multiplying the dot-product value with the value vector to generate a final value of the attention layer, and outputting the final value to a next layer.

Anmelder

Firma

Robert Bosch GmbH

Land

🇩🇪 Deutschland

Fachgebiete