Multimodale Sensorfusion unter Verwendung Natürlicher Sprache für Hmi-Agent-Systeme

Details

Veröffentlichungs-Nr.

EP4668166

Aktenzeichen

EP24183748

Anmeldetag

21. Juni 2024

Veröffentlichung

24. Dezember 2025

Rechtsraum

EP

IPC

G06N3/092

Abstract

The invention proposes a computer-implemented method for operating an HMI system (16), the method comprising:a) monitoring a user with a multi-modal sensor arrangement (20) to obtain multi-modal sensor data (30) that are indicative of a user state;b) feeding the multi-modal sensor data (30) to a user state estimation module (32), wherein the user state estimation module (32) processes the multi-modal sensor data (30) and outputs a user state embedding (34) that represents a natural language description of the user state or the natural language description of the user state;c) feeding the result of step b) to an HMI agent (40) and a reward model (36), wherein the reward model (36) determines a reward value (38) and feeds it to the HMI agent (40), and the HMI agent (40) determines a control signal (42) that is configured to cause a modification to an HMI device (18).

Anmelder

Firma

AUMOVIO Germany GmbH

Land

🇩🇪 Deutschland

Fachgebiete