Verfahren zur Frameworkunabhängigen Benutzereingabeverarbeitung in Mensch-Maschine-Schnittstellen

Details

Veröffentlichungs-Nr.

EP4803988

Anmeldetag

3. März 2026

Veröffentlichung

9. September 2026

Rechtsraum

EP

Abstract

A method for framework-independent user input processing in humanmachine interfaces (HMI). The method enables a unified way to interact with different UI frameworks without requiring modifications to the underlying applications, frameworks or operating systems. The system receives user input, such as voice, gestures, or text, processes it into a machine-understandable command, and dynamically maps it to actionable UI elements. Identified UI elements are retrieved from an interaction database and highlighted for user confirmation before executing simulated input events injected via a virtual input device of the operating system at pixel coordinates of a selected actionable region. The system can further learn from user interactions, refining UI mappings over time. By operating at the graphical buffer level, rather than requiring direct access to UI source code, the method supports a wide range of applications, including binary-only software. This approach enhances input adaptability, accuracy, and accessibility across multiple UI frameworks.

Anmelder

Firma

Seafarix Oy

Land

🇫🇮 Finnland