Durch Künstliche Intelligenz Gesteuerte Benutzerschnittstellenstrukturerdung zur Automatisierten Erzeugung von Aktionsfähigen Aufgaben

Details

Veröffentlichungs-Nr.

EP4592840

Aktenzeichen

EP25152517

Anmeldetag

17. Januar 2025

Veröffentlichung

30. Juli 2025

Rechtsraum

EP

IPC

G06F9/451G06N3/045G06N3/08G06F8/30G06N20/00

Abstract

A data processing system implements obtaining an image of a user interface of an application and a textual representation of a structure of the user interface from a client-side proxy on a client device; analyzing the image of the user interface using a vision language model to obtain one or more task recommendations; analyzing the textual representation of the structure of the user interface and the one or more task recommendations using a completion language model to generate one or more workflows, each workflow being associated with a task recommendation of the one or more task recommendations and comprising one or more actions and executable code to perform the one or more actions in the application; providing the one or more workflows to the client-side proxy; causing the client-side proxy to present a representation of each task recommendation of the one or more task recommendations on the user interface;

Anmelder

Firma

Microsoft Technology Licensing, LLC

Land

🇺🇸 USA

Fachgebiete