Verfahren und System zum Visio-Linguistischen Verstehen unter Verwendung Kontextueller Sprachmodell-Reasoner

Details

Veröffentlichungs-Nr.

EP3926531

Aktenzeichen

EP21179781

Anmeldetag

16. Juni 2021

Veröffentlichung

24. April 2024

Erteilung

24. April 2024

Rechtsraum

EP

IPC

G06V10/25G06V10/82G06V20/20G06V10/764

Abstract

This disclosure relates generally to visio-linguistic understanding. Conventional methods use contextual visio-linguistic reasoner for visio-linguistic understanding which requires more compute power and large amount of pre-training data. Embodiments of the present disclosure provide a method for visio-linguistic understanding using contextual language model reasoner. The method converts the visual information of an input image into a format that the contextual language model reasoner understands and accepts for a downstream task. The method utilizes the image captions and confidence score associated with the image captions along with a knowledge graph to obtain a combined input in a format compatible with the contextual language model reasoner. Contextual embeddings corresponding to the downstream task is obtained using the combined input. The disclosed method is used to solve several downstream tasks such as scene understanding, visual question answering, visual common-sense reasoning and so on.

Anmelder

Firma

Tata Consultancy Services Limited

Land

🇮🇳 Indien

Fachgebiete