Verfahren und System zur Domänenspezifischen Textverarbeitung mit Schlüsselwort-Wissensdestillation

Details

Veröffentlichungs-Nr.

EP4811207

Anmeldetag

24. September 2025

Veröffentlichung

23. September 2026

Rechtsraum

EP

Abstract

While knowledge distillation has shown promise in general NLP contexts, its application to domain-specific text remains largely underexplored. Existing models often struggle to generalize complex domain-specific vocabulary, specialized terminology, and unique sentence structures, resulting in degraded performance when applied to domain-specific tasks. Thus, embodiments of present disclosure provide a method and system for domain-specific text processing with keyword knowledge distillation. The disclosed method overcomes the challenges in conventional methods by utilizing a dynamic, weighted ensemble approach for keyword extraction, which adaptively selects domain-specific keywords extracted using multiple keyword extraction techniques. The method then applies knowledge distillation, where a student model is trained to predict the masked domain-specific keywords using context from surrounding tokens. By computing three loss functions, including distillation loss, keyword masking loss, and cosine embedding loss, the student model is optimized for performance while retaining critical domain knowledge.

Anmelder

Firma

Tata Consultancy Services Limited

Land

🇮🇳 Indien