Overview
Deep Learning and Knowledge Integration for Temporal Relations Extraction
Réf ABG-122426
Sujet de Thèse
08/04/2024
Contrat doctoral
LIFO – Université d’Orléans
Lieu de travail
Orléans – Centre Val de Loire – France
Intitulé du sujet
Deep Learning and Knowledge Integration for Temporal Relations Extraction
Champs scientifiques
• Informatique
Mots clés
Deep Learning; Knowledge Integration; Hybrid AI
Description du sujet
We propose to work within the framework of temporal information extraction, which associates a synthetic representation of the events described in natural language text. A classical representation of such data is a graph of temporal relations between the events described and/or between temporal expressions [1].
Recent advances in deep learning in terms of language skills lead us to question human mastery over natural language processing tasks. These models have increasingly complex architectures and are increasingly demanding in terms of computing power… and training data. However, they remain insufficient since general knowledge about temporal relations is not exploited to better guide and explain the results. In the context of this thesis topic, we propose to explore the integration of knowledge into a deep learning system, based on a language model, to solve temporal reasoning tasks.
A preliminary system [3] proposed to construct a temporal graph from medical texts by leveraging BERT, using rules in probabilistic logic during the model learning phase, as well as during the global inference phase. This hybrid work opened research avenues on the considerable contribution that temporal knowledge could represent through rule-based work. In order to make the systems more efficient, another study [4] proposed to successfully utilize syntactic analysis of inputs. In line with [2], we propose to leverage temporal knowledge representation to enhance system performance and explainability.
We are interested in integrating knowledge into these models to best solve temporal reasoning tasks, and this via constraint expression to:
• Leverage the best of both worlds, constraints, and language models acquired by deep learning
• Propose partly explainable hybrid models
• Base our systems on controlled computing power combined with a reproducible methodology of knowledge injection
Concretely, given a deep learning system based on a language model trained to translate text into a temporal graph representing the events narrated in the input text, injecting knowledge via constraint expression will modify the system’s outputs. We aim to incrementally inject knowledge to guide our system while controlling:
• The size of our model
• The size of our training data
• The complexity of our constraints
Références
• [1] T. Knez and S. Žitnik. Event-centric temporal knowledge graph construction: A survey. Mathematics, 11(23), 2023.
• [2] B. Zhang and L. Li. Piper: A logic-driven deep contrastive optimization pipeline for event temporal reasoning. Neural Networks, 164:186–202, 2023.
• [3] Y. Zhou, Y. Yan, R. Han, J. H. Caufield, K.-W. Chang, Y. Sun, P. Ping, and W. Wang. Clinical temporal relation extraction with probabilistic soft logic regularization and global inference. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 14647–14655, 2021.
• [4] L. Zhuang, H. Fei, and P. Hu. Syntax-based dynamic latent graph for event relation extraction. Information Processing Management, 60(5):103469, 2023
Prise de fonction :
01/10/2024
Nature du financement
Contrat doctoral
Précisions sur le financement
Présentation établissement et labo d’accueil
LIFO – Université d’Orléans
Le Laboratoire d’Informatique Fondamentale d’Orléans (LIFO, UR 4022) est un laboratoire de l’Université d’Orléans et de l’INSA Centre-Val de Loire.
Les recherches menées au LIFO concernent la science informatique et les STIC. Elles vont de l’algorithmique au traitement des langues naturelles, de l’apprentissage au parallélisme massif, de la vérification et la certification à la sécurité des systèmes, du Big Data aux systèmes embarqués. Le laboratoire est structuré en cinq équipes :
CA : https://www.univ-orleans.fr/lifo/equipes/CA/
GAMoC : https://www.univ-orleans.fr/lifo/equipe.php?id=3
LMV : https://www.univ-orleans.fr/lifo/equipes/lmv/
Pamda : https://www.univ-orleans.fr/lifo/equipes/PaMDA/?INTRODUCTION
SDS : http://sds-lifo.insa-cvl.fr/
Afin d’offrir une autre approche du laboratoire et de promouvoir la coopération entre équipes, les thématiques transversales suivantes ont été définies :
Masse de données et calcul haute performance,
Modélisation et algorithmique, et
Sécurité et sûreté.
Site web :
https://www.univ-orleans.fr/lifo/
Intitulé du doctorat
Informatique
Pays d’obtention du doctorat
France
Etablissement délivrant le doctorat
Université d’Orléans
Ecole doctorale
Mathématiques, informatique, physique théorique et ingénierie des systèmes
Profil du candidat
Ideally, the recruited person will hold a Master’s degree in computer science and have theoretical and practical knowledge in deep learning. An interest in language and its automatic processing would be appreciated but is not a prerequisite for recruitment. The candidate must demonstrate:
• Programming skills, such as proficiency in Python, for example
• Experience in Machine Learning, data mining, or applied mathematics
• Synthesis and writing skills allowing for clear and effective reporting of work done
• Ability to communicate in French or English, both orally and in writing
The recruited person will work at LIFO, University of Orléans (Campus de la Source, Orléans). They will be integrated into the Contraintes et Apprentissage team of LIFO(https://www.univ-orleans.fr/lifo/equipes/CA/).
The thesis will start in October 2024, and funding will last for three years.
An audition will take place before the MIPTIS doctoral school jury on June 12 to finalize the selection process.
Date limite de candidature
04/05/2024
Company:
LIFO – Université d’Orléans
Qualifications:
Language requirements:
Specific requirements:
Educational level:
Level of experience (years):
Senior (5+ years of experience)
