Knowledge Graph Embeddings : Walk-Based Multimodal and Explainable Knowledge Graph Embeddings

Call for expression of interest description

The Marie S. Curie Postdoctoral Fellowship (MSCA-PF) programme is a highly prestigious renowned EU-funded scheme. It offers talented scientists a unique chance to set up 2-year research and training projects with the support of a supervising team. Besides providing an attractive grant, it represents a major opportunity to boost the career of promising researchers. 

Research laboratories in Brittany arethus looking for excellent postdoctoral researchers with an international profile to write a persuasive proposal to apply for a Marie S. Curie Postdoctoral Fellowship grant in 2024 (deadline of the EU call set on 11 September 2024). The topic and research team presented below have been identified in this regard.

Main Research Field

• Information Science and Engineering (ENG)

Research sub-field(s)

Artificial Intelligence, Machine Learning, Knowledge Management

Keywords

Knowledge graph embeddings; Multimodal Learning ; Inductive Learning

Research project description

Context. Knowledge graphs (KGs) are large collections of curated structured information that are crucial to several information-centered tasks such as Web search, inference, and question answering.  A popular representation for KGs are knowledge graph embeddings (KGEs). The latent nature of KGEs makes them suitable for interacting with modern and accurate machine learning algorithms.

Despite their popularity, KGEs are not free of limitations: they are usually black boxes, and they cannot be trivially extended to handle other data modalities such as text, videos, or images. These two observations limit their applicability to arbitrary knowledge graphs and use cases. Given the emergence of LLM-powered chatbots and multimodal assistants, there is a huge potential to leverage the structured, well-defined, and interpretable semantics of KGs to boost the accuracy of LLM-powered systems when confronted to information needs. This is of particular interest for cases where the LLM-based agent has not seen the answer to a question in its training set – which may translate into the so-called “hallucinations”.

We therefore envision to extend existing KGE architectures to multimodal KGs, i.e., datasets including long texts, images, time series, etc. In a first stage we envision to exploit the power of LLMs to make knowledge graphs more interoperable with arbitrary textual content. Several integration architectures are possible [1] to this end. One solution is to use LLMs to guide the training of the KGE (called LLM-assisted KGE training). A second alternative are modality bridges that are trained to map or “project” KGEs to the realm of LLMs so that these two models can “talk” to each other. When ported to other data modalities bridges are appealing because of their modularity: each data modality can be connected to the KG via a relatively “simple” and “focused” bridge. In a second stage we will investigate if modality bridges are a viable solution for other data types such as images or time series.

Scientific goals.  Our research question revolves around the viability of KGE bridges as an algorithmic solution for multimodal information systems. Furthermore, an important aspect of our research is to make these multimodal systems “explainable”. Explainability in multimodal systems remains an under-developed research topic and we therefore envision to come up with methods and formalisms for explanations spanning across data and models of different nature, e.g., text and images (e.g., as in VQA systems [2]). The post-doctoral candidate will work on developing such methods and validating them, in a first stage, on public (a) academic and scientific KGs that include long articles with illustrations, and (b) encyclopedic multimodal KGs.

[1] Pan, Shirui, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. "Unifying Large Language Models and Knowledge Graphs: A Roadmap, 2023." arXiv preprint arXiv:2306.08302.

[2] M. Ziaeefad and F. Lecue. Towards Knowledge-Augmented Visual Question Answering. COLING 2020

Supervisor(s)

The Postdoctoral Fellow will be supervised by Dr. Luis Galárraga and Christine Largouët.

Luis Galárraga is a permanent research scientist at the Inria Centre at Rennes University since October 2017, being part of the LACODAM team (see below). He obtained his PhD in 2016 from Institut Polytechnique de Paris (Télécom ParisTech at that time). His research interest includes semantic web (knowledge representation, RDF query processing, provenance, completeness in semantic data), data mining (pattern mining, rule mining), and explainable machine learning in AI. He has supervised two postdocs and four PhD students.

Links :

https://luisgalarraga.de/

https://scholar.google.fr/citations?hl=fr&user=RAp3e3sAAAAJ

Selected Publications :

• Luis Galárraga. Effects of Locality and Rule Language on Explanations for Knowledge Graph Embeddings. Full paper at the Symposium on Intelligent Data Analysis (IDA ), Louvain-la-Neuve (2023).
• Julien Delaunay , Luis Galárraga, Christine Largouët . When Should We Use Linear Explanations?. Full Paper at the Conference on Information and Knowledge Management (CIKM ), Atlanta (2022).
• Jonathan Lajus, Luis Galárraga, Fabian Suchanek . Fast and Exact Rule Mining with AMIE 3. Extended Semantic Web Conference. Full Paper at the Extended Semantic Web Conference (ESWC ), Heraklion (2020).

Christine Largouët is an Associate Professor at Institut Agro Rennes-Angers and based at the LACODAM team at IRISA laboratory. Her research interest focuses on Artificial Intelligence tools for human decision support (learning Timed behavioural models, explainable AI, complex systems modelling and Analysis with Timed Automata and Model-Checking, data-driven decision support).

Links :

http://people.rennes.inria.fr/Christine.Largouet/

https://scholar.google.fr/citations?hl=fr&user=E0XX8HAAAAAJ

Selected Publications :

• Lénaïg Cornanguer, Christine Largouët, Laurence Rozé, Alexandre Termier. TAG: Learning Timed Automata from Logs. Full Paper at AAAI Conference on Artificial Intelligence (2022).
• Raphaël Gauthier, Christine Largouët, Laurence Rozé, Jean-Yves Dourmad. Online forecasting of daily feed intake in lactating sows supported by offline time-series clustering, for precision livestock farming. Journal on Computers and Electronics in Agriculture, (2021) 
• Alban Siffer, Pierre-Alain Fouque, Alexandre Termier, Christine Largouët
  . Are your data data gathered? The Folding Test of Unimodality Conference. Full paper at SIGKDD International Conference on Knowledge Discovery and Data Mining (2018).

Department/

Research                    

Inria is the French national research institute for digital science and technology. World-class research, technological innovation and entrepreneurial risk are its DNA. In 215 project teams, most of which are shared with major research universities, more than 3,900 researchers and engineers explore new paths, often in an interdisciplinary manner and in collaboration with industrial partners to meet ambitious challenges. As a technological institute, Inria supports the diversity of innovation pathways: from open source software publishing to the creation of technological startups (Deeptech).

The Inria Centre at Rennes University was established in 1980. The centre has 30 research teams, 24 being shared with the IRISA mixed research unit. Its  activities occupy over 600 people, scientists and research innovation support staff, including 50 different nationalities. The Inria Centre at Rennes University covers a wide range of expertise in Computer Science, with scientific priorities such as secure digital society, human-robot-virtual world interactions, digital biology and health and digital ecology

The fellow will integrate the LACODAM team. The objective of the LACODAM team is to facilitate the process of making sense out of (large) amounts of data. This can serve the purpose of deriving knowledge and insights for better decision-making. Our approaches are mostly dedicated to provide novel tools to data scientists, that can either perform tasks not addressed by any other tools, or that improve the performance in some area for existing tasks (for instance reducing execution time, improving accuracy or better handling imbalanced data). Our main research areas are pattern mining, interpretable machine learning, and semantic web.)

Location

Inria Centre of Rennes University

Campus de Beaulieu

Rennes, France

Deadline for sending an expression of interest

5th May 2024

Selection of the most promising application(s)

May – June 2024

Writing the MSCA-PF proposal with the support of the above-mentioned supervisor(s)

June – September 2024

  MSCA-PF 2024 call deadline

11th September 2024

Publication of the MSCA-PF evaluation results

February 2025

Start of the MSCA-PF project (if funded)

 May 2025 (at the earliest)