Post-doc : « Towards sustainable AI: Measuring the Computational Cost of Deep Learn-ing » – 18-month contract

10 Apr 2024
Job Information

Organisation/Company

Ecole Nationale Supérieure des Mines de Saint Etienne

Research Field

Other

Researcher Profile

First Stage Researcher (R1)

Country

France

Application Deadline

8 May 2024 - 00:00 (Africa/Abidjan)

Type of Contract

Temporary

Job Status

Full-time

Hours Per Week

40

Offer Starting Date

1 Jul 2024

Is the job funded through the EU Research Framework Programme?

Not funded by an EU programme

Is the Job related to staff position within a Research Infrastructure?

No

Offer Description

Working environment:

The Mines-Télécom Institute (IMT) is the 1st public group of Grandes Écoles (schools of engineering and management) in France. Consisting of eight public schools and two affiliate schools, IMT builds a rich ecosystem of academic and industrial partnerships dedicated to higher education, research and economic development.

IMT gives a central role to digital sovereignty and sobriety in its strategy, realized via activities of its Data&AI scientific community targeting sustainable AI, among others. This strategic axis led IMT to join the excellence network ENFIELD, which structures the research of thirty European partners in the context of trustworthy and green AI (European Lighthouse to Manifest Trustworthy and Green AI, voir https://www.enfield-project.eu/ ). Mines Saint-Étienne, one of the eight Grandes Écoles of IMT, is opening a post-doc position to strenghen its presence in ENFIELD.

Mines Saint-Étienne conducts research on sustainable AI from the angle of computational cost of machine learning and lifecycle assessment of AI systems.

Scientific challenges:

Language models and other foundational models based on deep neural networks have become ubiquitous in a few years. However, already after the introduction of the Transformer architecture in the literature was the question of their computational cost raised. Their great generalization abilities mainly comes from the fact they possess a large number of configurable parameters, compared to their predecessors. Yet, one can empirically observe that progress made since 2012 in machine learning has come with an exponential growth in the computational cost to train a model.

The growing adoption of neural networks in information systems raises the question of the sustainability of AI with respect to our environment. In turn, controlling the computational cost of AI calls for solid indicators to make decisions prior to the design phase of any information system. Software eco-design e.g. imposes to justify that any solution with high computational/environmental cost fulfils a specific need and cannot be replaced with a lower-cost solution.

Yet, currently, there is no a priori indicator related to the computational cost of a machine learning model. Numerous dimensions come into play, including the amount of training data, the complexity of the model or hardware performances. Some of these dimensions are available online: Hugging Face, as an example, publishes model cards that indicate how many trainable parameters are model has, what training data has been used and, sometimes, an estimate of energy consumption during training. However, a model card provides no information after the model is trained (i.e. at inference time).

Current practices in AI call for the development of an indicator of computational cost that is suitable for eco-design. This indicator could be evaluated on the various models developed by ENFIELD partners, as well as members of IMT’s Data&AI scientific community.

Main missions:

The main mission of the post-doc will be to design a static analysis approach to estimate the computational cost of a machine learning model implemented with a tensor library such as PyTorch, TensorFlow or NumPy. The approach will have to take several dimensions into account, including  data size, model size and hardware characteristics such as the size of memory and number of parallel cores.

The post-doc will also have the mission to moderate scientific exchanges between ENFIELD partners, structured in a matrix made of pillars (Green AI, Adaptive AI, Human-Centric AI et Trustworthy AI) and application domains (Energy, Healthcare, Manufacturing et Space). Each pillar and each application domain is associated with a working group that has to coordinate with other working groups.

Main activities :

The post-doc includes the following activities:

• Review of available models on Hugging Face, of models developed at IMT and of models developed by ENFIELD partners,
• Design of an indicator of computational cost for these models (e.g. a number of FLOPs),
• Development of a static analysis tool to calculate estimates from code (e.g. from PyTorch functions)
• Participation in ENFIELD working group
• Participation in activities organized by IMT’s Data&AI scientific community

Profile:

Skills, knowledge :

• PhD in AI, applied mathematics or software engineering

Working conditions:

• 18-month public contract (standardized salary based on experience levels)
• Beginning around July-September 2024
• Full-time
• Based in Saint-Étienne
• 49 days of vacation
• Partial remote work allowed

Requirements

Research Field

Other

Education Level

PhD or equivalent

Skills/Qualifications

• PhD in AI, applied mathematics or software engineering

Languages

FRENCH

Level

Good

Additional Information
Work Location(s)

Number of offers available

1

Company/Institute

MINES SAINT-ETIENNE

Country

France

Geofield

Where to apply

Website

https://institutminestelecom.recruitee.com/o/post-doctorante-ou-post-doctorant-…

Contact

City

Saint-Étienne

Website

http://www.emse.fr/

Street

158 cours Fauriel , CS 62362

Postal Code

42023

STATUS: EXPIRED