PhD Studentship - Cross-Modal Language Learning with Large Language Models through Human-Robot Interaction

DoS Dr. Amir ALY ([email protected] )

2nd Supervisor Dr. Thomas Wennekers ([email protected])

3rd Supervisor Professor Angelo Cangelosi ([email protected] )

Applications are invited for a three-year PhD studentship. The studentship will start on 1st October 2024.

Funding amount: The studentship is supported for 3 years and includes full Home tuition fees plus a stipend of £19,088 per annum 2024/25 rate (TBC).

Project Description

This exciting project seeks to explore the role of perceptual data, such as visual information, in language learning (grounding) through human-robot interaction. It, also, investigates the transformative role of large language models (LLMs), with their rich syntactic and semantic embeddings which surpass traditional language models, in enabling a robot to comprehend and utilize complex language structures and contexts across disciplines. The endeavour involves developing a computational model that integrates LLM embeddings with sensory inputs, such as visual data, facilitating a nuanced understanding of word meanings and the mechanisms by which individual words combine to form the meanings of larger linguistic constructs within the robot's learning process. Some basic questions (among others) that this project will try to explore are:

• How can embeddings generated by LLMs be combined to compose meaningful phrases and sentences, revealing the underlying mechanisms of language compositionality?
• How can perceptual information be leveraged in real-time to dynamically update and refine the syntactic and semantic structures of language?
• How can a robot effectively ground the meanings of words, phrases, and sentences in its perceptual data through interactive learning processes with humans and the environment?

This ambitious project will allow the student to have a multidisciplinary approach and will strongly enrich the student's vision of the challenges facing AI, language, and robotics. This project also benefits from a strong supervisory team and international collaborators. Interested students from the wide area of computer science and artificial intelligence are encouraged to apply. This project has no specific prerequisites.

Eligibility

Applicants should have a first or upper-second-class honours degree in an appropriate subject and preferably a relevant Masters qualification. Applications from both UK and overseas students are welcome.

The studentship is supported for 3 years and includes full Home tuition fees, Bench fee plus a stipend of £19,088 per annum (2024/25 rate TBC).  The studentship will only fully fund those applicants who are eligible for Home fees with relevant qualifications.  Applicants normally required to cover International fees will have to cover the difference between the Home and the International tuition fee rates.  The international component of the fee may be waived for outstanding international applicants.

NB: The studentship is supported for three years of the four-year registration period. The fourth year is a self-funded ‘writing-up’ year.

If you wish to discuss this project further informally, please contact Dr Amir ALY at [email protected] .

To apply for this position please click the ‘Apply ’ button.

Please clearly state the name of the DoS and the studentship title that you are applying for on the top of your personal statement.

Please see here for a list of supporting documents to upload with your application.

For more information on the admissions process generally, please contact [email protected]

The closing date is 12 noon on 15th March 2024. Shortlisted candidates will be invited for interview shortly thereafter. Applicants who have not received a response within six weeks of the closing date should consider their application has been unsuccessful on this occasion.