Efficient and Robust Alignment of Large Language Models

Details

Recent developments in generative Large Language Models (LLMs) such as GPT-4 by OpenAI and LLaMA by Meta have demonstrated that these models have incredible language understanding and generation capabilities. An integral part of this success is model alignment that typically consists of supervised fine-tuning (SFT) followed by preference optimization. This requires access to large volumes of human generated data for allowing LLMs performing downstream tasks and responding to various human prompts.

In this PhD, the student will explore developing new state-of-the-art methods for efficient and robust alignment of LLMs. Directions include: (i) data efficient fine-tuning and preference optimization; (ii) robustness to distribution shifts; and (iii) model compression.

If you wish to discuss any details of the project informally, please contact Prof. Nikos Aletras ([email protected]).

Supervisor Bio

Nikos Aletras is a Professor of Natural Language Processing (NLP) in the Computer Science Department, University of Sheffield where he has been a member since 2018. Prior to returning to academia, he gained industrial experience by working as a research scientist at Amazon where he developed industrial scale methods for language understanding. Niko's research interests include data and resource efficient language understanding, model explainability and NLP applications in law and social sciences (see https://nikosaletras.com/publications/ for more details).

About the Department/Research Group

Department of Computer Science: 99% of our research is rated in the highest two categories in the REF 2021, meaning it is classed as world-leading or internationally excellent. We are rated as 8th nationally for the quality of our research environment, showing that the Department of Computer Science is a vibrant and progressive place to undertake research.

Natural Language Processing Group: Established 1993, the University of Sheffield's NLP Group is one of the UK's largest natural language processing research centres. According to CSRankings, it is also one of the most productive groups in Europe and worldwide, frequently publishing in top-tier NLP conferences. Currently, the group has more than 50 members including academics, research associates and PhD students, offering many opportunities for collaborations.

Other information:

• Funding is available to support conference and summer school attendance.  
• Excellent computing resources.
• Sheffield is an extraordinary place to live and study. It’s one of the major cities in the UK, yet 60% of it is green space. It’s a city that’s safe, affordable, creative and welcoming. It has been named the 2023 UK’s foodie capital.

Candidate Requirements

• Undergraduate degree (at least a UK 2:1 honours degree, or its international equivalent) in Computer Science or related fields. AI related Master’s (preferably with Distinction) will be considered as a plus.
• Excellent programming skills (e.g. Python)
• If English is not your first language, you must have an IELTS score of 6.5 overall, with no less than 6.0 in each component.

How to Apply

To apply for a PhD studentship, applications must be made directly to the University of Sheffield using the Postgraduate Online Application Form. Make sure you name Nikos Aletras as your proposed supervisor.

Information on what documents are required and a link to the application form can be found here - https://www.sheffield.ac.uk/postgraduate/phd/apply/applying 

The form has comprehensive instructions for you to follow, and pop-up help is available. 

Your research proposal should:

-be no longer than 2 A4 pages + unlimited references

-outline your reasons for applying for this studentship

-explain how you would approach the research, including details of your skills and experience in the topic area