PhD Candidate in Point Cloud Prediction

NTNU is a broad-based university with a technical-scientific profile and a focus in professional education. The university is located in three cities with headquarters in Trondheim.

At NTNU, 9,000 employees and 42,000 students work to create knowledge for a better world.

You will find more information about working at NTNU and the application process here.

Volumetric Video (VV) is becoming a key technology for the creation of highly realistic and immersive environment. Given the expected growth of immersive technology in the coming years (e.g., Metaverse project launched by Facebook), it becomes necessary to develop innovative AI-based solutions to improve immersive media production / consumption workflow.

In this PhD project, we propose to focus on point cloud (PC) data prediction which is an important processing step that complements and benefits other immersive media production and consumption steps. More specifically, PC prediction is closely connected to volumetric video compression and thus will have a strong impact on the quality of user experience.

The PhD project objectives are to develop various Neural Network-based point cloud prediction models and validate them in the context of VV coding and rendering. Similar to conventional video compression algorithms based on motion estimation/compensation steps, it is expected that efficient point cloud prediction will improve the coding performance of volumetric data. Moreover, the prediction of point clouds can provide a better understanding of the future user pose and ensure low-latency VV streaming. In this respect, for an input sequence with n frames of point clouds, the objective consists in predicting one (or multiple) future frame(s) given some reference frames.

Unlike classic (single and multi-view) video prediction which has been widely investigated in the literature, there are very few works developed in the context of point cloud prediction. The latter is obviously a more challenging problem due to the unordered and unstructured nature of such data. With the ultimate goal of achieving efficient and accurate prediction and motivated by the great success of deep learning in image and video processing, this work will focus on graph and neural networks-based approaches. In this respect, our research approach will consist firstly in developing new graph neural network-based prediction models. Then, the latter will be validated on public PC datasets and evaluated using quantitative and qualitative techniques (i.e., objective and subjective quality assessment techniques).

While a few deep learning-based PC prediction approaches have been developed, mainly based on PointNet and recurrent neural networks, our method will rely on spatial-temporal transformers. The latter have already shown promising results in the context of conventional video motion estimation and prediction. Thus, the extension of such transformers to the context of dynamic PC will be investigated in this thesis. Moreover, to better exploit the local information/topology as well as the inter-frame correlation, we will resort to graph-based representations. Therefore, we will mainly design new graph-based neural networks for effective motion estimation and video prediction. Finally, regarding the choice of the loss function employed to train the developed models, we will investigate new perceptual metrics to better reflect the quality of user experience.

• Develop new graph neural network-based prediction models for PC.
• Validate them on public PC datasets and evaluate them using quantitative and qualitative techniques.
• Disseminate results in top ranked international conferences, workshops, and journals.
• Participate in the general activities of the Department.
• Complete a training component as part of the PhD of a minimum of 30 ECTS.
• Participate in international secondments at partner educational institutions abroad (e.g., University Sorbonne Paris Nord, France, University College London, UK).

• You must have a professionally relevant background in machine/deep learning, and in image/video/point cloud processing and analysis, or similar.
• Your education must correspond to a five-year Norwegian degree program, where 120 credits are obtained at master's level.
• You must have a strong academic background from your previous studies and an average grade from the master's degree program, or equivalent education, which is equal to B or better compared with NTNU's grading scale . If you do not have letter grades from previous studies, you must have an equally good academic basis. If you have a weaker grade background, you may be assessed if you can document that you are particularly suitable for this PhD position.
• Master's students in their final year of studies can apply, but the master's degree must be obtained and documented before the employment.
• Excellent oral and written communication skills in English and an ability to communicate effectively.
• You must meet the requirements for admission to the faculty´s doctoral programme .
• Good programming knowledge and experience.

The appointment is to be made in accordance with  Regulations concerning the degrees of Philosophiae Doctor (PhD) and Philosodophiae Doctor (PhD) in artistic research national guidelines for appointment as PhD, post doctor and research assistant  

• Experience with digital media compression.
• Experience with image/video/point cloud processing and analysis.
• OpenCV, Python, TensorFlow, PyTorch, Latex.
• Oral and verbal skills in Norwegian.

• Motivated for research.
• Social person.
• Team player.
• Organised.

As a PhD candidate (code 1017) you are normally paid from gross NOK 501 200 per annum before tax, depending on qualifications and seniority. From the salary, 2% is deducted as a contribution to the Norwegian Public Service Pension Fund.

The period of employment is 3 or 4 years. A 4-year position would require 25 % teaching duties.

Appointment to a PhD position requires that you are admitted to the PhD programme in computer science , within three months of employment, and that you participate in an organized PhD programme during the employment period.

The engagement is to be made in accordance with the regulations in force concerning State Employees and Civil Servants , and the acts relating to Control of the Export of Strategic Goods, Services and Technology. Candidates who by assessment of the application and attachment are seen to conflict with the criteria in the latter law will be prohibited from recruitment to NTNU.

After the appointment you must assume that there may be changes in the area of work.

It is a prerequisite you can be present at and accessible to the institution daily.

The application and supporting documentation to be used as the basis for the assessment must be in English.

Publications and other scientific work must follow the application. Please note that your application will be considered based solely on information submitted by the application deadline. You must therefore ensure that your application clearly demonstrates how your skills and experience fulfil the criteria specified above.

The application must include:

• CV and certificates.
• Transcripts and diplomas for bachelor's and master's degrees. If you have not completed the master's degree, you must submit a confirmation that the master's thesis has been submitted.
• A copy of the master's thesis. If you recently have submitted your master's thesis, you can attach a draft of the thesis. Documentation of a completed master's degree must be presented before taking up the position.
• A max 1-page motivation letter describing why the position is interesting for you and why your background and expertise are valuable for the project.
• Name and contact information of three referees.
• Any publications or other relevant research work (up to 3 items).

If all, or parts, of your education has been taken abroad, we also ask you to attach documentation of the scope and quality of your entire education, both bachelor's and master's education, in addition to other higher education. Description of the documentation required can be found here . If you already have a statement from NOKUT, please attach this as well.

We will take joint work into account. If it is difficult to identify your efforts in the joint work, you must enclose a short description of your participation.

In the evaluation of which candidate is best qualified, emphasis will be placed on education, experience and personal and interpersonal qualities. Motivation, ambitions, and potential will also count in the assessment of the candidates. 

NTNU is committed to following evaluation criteria for research quality according to The San Francisco Declaration on Research Assessment - DORA.

Working at NTNU

NTNU believes that inclusion and diversity is our strength. We want to recruit people with different competencies, educational backgrounds, life experiences and perspectives to contribute to solving our social responsibilities within education and research. We will facilitate for our employees’ needs.

NTNU is working actively to increase the number of women employed in scientific positions and has a number of resources to promote equality.  

The city of Gjøvik has a population of 30 000 and is a town known for its rich music and cultural life. The beautiful nature surrounding the city is ideal for an active outdoor life! The Norwegian welfare state, including healthcare, schools, kindergartens and overall equality, is probably the best of its kind in the world.

As an employee at NTNU, you must at all times adhere to the changes that the development in the subject entails and the organizational changes that are adopted.

A public list of applicants with name, age, job title and municipality of residence is prepared after the application deadline. If you want to reserve yourself from entry on the public applicant list, this must be justified. Assessment will be made in accordance with current legislation . You will be notified if the reservation is not accepted.

If you have any questions about the position, please contact Prof. Faouzi Alaya Cheikh, telephone +47 951 879 956, email [email protected] . If you have any questions about the recruitment process, please contact the HR-team at the Department of Computer Science by e-mail: [email protected] .

If you think this looks interesting and in line with your qualifications, please submit your application electronically via jobbnorge.no with your CV, diplomas and certificates attached. Applications submitted elsewhere will not be considered. Upon request, you must be able to obtain certified copies of your documentation.  

Application deadline: 15.10.22

NTNU - knowledge for a better world

The Norwegian University of Science and Technology (NTNU) creates knowledge for a better world and solutions that can change everyday life.

Department of Computer Science 

We are the leading academic IT environment in Norway, and offer a wide range of theoretical and applied IT programmes of study at all levels. Our subject areas include hardware, algorithms, visual computing, AI, databases, software engineering, information systems, learning technology, HCI, CSCW, IT operations and applied data processing. The Department has groups in both Trondheim and Gjøvik. The Department of Computer Science  is one of seven departments in the Faculty of Information Technology and Electrical Engineering  .