PhD student position

Learning-based Representation of Plenoptic Imagery

PhD student position

The Department of Electronics and Informatics (ETRO ) of the Vrije Universiteit Brussel (VUB ) and imec , has an open PhD position for a talented and motivated researcher in a cross-disciplinary research program funded by the Research Foundation – Flanders (FWO).

Motivation

Due to advances in medical imaging and VR/AR/MR devices, interest in advanced content representations allowing for higher resolution, richer portrayals of the real world is significantly growing. Currently deployed modalities aim to offer richer experiences: point clouds provide a representation of the scene space, light fields give a projection of the 3D scene in the observer plane, and holograms provide a projection using wave-based light models.

Unfortunately, existing solutions lead to suboptimal image quality because nonlinearities, such as occlusions and edges, are often not correctly captured. All of them are based on discrete representation of the 3D scene or its projection, meaning that their model is incomplete. Consequently, intermediate viewpoints will require additional calculations, which is often the case in interactive experiences with photo-realistic content.

Project

In this research project, we aim to design a versatile, compact but complete plenoptic representation of a unified image modality that facilitates, among others, an efficient extraction of light field or holographic modalities in the static photographic scenario. Such a representation would require storing a continuous 5D function, accounting for the observation position, and gaze angle.

This project will allow the PhD student to improve upon many skills ranging from advanced signal/image processing, deep learning, efficient representation and coding to the programming of a useable application enabling real-time view extractions from the plenoptic image representation. Research centres on investigating suitable representation models, associated coding algorithms and visual quality evaluations. The PhD student will collaborate with an international team.

Partners

The primary research location will be the ETRO department located in Brussels. The involved research team is also embedded in the nano-electronics research institute imec.

This project will run in collaboration with the Multimedia Signal Processing Group at EPFL Switzerland, headed by Prof. Touradj Ebrahimi, at which we plan a yearly internship. The research team is also intensively involved in the JPEG Pleno standardization, in which VUB and EPFL are leading partners.