PhD position Space-time Neural Memory Models and (Differential) Geometry

Updated: 5 days ago
Deadline: ;

The Informatics Institute  of the Faculty of Science  is looking for a PhD candidate on Space-time Neural Memory Models and (Differential) Geometry.

For the past decades the AI, Machine Learning, and Machine Vision community has primarily focused on analyzing and understanding static data, that is considering only space. However, spatiotemporal data comprise the vast majority of data. Notwithstanding the usual commercial (like YouTube) video recordings and benchmarks, we have that:

  • spatiotemporal data are necessary for the next generation of robust and scalable unsupervised and selfsupervised learning, or World-based Reinforcement Learning, using time as a ‘label’ in the learning objective;
  • they are present in scientific recordings: satellite spatiotemporal sequences of glaciers, astronomical spatiotemporal measurements of exoplanets, spatiotemporal biomedical recordings of lymphocytic Tcells attacking cancer cells, to name a few examples;
  • spatiotemporal is the natural way of recording data and if anything, static data is typically momentary snapshots of spatiotemporal recordings.

Modern Deep Machine Learning and Machine Vision have troubles when it comes to complex spatiotemporal recordings, when these are not stylized or span more than a few seconds. The reason is that today’s algorithms are:

  • Time extrinsic: (most) algorithms treat time as an extra external dimension or variable, appending it in the input or output channels of the neural network, effectively trying to ‘reverse engineer’ the effect of time on data;
  • Time euclidean: (most) algorithms are unstructured machines of finite nature and ‘flat’ geometry, while trying to model complex spatiotemporal recordings that evidently correspond to innumerable spatiotemporal dynamics;
  • Time deterministic: in their majority, most algorithms either explicitly or implicitly- consider the data points as ground truth and repeatable observations, which would occur exactly the same and without uncertainty if starting from the same initial condition.

Especially on memory models, designing, building and validating memory models of more than a few time steps has been a pivoting point for artificial intelligence, from Hopfield network associative memories to recurrent neural networks, LSTMs, and more recently Neural Turing Machines. Memory models have mostly been considered with symbolic data and tasks, where representations are already known in advance rather than learned, e.g., memorizing sequences of digit symbols or texts, performing arithmetic operations, or basic reasoning tasks with synthetic data - see CLEVR. Memory models, however, seem to have trouble with complex, non-symbolic data like spatiotemporal sequences, where the learning task must not only account for the noise in memory but also for the (significant) noise in representations. A likely reason why memory models suffer with such complex data is the chaotic nature of standard recurrent models (Pascanu and Bengio, 2012) parameterized on the flat and unbounded Euclidean space. Another problem is that there are no reliable mechanisms for aggregating complex spatiotemporal memories. Novel, if not paradigm-shifting, memory models are critically needed to move past the current single-shot classifications on single static image data.

In this PhD position, we will investigate the connection between Space-time Neural Memory Models and Differential Geometry, in an attempt to design novel Time Geometric spatiotemporal algorithms. We want to research neural memory models whose internal state is not flat Euclidean space, but conforms to certain advanced geometries, be it in the form of graphs, differential (Lie) manifolds, and beyond. Geometric memories have three advantages. First, they are likely harder to model but easier to optimize, avoiding chaotic regimes and subpar optima. Second, they allows for memory interpolation: one does not need to memorize explicitly all possible past observations. Instead, one needs only to make sure that the geodesics on the manifold correspond to associated and smoothly transitioning potential memories. That is even more relevant with graphs, where memories can be stored as specific routes on the graph. Third, with geometric memories it is natural to integrate over geodesic lines or graph paths to aggregate all relevant memories and make the required inferences, rather than relying on the conventional average or max pooling aggregation that destroy any notion of order and structure.

The aim is to fundamentally reformulate and reinterpret neural memory models so that to rely on geometry to encode their internal memory state. Using tools from differential geometry, Lie groups and so on, we will attempt to gain insights into existing neural memory structural and functional properties. This, in turn, will help us design novel neural memory models that are optimal for complex spatiotemporal data. By learning space-time neural memory models we can better interpret the role of time in machine learning, and we could hopefully come one step closer in unlocking the true power of complex spatiotemporal reasoning in machine learning. Some overarching questions include:

  • can we reinterpret existing neural memory models from the perspective of flat Euclidean and unbounded geometry? What are the implications?
  • can we use advanced geometric structures, in the form of graphs or manifolds, to design geometric models of neural memories?
  • what is the optimal way we can aggregate information on certain geometries? Are geodesics or graph paths good alternatives?
  • is there a particular relation between certain geometries and certain types of memories or certain types of spatiotemporal sequences? For instance, when building a memory model of object shapes, would specific differential manifolds make more sense, while for complex and long spatiotemporal sequences other types of manifolds or graphs are more suitable?

You will be supervised by Dr. E. Gavves, Associate Professor at the University of Amsterdam. This project is financed by the winning H2020 ERC Starting Grant ‘EVA: Expectational Visual Artificial Intelligence’ and NWO VIDI Grant ‘TIMING: Learning Time in Videos’.

What are you going to do

You will carry out research and development in the area of Deep Machine Learning and Vision. The research is embedded in the VISlab group at the University of Amsterdam.

Your tasks will be to:

  • develop new deep machine learning and/or computer vision methods on Space-Time Neural Memory Models and (Differential) Geometry;
  • collaborate with other researchers within the lab;
  • regularly present internally on your progress ;
  • regularly present intermediate research at international conferences and workshops, and publish them in proceedings and journals;
  • assist in relevant teaching activities.

Complete and defend a PhD thesis within the official appointment duration of four years.

What do we require
  • a MSc degree in Artificial Intelligence, (Applied) Mathematics/Physics, Computer Science, Engineering or related field
  • a strong background/knowledge in machine learning and statistics; computer vision is also a strong plus
  • a strong background/knowledge in stochastic differential equations, dynamical systems, chaos theory
  • excellent programming skills preferably in Python
  • solid mathematics foundations, especially statistics, calculus and linear algebra;
  • you are highly motivated, independent, and creative
  • strong communication, presentation and writing skills and excellent command of English.
  • Prior publications in relevant machine learning, vision, differential geometry, dynamical systems conferences or journals is advantageous.

Our offer

A temporary contract for 38 hours per week for the duration of 4 years (the initial contract will be for a period of 18 months and after satisfactory evaluation it will be extended for a total duration of 4 years). This should lead to a dissertation (PhD thesis). We will draft an educational plan that includes attendance of courses and (international) meetings. We also expect you to assist in teaching undergraduates and master students.

Based on a full-time appointment (38 hours per week) the gross monthly salary will range from €2.395 in the first year to €3.061 in the last year exclusive 8% holiday allowance and 8.3% end-of-year bonus. A favourable tax agreement, the ‘30% ruling’, may apply to non-Dutch applicants. The Collective Labour Agreement of Dutch Universities is applicable.

Are you curious about our extensive package of secondary employment benefits like our excellent opportunities for study and development? Take a look here .

About the Faculty of Science and the Informatics Institute

The Faculty of Science has a student body of around 7,000, as well as 1,600 members of staff working in education, research or support services. Researchers and students at the Faculty of Science are fascinated by every aspect of how the world works, be it elementary particles, the birth of the universe or the functioning of the brain.

The mission of the Informatics Institute Informatics Institute - University of Amsterdam ( is to perform curiosity-driven and use-inspired fundamental research in Computer Science. The main research themes are Artificial Intelligence, Computational Science and Systems and Network Engineering. Our research involves complex information systems at large, with a focus on collaborative, data driven, computational and intelligent systems, all with a strong interactive component.
The position is with Dr. Efstratios Gavves, Associate Professor in the Video & Image Sense lab (VISlab) led by Prof. C. Snoek. VISlab is a world-leading lab on Computer Vision and Machine Learning, and has over 40 PhD students, postdoctoral researchers and faculty members working on a broad variety of core computer vision and core machine learning subjects: from action and object recognition or efficient spatiotemporal deep learning, to stochastic probabilistic models, temporal causality and graph neural networks. In the lab we encourage strongly collaborations. Other labs on Machine Learning and Computer Vision at the Informatics Institute include AMLab by Prof. M. Welling and CVlab  by Prof. T. Gevers.


Do you have questions about this vacancy? Or do you want to know more about our organisation? Please contact:

  • Efstratios Gavves , Associate Professor
    T. +31 (0)20 525 8701

Job application

The UvA is an equal-opportunity employer. We prioritize diversity and are committed to creating an inclusive environment for everyone. We value a spirit of enquiry and perseverance, provide the space to keep asking questions, and promote a culture of curiosity and creativity.

The Informatics Institute strives for a better gender balance in its staff. We therefore strongly encourage women to apply for this position.

Do you recognize yourself in the job profile? Then we look forward to receiving your application by 13 June 2021. You can apply online by using the link below.

Applications in .pdf should include:

  • a motivation letter that motivates your choice for this position (max 1 page);
  • a CV, including a list of publications if applicable (max 2 pages);
  • a research statement with your ideas/inspiration about the project. We do not expect a fully-fledged proposal, a sketch of creative approaches will be appreciated (max 2 pages);
  • a copy of your MSc thesis. If you are still studying and do not have it completed, a short summary up to 4 pages is also possible;
  • a complete record of your MSc and BSc courses, including grades and explanation of the grading system;
  • a list of projects and publications you have worked on, with brief descriptions of your contributions (max 2 pages);
  • the names and contact addresses of at least two academic references (please do not include any recommendation letters).

We will invite potential candidates for interviews soon after the expiration of the vacancy.

No agencies please


View or Apply

Similar Positions