PhD-student: Vibrating structures you can talk to: speech recognition with mechanical neural networks

In the tale Ali Baba and the Forty thieves, saying ‘Open Sesame!’ opens the entrance to the thieves’ cave. Until now, the idea that an unpowered, inanimate object like a rock could respond to a spoken command has remained the stuff of folk tales. At the hypersmart matter group at AMOLF, we aim to turn this literary vision into a technological reality. Our objective, however, is not to protect a secret cave. We want to build smart devices (cell phones, smart speakers, IoT gadgets) that don’t consume power when they are not being used.

To be able to respond to events (like a person saying ‘Hey Alexa’), smart devices convert physical magnitudes (sound, acceleration, etc.) into electrical currents. These currents are then converted to digital signals and analyzed on a signal processor. Each of these processes constantly consumes energy, even when no event is taking place. Consequently, batteries drain quickly even when we aren’t using the device. At the hypersmart matter group, we aim to solve this problem using mechanical resonances: In the same way that a tuning fork vibrates when excited at its resonance frequency but not at other frequencies, we plan to build mechanical neural networks, brains of tuning forks, that respond when excited with a particular word but not others.

Conceptual structure that responds only to a particular wake-up phrase (left) and network of vibrating-plate resonators (right)

A vibrating structure that recognizes speech is hard to imagine, but there is a physical reason that makes such components possible: Elastic vibrations have very low power dissipation; oscillations take thousands or millions of periods to decay. Because of this long-memory, mechanical systems can recognize complex temporal patterns in a way that would not be possible with passive electronics, where oscillations last only a few tens of periods before they decay. We use the word springtronics to refer to the use of masses and springs to process information with extremely low power consumption. In contrast with conventional electronics, that builds on a century of experience, springtronics is a new area of research. Therefore, a PhD student working in this problem has an opportunity to discover the basic laws and methods of the field. You can read about our prior results in the arXiv pre-print .

Building a structure that responds to particular words but not others is a formidable engineering challenge; a neural network like this must have thousands to millions of vibrating ‘neurones’ to operate correctly. Designing a device of this complexity requires an interdisciplinary research effort combining speech recognition, nonlinear vibrations and microfabrication. We are now offering two Ph.D. positions focused on the theoretical side of these challenges.

Ph.D. 1 Speech recognition with masses and springs
The goal of this Ph.D. is to invent mass-spring models that perform advanced information processing. We will start by implementing basic functions such as logic gates or polynomials, move towards speech-feature extraction (e.g., Mel-Frequency Cepstral Coefficients) to finally build end-to-end mass-spring based speech recognition systems. More generally, this Ph.D. aims to discover the general rules of information processing with masses and springs. We will pursue this goal by combining advanced optimization methods such as Markov Chain Monte Carlo, backpropagation in time and recurrent neural networks. For this position, we seek a student with a background or interest in speech processing, optimization, neural networks or nonlinear dynamics. Our codes are usually written in Python and C++, and we also explore computation using GPUs, although experience with these specific languages and technologies is not required.

Ph.D. 2 Inverse design of large-scale elastic structures
The goal of this Ph.D. is to design complex nonlinear elastic structures composed of thousands to millions of vibrating parts. The transient dynamics of these structures are too complicated to simulate using the Finite Element Method. Therefore, we need to use reduced-order models that accurately simulate the dynamics while using a much smaller number of degrees of freedom. During this Ph.D., we will develop novel techniques to construct such accurate and performant models. Then, we will develop inversion algorithms to find structural geometries that behave accordingly to a desired model. For this position, we particularly value experience in nonlinear finite elements (including writing your own codes) and dynamic substructuring, both linear and nonlinear (Craig-Bampton, Rubin, Modal derivatives). We use Python in combination with the open-source finite element package FEniCS.
The advertised Ph.D. positions have a theoretical focus. However, we plan on fabricating the resulting designs at the AMOLF cleanroom. Therefore, theoretical students will be able to see their work translated into real devices and will be able to participate in experiments if interested. The Ph.D. students in this project will develop scientific leadership skills through co-supervision of master students, and there may be opportunities to do research stays abroad.

https://arxiv.org/abs/2111.08503

Dr. Marc Serra Garcia
Group leader Hypersmart matter
E-mail: [email protected]
Phone: +31 (0)20-754 7201

You can respond to this vacancy online via the button below.
Please annex your:
–  Resume;
–  Motivation on why you want to join the group (max. 1 page).

It is important to us to know why you want to join our team. This means that we will only consider your application if it entails your motivation letter.

Applications will be evaluated on a rolling basis and as soon as an excellent match is made, the position will be filled.

Online screening may be part of the selection.

Commercial activities in response to this ad are not appreciated.