Hello! I am a third-year PhD student at Telecom Paris, working on machine learning for audio and music processing under the supervision of prof. Gaël Richard and prof. Geoffroy Peeters. I am part of the ADASP group and the HI-Audio project. I am currently working as a research scientist intern at Deezer research. Previously, I was an intern at Sony CSL (Music Team).

My research focuses on developing novel approaches at the intersection of signal processing and deep learning for Music Information Retrieval (MIR), Music Source Separation and audio synthesis. I am particularly interested in unsupervised and self-supervised learning for training smart and efficient models.

I hold a Bachelor's in Electrical and Computer Engineering from Universidade Federal de Minas Gerais and an engineering degree from Telecom Paris. In 2022, I completed my Master's degree in Artificial Intelligence from the MVA program at Ecole Normale Superieure Paris-Saclay.

Contact: bernardo [dot] torres AT telecom-paris [dot] fr

Github LinkedIn Bluesky Twitter Google Scholar

You can find my CV here.

Publications

---

PESTO: Real-Time Pitch Estimation with Self-supervised Transposition-equivariant Objective

Alain Riou, Bernardo Torres, Ben Hayes, Stefan Lattner, Gaëtan Hadjeres, Gaël Richard, Geoffroy Peeters
Submitted to Transactions of the International Society of Music Information Retrieval (TISMIR).
ABS Preprint (soon) code

In this paper, we introduce PESTO, a self-supervised learning approach for single-pitch estimation using a Siamese architecture. Our model processes individual frames of a Variable-$Q$ Transform (VQT) and predicts pitch distributions. The neural network is designed to be equivariant to translations, notably by introducing a Toeplitz fully-connected layer. In addition, we construct pitch-shifted pairs by translating and cropping the VQT frames and train our model with a novel class-based transposition-equivariant objective, eliminating the need for annotated data. Thanks to this architecture and training objective, our model achieves remarkable performances while being very lightweight (130k parameters). Evaluations on music and speech datasets (MIR-1K, MDB-stem-synth, and PTDB) demonstrate that PESTO not only outperforms self-supervised baselines but also competes with supervised methods, exhibiting superior cross-dataset generalization. Finally, we enhance PESTO's practical utility by developing a streamable VQT implementation using cached convolutions. Combined with our model's low latency (less than 10 milliseconds) and minimal parameter count, this makes PESTO particularly suitable for real-time applications.

Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transport

Bernardo Torres, Geoffroy Peeters and Gaël Richard
In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2024).
ABS PDF (arXiv) code poster

In neural audio signal processing, pitch conditioning has been used to enhance the performance of synthesizers. However, jointly training pitch estimators and synthesizers is a challenge when using standard audio-to-audio reconstruction loss, leading to reliance on external pitch trackers. To address this issue, we propose using a spectral loss function inspired by optimal transportation theory that minimizes the displacement of spectral energy. We validate this approach through an unsupervised autoencoding task that fits a harmonic template to harmonic signals. We jointly estimate the fundamental frequency and amplitudes of harmonics using a lightweight encoder and reconstruct the signals using a differentiable harmonic synthesizer. The proposed approach offers a promising direction for improving unsupervised parameter estimation in neural audio applications.

A Fully Differentiable Model for Unsupervised Singing Voice Separation

Gaël Richard, Pierre Chouteau, and Bernardo Torres
In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2024).
ABS PDF (hal)

A novel model was recently proposed by Schulze-Forster et al. in [1] for unsupervised music source separation. This model allows to tackle some of the major shortcomings of existing source separation frameworks. Specifically, it eliminates the need for isolated sources during training, performs efficiently with limited data, and can handle homogeneous sources (such as singing voice). But, this model relies on an external multipitch estimator and incorporates an Ad hoc voice assignment procedure. In this paper, we propose to extend this framework and to build a fully differentiable model by integrating a multipitch estimator and a novel differentiable assignment module within the core model. We show the merits of our approach through a set of experiments, and we highlight in particular its potential for processing diverse and unseen data.

Singer Identity Representation Learning using Self-Supervised Techniques

Bernardo Torres, Stefan Lattner and Gaël Richard
In International Society for Music Information Retrieval Conference (ISMIR 2023).
ABS PDF (hal) code blog poster

Significant strides have been made in creating voice identity representations using speech data. However, the same level of progress has not been achieved for singing voices. To bridge this gap, we suggest a framework for training singer identity encoders to extract representations suitable for various singing-related tasks, such as singing voice similarity and synthesis. We explore different selfsupervised learning techniques on a large collection of isolated vocal tracks and apply data augmentations during training to ensure that the representations are invariant to pitch and content variations. We evaluate the quality of the resulting representations on singer similarity and identification tasks across multiple datasets, with a particular emphasis on out-of-domain generalization. Our proposed framework produces high-quality embeddings that outperform both speaker verification and wav2vec 2.0 pre-trained baselines on singing voice while operating at 44.1 kHz. We release our code and trained models to facilitate further research on singing voice and related areas.