about me

Hi, my name is João. I am a Senior Deep Learning Engineer at Bosch (Braga, Portugal) and currently finishing a Ph.D. degree in Electrical and Computers Engineering at Faculdade de Engenharia da Universidade do Porto (FEUP). In 2017, I have completed my integrated masters degree in Bioengineering (field of Biomedical Engineering) with a thesis on the use of the electrocardiogram (ECG) for biometric identification of vehicle drivers.

From 2017 to 2022, my doctoral research with the Visual Computing and Machine Intelligence (VCMI) research group at INESC TEC (Porto, Portugal) has focused on developing novel and robust pattern recognition methodologies for biometric recognition and wellbeing monitoring (emotion, drowsiness, activity, etc.). These focused on the use of seamless/unobtrusive data sources, especially in the scenario of driver/passenger monitoring on intelligent vehicles, and included major contributions to the topic of ECG biometrics.

Since 2022, I work on the THEIA - Automated Perception Driving project at Bosch, pushing the boundaries of self-driving cars. I contribute to the development of intelligent perception algorithms to interpret sensor data (especially LIDAR), to give vehicles information on their environment and surroundings and support autonomous driving decisions.

Welcome to my website, where I present a little bit more about myself and my work. Feel free to contact me with any question or proposal, I am always looking for new opportunities to develop my skills and expand my knowledge.

experience
see more
research
Biometrics

Biometric recognition is the art of recognising individuals based on intrinsic characteristics, like fingerprints, face video, or iris images. These characteristics are always carried by the person, and are harder to forget, misplace, steal, or copy than ID cards or passwords.

My research is focused on developing more effective, robust, and secure biometric systems. I have focused mainly on the electrocardiogram as a biometric trait, given its hidden nature that makes it more difficult to steal or counterfeit. However, I have also worked on other traits, especially face image and video.

While biometric recognition is the main focus of my research, I have also delved into the topics of template security, presentation attack detection and, more recently, the use of interpretability tools to explain decisions given by deep biometric models.

Pattern Recognition

When looking at a picture, humans immediately recognise there are people in it, maybe some trees, a house, and the sky in the background. For a computer, however, a picture is no more than a matrix of organised numeric values. The same applies to signals, video, or any other data.

In Pattern Recognition, a subarea of Artificial Intelligence, researchers focus on making computers able to detect, discriminate, and identify patterns in data. For that, we use signal processing, image analysis, and/or machine learning methods for computers to learn to understand data.

My research focuses mainly in pattern recognition for biometrics. However, I have also worked on image analysis for automatic medical diagnosis, and on more fundamental machine learning topics including weakly-supervised learning and ordinal classification.

Electrocardiogram

The electrocardiogram (ECG) is the signal that measures the flow of electrical currents across the heart. It is not only a key tool for clinical diagnosis of heart conditions, but a carrier of valuable personal information on identity, health, and emotional states.

Through my research, I create algorithms that use the ECG for automatic recognition of identity, emotions, and fatigue in human-computer interfaces. Specifically, I focus on challenging off-the-person signals acquired unobtrusively, through electrodes seamlessly integrated in common objects such as wearables, smartphones, or steering wheels.

One of my main goals is to make the ECG biometric algorithms as effective and robust as face or fingerprint biometrics, while reataining the higher security it offers as a hidden biometric trait.

Wellbeing Monitoring

The variability of a person's traits is often a nuisance for biometric recognition. However, it often reveals valuable information that could be used to understand other aspects and characteristics of the subjects, including their health and emotional states.

Face image and video has long been used to understand facial expressions and link them to emotional states, despite claims of unreliability. Physiological traits like the ECG include more intrinsic information on health and emotions, that is much harder to fake than facial expressions.

My research in wellbeing monitoring is mainly focused on combining ECG and face video information for more robust emotion, stress, and fatigue recognition. This work is especially focused on human-machine interface contexts, e.g., for real-time monitoring of vehicle drivers.

publications
selected publications
Secure Triplet Loss: Achieving Cancelability and Non-Linkability in End-to-End Deep Biometrics IEEE Transactions on Biometrics, Behavior, and Identity Science J. R. Pinto, M. V. Correia, and J. S. Cardoso 2021
Explaining ECG Biometrics: Is It All In The QRS? 19th International Conference of the Biometrics Special Interest Group (BIOSIG 2020) J. R. Pinto and J. S. Cardoso 2020
An End-to-End Convolutional Neural Network for ECG-Based Biometric Authentication 10th IEEE International Conference in Biometrics: Theory, Applications and Systems (BTAS '19) J. R. Pinto and J. S. Cardoso 2019
Evolution, Current Challenges, and Future Possibilities in ECG Biometrics IEEE Access J. R. Pinto, J. S. Cardoso, and A. Lourenço 2018
see full list
activity
supervised msc students current students (4)
Co-supervisor of Vítor Barbosa (2021/22)
"Activity and Emotion Classification in Shared Vehicles", Master in Informatics and Computing Engineering, Universidade do Porto
Co-supervisor of Mariana S. Xavier (2021/22)
"Inside Out: Fusing ECG and Face Information to Recognise Emotions", Master in Bioengineering, Universidade do Porto
Co-supervisor of Guilherme T. A. R. Barbosa (2021/22)
"Going 2D: Exploring Learnable Bidimensional Representations for ECG Biometrics", Master in Bioengineering, Universidade do Porto
External Supervisor of Pedro Duarte Lopes (2020/21)
"Deep Neural Networks for Face-based Emotion Recognition", Master in Bioengineering, Universidade do Porto
past students (8)
External Supervisor of Telma Esteves (2020/21)
"Sleepy Drivers: Drowsiness Monitoring Using ECG and Face Video", Master in Biomedical Engineering, Universidade Nova de Lisboa
Co-supervisor of Sofia C. Beco (2020/21)
"Make My Heartbeat: Generation and Interlead Conversion of ECG Signals", Master in Bioengineering, Universidade do Porto
Co-supervisor of Inês A. T. A. Magalhães (2020/21)
"Feel My Heart: Emotion Recognition Using the Electrocardiogram", Master in Bioengineering, Universidade do Porto
Co-supervisor of Arthur J. Matta (2019/20)
"Deep Learning For Face Recognition: A Real-Time Face Biometric Recognition System", Master in Informatics and Computing Engineering, Universidade do Porto
Co-supervisor of Carolina M. B. R. Afonso (2019/20)
"Changing Perspectives: Interlead Conversion in Electrocardiographic Signals", Master in Network and Information Systems Engineering, Universidade do Porto
Co-supervisor of João M. G. Ferreira (2019/20)
"Head Pose Estimation for Facial Biometric Recognition Systems", Master in Informatics and Computing Engineering, Universidade do Porto
Co-supervisor of Leonardo G. Capozzi (2019/20)
"Face Recognition For Forensic Applications: Methods for Matching Facial Sketches to Mugshot Pictures", Master in Informatics and Computing Engineering, Universidade do Porto
Co-supervisor of Gabriel C. Lopes (2018/19)
"Don't You Forget About Me: Enhancing Long Term Performance in Electrocardiogram Biometrics", Master in Bioengineering, Universidade do Porto
supervised internships past (23)
Co-supervisor of João Carvalho and Paula Ogata (2022)
Face recognition, INESC TEC
Co-supervisor of Mariana Calado (2021)
Drowsiness detection using ECG signals, INESC TEC
Co-supervisor of Brenda Nogueira and Ana Maria Sousa (July 2021)
Driver drowsiness detection using face video, CTM Summer Internships 2021, INESC TEC
Co-supervisor of João Romão and Bernardo Gabriel (July 2021)
Motion estimation for activity recognition, CTM Summer Internships 2021, INESC TEC
Co-supervisor of Renata Lei and Rafael Cristino (July 2021)
Hotel recognition for fighting human trafficking, CTM Summer Internships 2021, INESC TEC
Co-supervisor of Guilherme Barbosa (2021)
An End-to-End Learnt 2D representation for ECG-Based Biometric Authentication, INESC TEC
Co-supervisor of Duarte Lopes (2021)
Interpretability for Face Presentation Attack Detection, INESC TEC
Co-supervisor of Susana Lima (July 2020)
Detecting Drowsiness Through ECG and Steering Wheel Data, CTM Summer Internships 2020, INESC TEC
Co-supervisor of João Fonseca (2020)
NEB Outclass Internship: Face Photograph Analytics
Co-supervisor of Catarina Lopes (2019-2020)
Face photograph analytics for automatic assessment of ICAO guideline compliance, INESC TEC
Co-supervisor of Pedro Silva, Paulo Costa, Manuel Curral, Gil Teixeira, Pedro Neto, Martim Silva, João Mendes, João Moura, and Carolina Afonso (July 2019)
Various topics related to face biometrics, forensics, and analytics, CTM Summer Internships 2019, INESC TEC
research projects current (2)
EasyRide Project (Collaborator since February 2020)
In-vehicle activity recognition and wellbeing measurement.
AUTOMOTIVE Project (Collaborator from July 2019 to November 2021)
Driver drowsiness monitoring using unconstrained video and physiological signals.
past (1)
CHIC Project (2018/19)
Quality assessment of print text documents for optical character and image recognition.
my cv short full
Deep Learning Engineer
PhD Student in Biometrics