Interaction analytics for automatic assessment of communication quality in primary care

Abstract

Effective doctor-patient communication is a crucial element of health care, influencing patients’ personal and medical outcomes following the interview. The set of skills used in interpersonal interaction is complex, involving verbal and non-verbal behaviour. Precise attributes of good non-verbal behaviour are difficult to characterise, but models and studies offer insight on relevant factors. In this PhD, I studied how the attributes of non-verbal behaviour can be automatically extracted and assessed, focusing on turn-taking patterns of and the prosody of patient-clinician dialogues. I described clinician-patient communication and the tools and methods used to train and assess communication during the consultation. I then proceeded to a review of the literature on the existing efforts to automate assessment, depicting an emerging domain focused on the semantic content of the exchange and a lack of investigation on interaction dynamics, notably on the structure of turns and prosody. To undertake the study of these aspects, I initially planned the collection of data. I underlined the need for a system that follows the requirements of sensitive data collection regarding data quality and security. I went on to design a secure system which records participants’ speech as well as the body posture of the clinician. I provided an open-source implementation and I supported its use by the scientific community. I investigated the automatic extraction and analysis of some non-verbal components of the clinician-patient communication on an existing corpus of GP consultations. I outlined different patterns in the clinician-patient interaction and I further developed explanations of known consulting behaviours, such as the general imbalance of the doctor-patient interaction and differences in the control of the conversation. I compared behaviours present in face to face, telephone, and video consultations, finding overall similarities alongside noticeable differences in patterns of overlapping speech and switching behaviour. I further studied non-verbal signals by analysing speech prosodic features, investigating differences in participants’ behaviour and relations between the assessment of the clinician-patient communication and prosodic features. While limited in their interpretative power on the explored dataset, these signals nonetheless provide additional metrics to identify and characterise variations in the non-verbal behaviour of the participants. Analysing clinician-patient communication is difficult even for human experts. Automating that process in this work has been particularly challenging. I demonstrated the capacity of automated processing of non-verbal behaviours to analyse clinician-patient communication. I outlined the ability to explore new aspects, interaction dynamics, and objectively describe how patients and clinicians interact. I further explained known aspects such as clinician dominance in more detail. I also provided a methodology to characterise participants’ turns taking behaviour and speech prosody for the objective appraisal of the quality of non-verbal communication. This methodology is aimed at further use in research and education