Dynamics of Human-Voice-AI Interaction

I am currently working on projects related to how individuals engage with voice-activated artificially intelligent (voice-AI) devices, such as Amazon’s Alexa or Apple’s Siri.

Over the past year, I have worked with Dr. Georgia Zellou (PI: UC Davis Phonetics Lab, Dept. of Linguistics) to conduct a series of psycholinguistic experiments to test whether individuals engage with these devices in similar ways as they do with other humans. I have also collaborated with Dr. Zhou Yu (PI: UC Davis Language and Multimodal Lab, Dept. of Computer Science) on projects related to the Amazon Alexa Prize winning socialbot, Gunrock.

I am thrilled to serve as a PI for a two-year NSF-funded postdoctoral research fellowship with Drs. Georgia Zellou, Zhou Yu, and Katharine Graf Estes to explore human-voice AI interaction. (Click here to see the official NSF posting)

NSF Postdoc Fellowship

Image result for national science foundation

Current Project

We explore ways in which adults and children adapt their speech when talking to voice-activated digital assistants (e.g., Amazon’s Alexa), compared to adult human interlocutors.

This line of work provides a way to test differing theoretical predictions as to the extent that speech-register adjustments are driven by functional motives (e.g., intelligibility) and social factors (e.g., gender).

For instance, this research explores whether the same functional motivations that apply when correcting comprehension errors to human interlocutors apply in device-directed speech (DS), such as in manipulating the phonological nature of errors, to carefully control the level of intelligibility-related pressures in communication.

At the same time, this project explores how social factors may impact speech adaptation strategies, such as by interlocutor type, speaker age, or device gender. This project additionally involves important methodological innovations in programming and running experiments directly through a digital device platform.

Overall, this project aims to fill a gap in our knowledge in the acoustic-phonetic adjustments humans make when talking to voice-AI devices, and can ultimately reveal the underlying mechanisms in speech production by different speakers (e.g., based on age, gender, device experience), contributing to basic science research.

Human-socialbot interaction: Gunrock

I am also currently collaborating with Dr. Zhou Yu (PI: UC Davis Language and Multimodal Interaction Lab, Dept. of Computer Science) on projects related to the Amazon Alexa Prize chatbot, Gunrock.

At the celebration for “Gunrock” winning the 2018 Amazon Alexa Prize!

We have a demo paper forthcoming at the EMNLP 2019 Conference in Hong Kong. For a system demonstration, see:

We are thrilled to present an experiment on Gunrock we conducted through the Amazon Alexa Competition at SIGDIAL 2019:

Cohn, M., Chen, C., Yu, Z. (2019). A Large-Scale User Study of an Alexa Prize Chatbot: Effect of TTS Dynamism on Perceived Quality of Social Dialog. (In press). 2019 Special Interest Group on Discourse and Dialogue, SIGDIAL.

Voice-AI Research News

We recently had a proceedings paper accepted to the 2019 International Phonetic Congress (ICPhS)! Cohn, M., Ferenc Segedin, B., Zellou, G. Imitating Siri: Socially-mediated vocal alignment to device and human voices. (In press). 2019 International Congress of Phonetic Sciences (ICPhS).

I was a finalist for the 5 Minute Linguist competition at the Linguistic Society of America (LSA) annual meeting for our project “Phonologically motivated phonetic repair strategies in Siri- and human-directed speech”. Click here for information about the competition & see below for the video!