CogSci 2020 Papers

We are thrilled that three of our papers have been accepted to the 2020 Cognitive Science Society Meeting!

  • Cohn, M., Jonell, P., Kim, T., Beskow, J., Zellou, G. Embodiment and gender interact in alignment to TTS voices.

While at the KTH Royal Institute of Technology (Stockholm, Sweden) in September 2019, I met up with Dr. Jonas Beskow (pictured in the center), co-founder of Furhat Robotics, and Ph.D. student Patrik Jonell (pictured on the right). Together with Georgia Zellou and Taylor Kim, we’re conducting a study to test the role of embodiment and gender in human’s voice-AI interaction with three platforms: Amazon Echo, Nao, and Furhat. 
  • Zellou, G., & Cohn, M. Top-down effects of apparent humanness on vocal alignment toward human and device interlocutors.
  • Zellou, G., Cohn, M., Block, A. Top-down effect of speaker age guise on perceptual compensation for coarticulatory /u/-fronting.

LSA 2020

We’ll present two projects at the annual Linguistic Society of America (LSA) meeting in January:

  • California listeners’ patterns of partial compensation for coarticulatory /u/-fronting is influenced by the apparent age of the speaker (Aleese Block, Michelle Cohn, Georgia Zellou)
  • Conversational role influences speech alignment toward digital assistant and human voices (Georgia Zellou, Michelle Cohn, Tyler Kline, Bruno Ferenc Segedin)

EMNLP 2019 Paper

Congrats to the Gunrock team, led by Prof. Zhou Yu, for our demo paper acceptance at the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP) in Hong Kong!

Gunrock: A Social Bot for Complex and Engaging Long Conversations
Dian Yu, Michelle Cohn, Yi Mang Yang, Chun Yen Chen, Weiming Wen, Jiaping Zhang, Mingyang Zhou, Kevin Jesse, Austin Chau, Antara Bhowmick, Shreenath Iyer, Giritheja Sreenivasulu, Sam Davidson, Ashwin Bhandare and Zhou Yu [pdf]

You can see our system demonstration (2 minute video):

Picture
The Gunrock team at the 2018 reception after winning the Alexa Prize!

CAMP3 Talks

Great job to Tyler Kline and Aleese Block who presented two of our projects at the California Meeting on Psycholinguistics (CAMP3) this weekend at UC Santa Cruz!

  • Speech Alignment of Females toward Voice-AI and Human Voices: Conversational Role Influences Phonetic Imitation in a Map Task (Tyler Kline, Bruno Ferenc Segedin, Michelle Cohn & Georgia Zellou) 
  • California listeners’ patterns of partial compensation for coarticulatory /u/-fronting is influenced by the apparent age of the speaker (Aleese Block, Michelle Cohn & Georgia Zellou)
Picture
Tyler Kline presenting

Picture
Aleese Block presenting via handout during the power outage

Interspeech 2019

Along with Georgia Zellou & Bruno Ferenc Segedin (UC Davis Phonetics Lab), I traveled to Graz, Austria to present some of our research at the 2019 Interspeech Conference!

Picture
Georgia Zellou presenting our research exploring individual variation in speech toward Siri vs. human voices
Picture
Presenting our project looking at alignment toward emotionally expressive productions by Amazon Alexa

Bruno Ferenc Segedin presenting our research exploring phonetic adaptation to human vs. Amazon Alexa voices
Presenting research on acoustic cue weighting for musicians / nonmusicians

See below for links for the papers: 

Collaboration with KTH & Furhat Robotics

While at the KTH Royal Institute of Technology (Stockholm, Sweden) this September, Michelle Cohn met up with Dr. Jonas Beskow, co-founder of Furhat Robotics, and Ph.D. student Patrik Jonell. Together with Georgia Zellou, they are conducting a study to test the role of embodiment and gender in human’s voice-AI interaction with three platforms: Amazon Echo, Nao, and Furhat. 

Picture
Michelle Cohn, Jonas Beskow, & Patrik Jonell at the KTH Studio

Sigdial 2019

Along with Dr. Zhou Yu and Arbit Chen (UCD Computer Science), we are thrilled that we have a paper accepted to the Special Interest Group on Discourse (SIGDIAL) meeting in Stockholm, Sweden.

Our paper explores how different text-to-speech (TTS) modifications to the 2018 Alexa Prize Winner chatbot, Gunrock, impact user ratings.

Cohn, M., Chen, C., Yu, Z. (2019). A Large-Scale User Study of an Alexa Prize Chatbot: Effect of TTS Dynamism on Perceived Quality of Social Dialog. (In press). 2019 Special Interest Group on Discourse and Dialogue, SIGDIAL. 

NSF Postdoctoral Fellowship (SPRF) Awardee

I am thrilled to serve as a PI for a two-year NSF-funded postdoctoral research fellowship with Drs. Georgia Zellou, Zhou Yu, and Katharine Graf Estes to explore human-voice AI interaction. (Click here to see the official NSF posting)

 

The project

We explore ways in which adults and children adapt their speech when talking to voice-activated digital assistants (e.g., Amazon’s Alexa), compared to adult human interlocutors.

This line of work provides a way to test differing theoretical predictions as to the extent that speech-register adjustments are driven by functional motives (e.g., intelligibility) and social factors (e.g., gender).

For instance, this research explores whether the same functional motivations that apply when correcting comprehension errors to human interlocutors apply in device-directed speech (DS), such as in manipulating the phonological nature of errors, to carefully control the level of intelligibility-related pressures in communication.

At the same time, this project explores how social factors may impact speech adaptation strategies, such as by interlocutor type, speaker age, or device gender. This project additionally involves important methodological innovations in programming and running experiments directly through a digital device platform.

Overall, this project aims to fill a gap in our knowledge in the acoustic-phonetic adjustments humans make when talking to voice-AI devices, and can ultimately reveal the underlying mechanisms in speech production by different speakers (e.g., based on age, gender, device experience), contributing to basic science research.

Interspeech 2019

We are excited that several papers have been accepted for the Interspeech 2019 meeting in Graz, Austria!

Papers on human-voice AI interaction

Cohn, M., & Zellou, G.(2019). Expressiveness influences human vocal alignment toward voice-AI. (In press). 2019 Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. 

Snyder, C. Cohn, M., & Zellou, G. (2019). Individual variation in cognitive processing style predicts differences in phonetic imitation of device and human voices. (In press). 2019 Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. 

Ferenc Segedin, C. Cohn, M., & Zellou, G. (2019). Perceptual adaptation to device and human voices:  learning and generalization of a phonetic shift across real and voice-AI talkers. (In press). 2019 Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. 

Paper on musical training & speech perception

Cohn, M., Zellou, G., Barreda, S. (2019) The role of musical experience in the perceptual weighting of acoustic cues for the obstruent coda voicing contrast in American English. (In press). 2019 Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH.