|
" Feedback Strategies on Verbal and Nonverbal Cues to Improve Communication Skills "
Ali, Mohammad Rafayet
Hoque, Mohammed Ehsan
| Document Type
|
:
|
Latin Dissertation
|
| Language of Document
|
:
|
English
|
| Record Number
|
:
|
1107451
|
| Doc. No
|
:
|
TLpq2446699387
|
| Main Entry
|
:
|
Ali, Mohammad Rafayet
|
|
|
:
|
Hoque, Mohammed Ehsan
|
| Title & Author
|
:
|
Feedback Strategies on Verbal and Nonverbal Cues to Improve Communication Skills\ Ali, Mohammad RafayetHoque, Mohammed Ehsan
|
| College
|
:
|
University of Rochester
|
| Date
|
:
|
2020
|
| student score
|
:
|
2020
|
| Degree
|
:
|
Ph.D.
|
| Page No
|
:
|
154
|
| Abstract
|
:
|
In this thesis, we present findings on designing and validating real-time and post feedback on nonverbal skills in face to face communication skills with a humanoid agent. The technical challenges included a real-time machine learning framework that can automatically process the audio-video data via a webcam, allowing the users to converse in natural language and receive live and post feedback on smile intensity, volume modulation, pauses, synchronicity, body language, eye-contact, sentiment and turn-taking. Our initial exploration included designing a wizard-of-oz experiment validating the form factors (i.e., flashing icons using the traffic light analogy) for real-time feedback using 46 college students. Using the data, we trained a hidden Markov model to generate feedback. The feedback on verbal cue was generated by performing sentiment and word category analysis. For post feedback, we summarized the nonverbal feedback using the support vector machine. The technical contributions were validated in three unique contexts: 1) helping individuals with autism; 2) helping elderly with their social skills; 3) helping physicians improve their interactions skills with patients. Applications to speed-dating and autism: In a randomized control study with 47 college students, we found that the feedback helped improve eye contact and gesture. In a preliminary study with nine teenagers with autism, we identified several design guidelines which include, briefing the users, making positive acknowledgments, and personalizing dialogue. Applications to aging: In a pilot study with 25 older adults, participants found the feedback useful and were able to reflect on the feedback. In a subsequent longitudinal study with 18 older adults, participants improved their eye contact and smiling. Applications to patient-physician communication: In the context of patient-physician communication, we conducted a study with eight clinicians where they found the feedback intuitive and easy to follow. Additionally, we identified two communication behaviors of physicians that help improve patients' prognosis understanding – 1) lecturing style of a conversational structure by maximizing entropy, and 2) the positive language patterns (i.e., sentiment trajectory) using k-means clustering. We used a data set that includes conversations between physicians (N=38) and late-stage cancer patients (N=382). With statistical analysis, we show that physicians who were lecturing their patients and did not vary their positive sentiment had patients with prognosis misunderstanding. During global pandemics (e.g., COVID-19), when social distancing is recommended, most communication is taking place online. This indicates the need for online communication training programs that can overcome social and global boundaries.
|
| Subject
|
:
|
Computer science
|
|
|
:
|
COVID-19
|
| |