Understanding human conversations is a perennial challenge in designing human-centered artificial intelligence systems. While methodologies to extract what has been said from speech (a.k.a. speech recognition) have been studied intensely, there are many open challenges in understanding how it has been said. In this talk, I’d like to introduce an example of efforts that try to solve the open challenges. It studies social signals - they are produced during social interactions and play an important role in understanding the interactions and their participants. Particularly, in this talk, we focus on detecting conflicts during face-to-face debates. The study analyzes a database from political domain where genuine conflicts exist.