Abstract
The paralinguistic component of communication attracted a great deal of attention from contemporary linguists in the 1960s. The seminal works written then by Trager, Crystal and others had a powerful influence on the concept of paralanguage that lasted for many years. But, with the focus shifting towards the socio-psychological context of communication in the 1970s, the development of spoken corpora and databases and the significant progress in speech technology in the 1980s and 1990s, the need has arisen for a more comprehensive, coherent and formalised - but also flexible - approach to paralinguistic features. This study advances some preliminary proposals for a revised treatment of paralanguage that would meet some of these requirements and provide a conceptual basis for a new system of annotation for paralinguistic features. A range of views on paralinguistic features, which come mostly from the fields of speech prosody and gesture analysis, are briefly discussed. A number of assumptions and postulates are formulated to allow for a more consistent approach to paralinguistic features. The study suggests that there should be more reliance on continua than on binary categorisations of features, that multi-functionality and multimodality should be fully acknowledged and that clear distinctions should be made among the levels of description, and between the properties of speakers and the speech signal itself.
References
ABE Isamu. 1967. “On Bio-phonetics.” Bulletin of the Tokyo Institute of Technology 79, 53-57 (cited from CRYSTAL 1976).
ABELIN Åsa, ALLWOOD Jens. 2000. “Cross Linguistic Interpretation of Emotional Prosody.” SpeechEmotion-2000, Newcastle, 110-113.
ABERCROMBIE David. 1967. Elements of General Phonetics. Chicago: Aldine.
ALAHVERDZHIEVA Katya, FLICKINGER Dan, LASCARIDES Alex. 2012. Multimodal Grammar Implementation, Proceedingsof NAACL-HLT, Montpellier.
ARBIB Michael A. 2005. “From Monkey-like Action Recognition to Human Language: An Evolutionary Framework for Neurolinguistics.” Behavioral and Brain Sciences 28, 105-124.
BANSE R., SCHERER Klaus R. 1996. “Acoustic Profiles in Vocal Emotion Expression.” Journal of Personality andSocial Psychology 70(3), 614-636.
BÄNZIGER Tanja, SCHERER Klaus R. 2005. “The Role of Intonation in Emotional Expressions.” Speech Communication 46, 252-267.
BASSO Keith, SELBY Henry (eds.). 1976. Meaning in Anthropology. Albuquerque: University of New Mexico Press.
BAVELAS Janet. 1994. “Gestures as Part of Speech: Methodological Implications.” Research on Language & SocialInteraction 27(3).
BENTHALL J., POLHEMUS T. (eds.). 1975. The Body as a Medium of Expression. London: Institute of Contemporary Arts.
BIEMANS Monique. 2000. Gender Variation in Voice Quality. PhD Thesis, Utrecht: LOT.
BOLSHAKOV Igor, GELBUKH Alexander. 2004. Computational Linguistics: Models, Resources, Applications. Mexico: IPN-UNAM-FCE.
BRAUN A., MASTHOFF H.R. (eds.). 2002. Phonetics and its Applications. Festschrift for Jens-Peter Koester. Wiesbaden: Franz Steiner Verlag, 289-297.
BROWN George. 1990. Listening to Spoken English. London: Longman.
BROWN Keith (ed.). 2006. Encyclopedia of Language and Linguistics. Amsterdam-Boston: Elseviere, vol. 5.
BUNT Harry. 2011. “Multifunctionality in Dialogue.” Computer, Speech and Language 25, 225-245.
BUNT Harry, ALEXANDERSSON Jan, CARLETTA Jean, CHOE Jae-Woong, CHENGYU FANG Alex, HASIDA Koiti, LEE Kyong, PETUKHOVA Volha, POPESCU-BELIS Andrei, ROMARY Laurent, SORIA Claudia, TRAUM David. 2010. “Towards an ISO Standard for Dialogue Act Annotation.” In: Proceedings of LREC 2010, the Seventh InternationalConference on Language Resources and Evaluation, Malta, May 16th - 23rd 2010.
BURKHARDT Franz, AUDIBERT N., MALATESTA L., TUERK Oytun, ARSLAN L., AUBERGE Veronique. 2006. “Emotional Prosody - Does Culture Make a Difference?” In: Proceedings of Speech Prosody 2006 Conference, Dresden.
CAMPBELL Nick. 2002. “Towards a Grammar of Spoken Language: Incorporating Paralinguistic Information.” In: Proceedings ICSLP 2002, Denver, Colorado.
CAMPBELL Nick, MOKHTARI Parham. 2003. “Voice Quality: The 4th Prosodic Dimension.” In: Proceedings of the15th International Congress of Phonetic Sciences (ICPhS’03), Barcelona, Spain, 2417-2420.
CAREY John. 1980. “Paralanguage in Computer-mediated Communication.” In: Proceedings of the 18th AnnualMeeting of the Association for Computational Linguistics, Stroudsburg PA.
CASSEL Justine, STONE Mathew, DOUVILLE Brett, PREVOST Scott, ACHORN Brett, STEEDMAN Mark. 1994. “Modeling the Interaction Between Speech and Gesture.” In: Proceedings of Cognitive Science Society Annual Meeting1994, George Institute of Technology.
CHEN Aoju. 2005. Universal and Language-specific Perception of Paralinguistic Intonational Meaning. PhD Thesis, Radboud Universiteit Nijmegen.
CHEN Aoju, GUSSENHOVEN Carlos, RIETVELD Toni. 2004. “Language-Specificity in the Perception of Paralinguistic Intonational Meaning.” Language and Speech 47(4), 311-349.
CHEVRIE-MULLER Claude, SEGUIER N., SPIRA A., DORDAIN M. 1978. “Recognition of Psychiatric Disorders From Voice Quality.” Language and Speech 21(1), 87-111.
CHOMSKY Noam. 1993. “A Minimalist Program for Linguistic Theory.” In: HALE & KEYSER 1993: 1-52.
CLARK Alexander, FOX Chris, LAPPIN Shalom (eds.). 2010. The Handbook of Computational Linguistics and NaturalLanguage Processing. Malden-Oxford: Wiley-Blackwell.
COHEN Philip R., MORGAN Jerry, POLLACK Martha E. (eds.). 1990. Intentions in Communication. Cambridge: MIT Press.
CORNISH Francis. 2005. “Prosody, Discourse Deixis and Anaphora.” In: Proceedings of DPI-2005 Discourse-Prosody Interface, l’Université de Provence (Aix-Marseille I), Aix-en-Provence, from 8th to 9th September 2005.
CRUTTENDEN Alan. 1986. Intonation. Cambridge: CUP.
CRYSTAL David. 1966. “The Linguistic Status of Prosodic and Paralinguistic Features.” In: Proceedings of theUniversity of Newcastle-upon Tyne Philosophical Society 1(8), 93-108.
CRYSTAL David. 1969. Prosodic Systems and Intonation in English. Cambridge: CUP.
CRYSTAL David. 1974. “Paralinguistics.” In: SEBEOK 1974: 265-295.
CRYSTAL David. 1975. “Paralinguistics.” In: BENTHALL & POLHEMUS 1975: 162-174.
CRYSTAL David. 1976. “Paralinguistic Behaviour as Continuity Between Animal and Human Communication.” In: MCCORMACK & WURM 1976: 13-27.
CRYSTAL David. 1987. Cambridge Encyclopedia of Language. Cambridge: CUP.
CRYSTAL David, QUIRK Randolph. 1964. Systems of Prosodic and Paralinguistic Features in English. The Hague, Netherlands: Mouton.
CUMMINS Fred & PORT Robert F. 1996. “Rhythmic Commonalities Between Hand Gestures and Speech.” In: Proceedingsof the Eighteenth Annual Conference of the Cognitive Science Society, Lawrence Erlbaum Associates, 415-419.
CUTLER Anne. 1977. “The Context-dependence of ‘Intonational Meanings’.” In: Papers from the 13th RegionalMeeting of the Chicago Linguistic Society, 104-115.
DAVIES Alan, WIDDOWSON Henry. 1974. “Reading and Writing.” Techniques in Applied Linguistics. London: OUP, 154-201.
DEMENKO G., JASSEM K., SZPAKOWICZ S. (eds.). 2009. Speech and Language Technology 11. Poznań: Polish Phonetic Association.
DIJKSTRA Ton, DE SMEDT Konrad. 1996. Computational Psycholinguistics. London: Taylor and Francis.
DURANTI Alessandro, GOODWIN Charles (eds.). 1992. Rethinking Context: Language as an Interactive Phenomenon. Cambridge: CUP.
EKMAN Paul, FRIESEN Wallace. 1969. “The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding.” Semiotica 1, 49-98.
FOX Anthony. 2000. Prosodic Features and Prosodic Structure: The Phonology of Suprasegmentals. Oxford: OUP.
FRICKE Ellen. 2012. Grammatik multimodal. Wie Wörter und Gesten zusammenwirken. De Gruyter Mouton.
FRICKE Ellen, LAUSBERG Hedda, LIEBAL Katja, MÜLLER Cornelia (in prep.). Towards a Grammar of Gesture: Evolution,Brain, and Linguistic Structures. Series: Gesture Studies. John Benjamins.
GARROD Simon, PICKERING Martin. 2000. Language Processing. London: Psychology Press.
GHIO Alain, WEISZ F., BARACCA G., CANTARELLA G., ROBERT D., WOISARD V., FUSSI F., GIOVANNI A. 2011. “Is the Perception of Voice Quality Language-dependant? A Comparison of French and Italian Listeners and Dysphonic Speakers.” In: Proceedings of InterSpeech 2011, Florence, Italy, 525-528.
GIBBON Dafydd. 2011. “Modelling Gesture as Speech.” Poznań Studies in Contemporary Linguistics 47(3), 447.
GOBL Christer, CHASAIDE Ailbe Ní. 2003. “The Role of Voice Quality in Communicating Emotion, Mood and Attitude.” Speech Communication 40, 189-212.
GUMPERZ John. 1992. “Contextualization and Understanding.” In: DURANTI & GOODWIN 1992: 229-252.
GUMPERZ John. 1995. “Mutual Inferencing in Conversation.” In: MARKOVÁ et al. 1995: 101-123.
GUSSENHOVEN Carlos. 2002. “Intonation and Interpretation: Phonetics and Phonology.” In: Proceedings of SpeechProsody 2002 Conference, Aix-en-Provence.
GUSSENHOVEN Carlos. 2004. Phonology of Tone and Intonation. Cambridge: CUP.
HALE Kenneth Locke, KEYSER SAMUEL JAY (eds.). 1993. The View from Building 20: Essays in Linguistics inHonor of Sylvain Bromberger. Cambridge, MA: MIT Press. ’T HART Johan, COLLIER Rene, COHEN Antonie. 1990. A Perceptual Study of Intonation. Cambridge: CUP.
HAUSER Marc D., CHOMSKY Noam, FITCH William T. 2002. “The Faculty of Language: What Is It, Who Has It, and How Did It Evolve?” Science 298, 1569-1579.
HEUSINGER K. von. 1999. Intonation and Information Structure. Habilitationsschrift, accepted by the Faculty of Philosophy, University of Konstanz.
HILL Archibald E. 1952. “A Note on Primitive Languages.” International Journal of American Linguistics 18, 172-177.
HILL Archibald E. 1958. Introduction to Linguistic Structures. New York: Harcourt, Brace and World.
HINDE Robert Aubrey (ed.). 1972. Non-verbal Communication. London: CUP.
HIRSCHBERG Julia. 2002. “Pragmatics of Intonational Meaning.” In: Proceedings of Speech Prosody 2002 Conference, Aix-en-Provence.
HIRSCHBERG Julia, PIERREHUMBERT Janet. 1986. “Intonational Structuring of Discourse.” In: Proceedings of the24th Meeting of the Association for Computational Linguistics, 136-144.
HIRST Daniel, DI CRISTO Albert, ESPESSER Robert 2000. “Levels of Representation and Levels of Analysis for Intonation.” In: HORNE 2000.
HORNE Merle (ed.). 2000. Prosody: Theory and Experiment. Dordrecht: Kluwer.
HUGGINS A.W.F. 1979. “Some Effects on Intelligibility of Inappropriate Temporal Relations within Speech Units.” In: Proceedings of the 9th ICPhS, vol. II, 275-282.
HYMES Dell. 1974. Foundations of Sociolinguistics: An Ethnographic Approach. Philadelphia: University of Pennsylvania Press.
ISHI Carlos Toshinori, ISHIGURO H., HAGITA N. 2006. “Using Prosodic and Voice Quality Features for Paralinguistic Information Extraction.” In: Proceedings of Speech Prosody Conference 2006, Dresden.
JAKOBSON Roman, WAUGH Linda. 1979. The Sound Shape of Language. Bloomington: Indiana University Press.
JARMOŁOWICZ-NOWIKOW Ewa, KARPIŃSKI Maciej. 2011. “Communicative Intentions behind Pointing Gestures in Task-oriented Dialogues.” In: Proceedings of GESPIN 2011 Conference, Bielefeld.
JARVELLA Robert J., KLEIN Wolfgang (eds.). 1982. Speech, Place, and Action. Chichester: John Wiley and Sons.
JASSEM Wiktor. 2002. “Classification and Organisation of Data in Intonation Research”. In: BRAUN & MASTHOFF 2002: 289-297.
KARPIŃSKI Maciej. 2006. Struktura i intonacja polskiego dialogu zadaniowego. (Structure and Intonation of PolishTask-oriented Dialogues). Poznań: Wydawnictwo Naukowe UAM.
KARPIŃSKI Maciej. 2007. “Selected Quasi-lexical and Non-lexical Units in Polish Task-oriented Dialogues.” Archivesof Acoustics 32(1), 51-65.
KARPIŃSKI Maciej, JARMOŁOWICZ-NOWIKOW Ewa, MALISZ Zofia. 2009. “Aspects of Gestural and Prosodic Structure of Multimodal Utterances in Polish Task-oriented Dialogues.” In: DEMENKO et al. 2009.
KARPIŃSKI Maciej, JARMOŁOWICZ-NOWIKOW Ewa. 2010. “Prosodic and Gestural Features of Phrase-internal Disfluencies in Polish Spontaneous Utterances.” In: Proceedings of Speech Prosody 2010 Conference, Chicago.
KEATING Patricia A., ESPOSITO Christina. 2006. “Linguistic Voice Quality.” UCLA Working Papers in Phonetics 105, 85-91.
KELLER Eric. 2004. “The Analysis of Voice Quality in Speech Processing.” In: Nonlinear Speech Processing:Algorithms and Analysis, 12th-17th September, 2004, Vietri. Berlin: Springer Verlag.
KENDON Adam. 2005. Gesture. Visible Action as Utterance. Cambridge: CUP.
KLESSA Katarzyna, KARPIŃSKI Maciej. 2012. “Annotating Paralinguistic Features in Quasi-spontaneous Speech. Adding the ‘Vision’ Component.” In: Abstracts of Vision and Language Workshop. Shefield.
KREIMAN Jody, VANLANCKER-SIDTIS Diana, GERRATT Bruce. 2004. “Defining and Measuring Voice Quality.” In: Proceedings From Sound to Sense, June 11th-June 13th, MIT.
LAMEL L.F., GAUVIN J.-L. 1993. “Identifying Non-linguistic Speech Features.” In: Proceedings of EuroSpeechConference, Berlin, September 1993, 121-124.
LAVER John. 1980. The Phonetic Description of Voice Quality. Cambridge: CUP.
LAVER John. 1994. Principles of Phonetics. Cambridge: CUP.
LAVER John. 1995. “The Phonetic Description of Paralinguistic Phenomena.” In: Proceedings of the XIIIth ICPhS, Stockholm, Sweden.
LAVER John. 2009. The Description of Voice Quality in General Phonetic. Cambridge: CUP.
LAVER John, TRUDGILL Peter. 1979. “Phonetic and Linguistic Markers in Speech.” In: SCHERER & GILES 1979: 1-32.
LIEBERMAN Philip, MICHAELS S.B. 1962. “Some Aspects of the Fundamental Frequency and Envelope Amplitude as Related to Emotional Content of Speech”. Journal of the Acoustical Society of America 34, 922-927.
LISCOMBE Jackson. 2007. Prosody and Speaker State: Paralinguistics, Pragmatics, and Proficiency. PhD Thesis, Columbia University.
LUGGER Marko, YANG Bin. 2008. “Psychologically Motivated Multi-Stage Emotion Classification Exploiting Voice Quality Features.” In: MIHELIČ & ŽIBERT 2008: 395-410.
MARKOVÁ Ivana, GRAUMANN Carl F., FOPPA Klaus (eds.). 1995. Mutualities in Dialogue. Cambridge: CUP.
MCCORMACK W.C., WURM S.A. (eds.). 1976. Language and Man: Anthropological Issues. The Hague: Mouton.
MCNEILL David. 1985. “So You Think Gestures Are Nonverbal?” Psychological Review 92(3), 350-371.
MCNEILL David. 1992. Hand and Mind: What Gestures Reveal about Thought. Chicago: University of Chicago Press.
MCNEILL David. 2006. “Gesture and Communication.” In: BROWN 2006: 58-65.
MCNEILL David, LEVY Elene. 1982. “Conceptual Representations in Language Activity and Gesture.” In: JARVELLA & KLEIN 1982: 271-295.
MCNEILL David, BERTENTHAL Bennett, COLE Jennifer, GALLAGHER Shaun. 2005. “Gesture-first, but Nogestures? Commentary on Michael Arbib ‘From Monkey-like Action Recognition to Human Language: An Evolutionary Framework Forneurolinguistics’.” Behavioral and Brain Sciences 28, 105-124.
MIHELIČ France, ŽIBERT Janes (eds.). 2008. Speech Recognition, Technologies and Applications. Vienna: I-Tech.
MOZZICONACCI Sylvia J.L. 1995. “Pitch Variations and Emotion in Speech.” Proceedings of the XIIIth ICPhS, Stockholm, Sweden, vol. 1, 178-181.
MOZZICONACCI Sylvia J.L., HERMES Dik J. 1997. “A Study of Intonation Patterns in Speech Expressing Emotion or Attitude: Production and Perception.” In: IPO Annual Progress Report 32, Eindhoven, the Netherlands, 154-160.
OHALA John. 1983. “Cross-language Use of Pitch: An Ethological View.” Phonetica 40, 1-18.
PIERREHUMBERT Jannet, HIRSCHBERG Julia. 1990. “The Meaning of Intonational Contours in the Interpretation of Discourse.” In: COHEN et al. 1990: 271-311.
PIKE Kenneth. 1943. Phonetics. Ann Arbor: University of Michigan Press.
ROACH Peter. 2000. “Techniques for the Description of Emotional Speech.” In: Proceedings of the ISCA Workshopon Speech and Emotion, Belfast, 53-59.
ROSSINI Nicola. 2012. Reinterpreting Gesture as Language. Amsterdam: IOS Press.
SCHERER Klaus R. 2003. “Vocal Communication of Emotion: A Review of Research Paradigms.” Speech Communication 40, 227-256.
SCHERER Klaus R., GILES Howard (eds.). 1979. Social Markers in Speech. Cambridge: CUP.
SCHERER Klaus R., BANSE Rainer, WALLBOTT Herald. G. 2001. “Emotion Inferences from Vocal Expression Correlate across Languages and Cultures.” Journal of Cross-Cultural Psychology 32(1), 76-92.
SCHÖTZ Susanne. 2002. “Linguistic & Paralinguistic Phonetic Variation in Speaker Recognition & Text-to-Speech Synthesis.” In: GSLT Papers: Speech Technology 1.
SEBEOK Thomas Albert (ed.). 1974. Current Trends in Linguistics, vol. 12. The Hague: Mouton.
SILVERSTEIN Michael. 1976. “Shifters, Linguistic Categories, and Cultural Description.” In: BASSO & SELBY 1976.
SMITH Henry L. 1952. An Outline of Metalinguistic Analysis. Washington, D. C.: Department of State, Foreign Service Institute.
SMITH Henry L. 1953. The Communication Situation. Washington, D. C.: Department of State. Foreign Service Institute.
STURTEVANT Edgar H. 1947. An Introduction to Linguistic Science. New Haven: Yale University (cited from CRYSTAL 1976).
TENCH Paul. 1990. The Roles of Intonation in English Discourse. Frankfurt am Main: Verlag Peter Lang.
THOMPSON William Forde, BALKWILL Laura-Lee. 2006. “Decoding Speech Prosody in Five Languages.” Semiotica 158(1/4), 407-424.
THORPE William Homan. 1972. “The Comparison of Vocal Communication in Animal and Man.” In: HINDE 1972. (cited from CRYSTAL 1976).
TRAGER George L. 1958. “Paralanguage: A First Approximation.” In: HYMES 1958: 1-12.
TRAGER George L. 1960. “Taos III: Paralanguage.” Anthropological Linguistics 2(2), 24-30.
TRAGER George L. 1961. “The Typology of Paralanguage.” Anthropological Linguistics 3(1), 17-21.
TRAGER George L. 1964. Phonetics: Glossary and Tables. New York: Studies in Linguistics.
TRASK Robert Lawrence. 1996. A Dictionary of Phonetics and Phonology. London: Routledge.
WICHMANN Anne. 2002. “Attitudinal Intonation and the Inferential Process.” In: Proceedings of Speech Prosody2002 Conference, Aix-en-Provence.
WŁODARCZAK Marcin. 2012. “Ranked Multidimensional Dialogue Act Annotation.” In: Proceedings of ESSLLI2010: International Conference on New Directions in Logic, Language and Computation. Berlin-Heidelberg: Springer-Verlag, 67-77.
YASINNIK Yelena, RENWICK Margaret, SHATTUCK-HUFNAGEL Stefani. 2004. “The Timing of Speech-accompanying Gestures with Respect to Prosody.” In: From Sound to Sense, June 11-June 13, 2004, Cambridge, MA, MIT.
License
Copyright (c) 2013 Maciej Karpiński
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.