Does intensity correlate with prominence in French?

According to a bunch of French researchers who study prosody: No.

I learned this at a prominence workshop at Speech Prosody 2010.  I asked Mathieu Avanzi why, in his paper "A Corpus-based Learning Method for Prominence Detection in Spontaneous Speech", he and his co-authors looked at pitch, duration and pause features, but not intensity or spectral emphasis.  The response: "Intensity does not correlate with prominence in French".

Now, I don't speak French, so far be it from me to comment on what is perceived as intonational prominence in French by French speakers.

But...

Intensity correlates with prominence in (at least) English, Dutch and Italian.  So my curiosity was piqued.

And...

At the same workshop, Mathieu and others released C-PROM, a corpus of French speech which has been annotated for prominence, and labeled by French speakers no less!

So I figured it would only take a few minutes to check it out.  Using the feature extraction routines in AuToBI, I pulled out mean values of pitch, intensity and duration for each annotated syllable.  Armed with a t-test and R, I looked to see which if any of these features correlate with the labels of prominence.  (For this analysis, I collapsed the annotations for strong and weak prominence.)

So, let's look at the data.

Bold Claim 1: Pitch correlates with prominence in French.

The bimodal distribution of mean pitch is almost certainly due to the presence of male and female speakers in the C-PROM material.  But even without any speaker or gender normalization of pitch, we can still evidence of the correlation between mean pitch and prominence.  The mean value of prominent syllables is 185.6Hz compared to 158.0Hz for non-prominent syllables.  This has an associated t-score of 24.117 (p < 2.2*10^-16).

Bold Claim 2: Duration correlates with prominence in French.

This result is even clearer.  Prominent syllables are on average 97ms longer (261ms) than non-prominent syllables (164ms). This has a t-value of 54.240 (p < 2.2*10^-16).

Bold Claim 3: Intensity correlates with prominence in French.

Well there it is.  It's not as pronounced a difference as the difference in pitch or duration, but the data shows a clear correlation between mean intensity in a syllable and whether the syllable is prominent or not.  Prominent syllables are on average 1.6dB louder than non-prominent syllables (72.08dB vs. 70.48dB).  This corresponds to a t-value (15.174) that is lower than that seen in the pitch and duration analyses, but still significant (p < 2.2*10^-16).

Now...This is clearly a very basic analysis of correlates of prominence in French speech.  But based on these results, I'm comfortable answering the question now.

Does intensity correlate with prominence in French? Yes.

[edited at 12:43pm 6/16/2010]

Following up on a comment from Raul Fernandez, I thought I'd post a parallel plot on the correlation of intensity and prominence in English.

Note that this chart is based on data on American English *words* from the Boston Directions Corpus.  Because these are words, the prominent distribution includes some data from non-prominent syllables, so it's not exactly a one-to-one comparison.  But there is evidence that acoustic aggregations drawn from words make *better* predictors of prominence (cf. Rosenberg & Hirschberg 2009).

Here we find a similar difference in mean intensity (1.9dB) between prominent (60.8dB) and non-prominent words (58.9dB). This has an associated t-value of 21.234 (p < 2.2*10^-16).

There is little controversy about the correlation of intensity with prominence in English. (In the last few years, there has been work even suggesting that intensity is a better predictor of prominence than pitch, (cf. Kochanski et al. 2005, Silipo & Greenberg 2000, and Rosenberg & Hirschberg 2009).)   Of course, this chart doesn't indicate that there are equivalent relationships between intensity and prominence in French and English -- merely that the French correlation deserves more attention.

Speech Prosody 2010 Recap

Speech Prosody is a biannual conference held by a special interest group of ISCA.  Despite working on intonation and prosody since about 2006, this is the first year I've attended.

The conference has only been held five times and has the feeling of a workshop -- no parallel sessions is nice, but the quality of work was very varied.  I'm sympathetic to conference organizers and particularly sympathetic when you're coordinating a conference that is still relatively new.  I'm sure you want to make sure people attend, and the easiest way to do that is to accept work for presentation.  But by casting a wider net some work gets in that probably could have used another round of revision.  Mark Hasegawa-Johnson and his team logistically executed a very successful conference.  But (at the risk of having my own work rejected) I think the conference is mature enough that Speech Prosody 2012 could stand more wheat, less chaff.

Two major themes stuck out:

• Recognizing emotion from speech -- A lot of work, but very little novelty in corpus, machine learning approach, or findings.  It's easy to recognize high vs. low activation/arousal, and hard to recognize high vs. low valence.  (I'll return to this theme in a post on Interspeech reviewing...)
• Analyzing non-native intonation -- There were scads of papers on this topic covering both how intonation is perceived and produced by non-native speakers of a language (including two by me!). I had no anticipation that this would be so popular, but if this conference were any indication, I would expect to see a lot of work on non-native spoken language processing at Interspeech and SLT.

Here are some of my favorite papers. This list is *heavily* biased towards work I might cite in the future more than the "best" papers at the conference.

• The effect of global F0 contour shape on the perception of tonal timing contrasts in American English intonation Jonathan Barnes, Nanette Veilleux, Alejna Brugos and Stefanie Shattuck-Huffnagel  The paper examines the question of peak timing in the perception of L+H* vs. L*+H accents.  Generally has been assumed (including in the ToBI guidelines) that the difference is in the peak timing relative to the accented vowel or syllable onset -- with L*+H having later peaks. Barnes et al. found that this can be manipulated such that if you keep the peak timing the same, but make the F0 approach to that peak more "dome-y" or "scoop-y", subjects will perceive a difference in accent type.  This is interesting to me, though *very* inside baseball.  What I liked most about the paper and presentation was the use of Tonal Center of Gravity or ToCG to identify a time associated with the greatest mass of f0 information.  This aggregation is more robust to spurious f0 estimates than a simple maximum, and is a really interesting way to parameterize an acoustic contour.  Oh, and it's incredibly simple to calculate: \[ T_{cog} = \frac{\sum_i f_i t_i}{\sum_i f_i}\]. 
• Cross-genre training for automatic prosody classification Anna Margolis, Mari Ostendorf, Karen Livescu Mari Ostendorf and her students have been looking at automatic analysis of prosody for as long as anyone.  In this work, her student Anna is looking at how detection of accents and breaks performance degrades across corpora, looking at the BURNC (radio news speech) and Switchboard (spontaneous telephone speech) data.  This paper found that you suffer a significant degradation of performance when models are trained on one corpus and evaluated on the other, compared to within-genre training -- with lexical features suffering more dramatically than acoustic features (unsurprisingly...).  However, the interesting effect is that by using a combined training set from both genres, the performance doesn't suffer very much.  This is encouraging for the notion that when analyzing prosody, we can include diverse genres of training data and increase robustness.  It's not so much that this is a surprising result as it is comforting.  
• C-PROM: An Annotated Corpus for French Prominence Study M. Avanzi, A.C. Simon, J.-P. Goldman & A. Auchlin This was presented at the Prominence Workshop preceding the conference.  The title pretty much says it all. But it deserves acknowledgement for sharing data with the community.  Everyone working with prosody bemoans the lack of intonationally annotated data.  It's time consuming, exhausting work. (Until last year I was a graduate student -- I've done more than my share of ToBI labeling, and I'm sure there will be more to come.)  These lovely folks from France (mostly) have labeled about an hour of data for prominence from 7 different genres, and ... wait for it ... posted it online for all the world to see, download and abuse.  They've set a great example, and even though I haven't worked with French before and have no intuitions about it, I appreciate open data a lot.

Full disclosure: I had two papers at Speech Prosody 2010, one in the main workshop (circuitously placed in a Special Session) and the other in a workshop on prominence the day before.  And I would certainly include these in the "varied" description of the caliber of work.  Not that I'd describe my work as shoddy papers (though there were some at SP2010, to be sure) but they are quite limited in scope, reporting on two small perception and production studies of non-native intonation.