AuToBI v1.3

This release to AuToBI is a more traditional milestone release than v1.2 was.  Trained models and a new .jar file will be available on the AuToBI site shortly.

There are improvements to performance that are thoroughly documented in a submission to IEEE SLT 2012.  These improvements were achieved from two sources.

First, AuToBI uses importance weighting to improve classification performance on skewed distributions.  I found this to be a more useful approach than the standard under- or over-sampling.  This is discussed in a paper that will appear at Interspeech next month.

Second, inspired by features that Taniya Mishra, Vivek Sridhar and Aliaster Conkie developed at AT&T, I included some new features which had a big payoff.  (They described these features in an upcoming Interspeech 2012 paper).  One of the most significant was to calculate the area under a normalized intensity curve.  This has a strong correlation with duration, but is more robust.  You could make an argument that it approximates "loudness" by incorporating duration and intensity.  This is a pretty poor psycholinguistic or perceptual argument so I wouldn't make it too strongly, but it could be part of the story.

Here is a recap of speaker-independent acoustic-only performance on the six ToBI classification tasks on BURNC speaker f2b.

Task	Version 1.2	Version 1.3
Pitch Accent Detection	81.01% F1:83.28	84.83% F1:86.58
Intermediate Phrase Detection	75.41% F1:43.15	77.97% F1:44.43
Intonational Phrase Detection	86.91% F1:74.50	90.36% F1:76.49
Pitch Accent Classification	18.46% Average Recall:18.97	16.33% Average Recall:21.06
Phrase Accent Classification	48.34% Average Recall:47.99	47.44% Average Recall:48.31
Phrase Accent/Boundary Tone Classification	73.18% Average Recall:25.92	74.47% Average Recall:26.02

There are also a number of improvements to AuToBI from a technical side and as a piece of code.

First of all, unit test coverage has increased from ~11% to ~73% between v1.2 and v1.3.

Second, there was a bug in the PitchExtractor code causing a pretty serious under prediction of unvoiced grames.  (A big thanks to Victor Soto for finding this bug.)

Third, memory use is much lower by a more aggressive deletion of prediction attributes, and through a modification of how WavReader works.

I'd like to thank Victor Soto, Fabio Tesser, Samuel Sanchez, Jay Liang, Ian Kaplan, Erica Cooper and, as ever, Julia Hirschberg and anyone else who has been using AuToBI, for their patience and feedback.

I've been pretty lax about posting here.  I'll try to get better about it in the coming academic year.

This fall is full of travel which will lead to a lot of ideas and not enough time to work on them.

AuToBI Version 1.2

I hadn't really planned for this current improvement to AuToBI be a milestone release.

I'm about halfway through an effort to get test coverage up to 90-95% of lines and 100% of classes.  I promise it'll get there eventually.

But in the mean time, I was playing with an improvement to how attributes are associated to data points.  I knew this was a significant source of inefficiency, but didn't quite expect this much.

Here are memory usage graphs for training a Pitch Accent Detection model on the Boston University Radio News Corpus -- about 22k data points and 136 features.  The first one is on my MacBookPro Laptop with 4G RAM (and a lot of other nonsense running).

The max memory usage of Version 1.1 was 1914Mb, with this improvement it tops out at 1049Mb. An improvement of about 45%.  (You'll notice it also ends a little bit quicker too, but this is probably because of fewer or quicker garbage collection calls.)

I figured I'd check on a compute server too, one of the Speech Lab @ Queens College's Quad Core Intel Xeon Processor E5450 (3.0GHz,2X6ML2,1333) with 4Gb RAM.  

Similar results here.  Max memory usage of version 1.1 was 2343Mb and with the improvement 1392Mb. Improving by 40%. (And the speed improvement is here too.) I don't have a good explanation for why the linux version is taking more memory to run, but for now I'll assume it has something to do with the difference to the JVM.

There are some other bugfixes in this version, but this is the big reason to upgrade.

The version 1.2 is available from github
git clone git@github.com:AndrewRosenberg/AuToBI.git

AuToBI Version 1.1

I've made enough improvements to AuToBI to consider the toolkit a milestone more mature.

In addition to some uninteresting bug fixes, and refactoring, the version 1.1 is made up by 3 significant changes.  As ever, the AuToBI homepage includes milestone releases, and the project itself is hosted on github.

1) Package restructuring.

The internal structure makes more sense now.  Classes are divided into 'core', 'feature extractors', 'classifiers', 'feature sets', 'io' and 'utilities'.  With over 100 classes, the code had outgrown a flat package structure.  Unfortunately this restructuring changed the serialization signatures of class names.  This means that old models won't work with the version 1.1 release.  But I have new models trained on the Boston Directions Corpus and Boston Radio News Corpus which will be available from the AuToBI homepage shortly.

2) Implemented reference counting for Feature maintenance.

The feature extraction process allows a user to specify the features they want without explicitly going through all the intermediate steps to extract them.  For example, extracting the mean speaker normalized pitch from the second half of a word requires pitch to be extracted, speaker normalization parameters to be calculated or retrieved, the pitch to be normalized, the second half of the word to be identified, and finally the mean calculated.  In AuToBI each of these steps are treated as features that are required by the feature extraction of the user-desired feature.  In version 1.0, AuToBI was able to identify which features were required, but never recognized when a feature wasn't needed any longer.  Version 1.1 includes functionality that maintains a reference count for each feature based on how many features that still need to be extracted are going to need it -- speaker normalization parameters may be required by many requested features.  This allows for tidier memory management, which should, in turn, allow AuToBI to operate on more material.

3) Reduced storage for acoustic contours.

In version 1.0 pitch and intensity contours were stored as lists of time-value pairs -- a storage class containing 2 doubles.  Assuming 10ms samples, and 16bytes per sample, this is about 1600bytes per second.  Not unacceptably inefficient, but definitely could be improved.  The other approach would be to only store values, and let the time of each point be specified by a start time (t0) and step size (dt).  This allows storage with 8bytes per sample plus 16bytes for the parameters.  The problem with this approach is that pitch points are invalid when there is no periodic material detected.  The time-value pair approach handled this simply and easily.  The new solution uses an array of one-bit booleans for each sample in the contour, describing whether it was a 'valid' point or not.  If the rate of periodic to aperiodic material is  less than 32:1, the new approach will reduce the memory requirements.  (As I'm writing this, I realize that this ratio could be explicitly tested during pitch extraction, selecting the most efficient storage solution at runtime.  Keep an eye out in a new release.)

The changes are mostly inside baseball kind of things, but possibly useful for AuToBI users who get their hands dirty in the code -- whoever you are.   For everyone else, the most obvious changes you'll notice is that version 1.0 models don't work any more, and a speedup of ~6%.