Unsupervised denoisers such as MixIT and noisy-target training learn directly from noisy recordings, avoiding
the domain mismatch of synthetic training data, but remain well below supervised training. Teacher–student
self-training, as in RemixIT, can close part of that gap without clean references.
Problem: The student is trained only toward the teacher's estimates, so teacher errors, in particular
over-suppressed speech, cannot be recovered and are propagated: the regression analogue of confirmation bias in
pseudo-labelling.
Proposed method: SNOAR (self-training with noise filtering, observation adding and remixing) corrects the
teacher's estimates before they are used as self-supervision:
noise filtering: differential noise filtering (DNF) subtracts the teacher's own noise estimate,
removing the noise it leaks into its speech estimate;
observation adding: combining the result with the noisy observation partly restores the
suppressed speech and attenuates denoising artifacts.
Results (WHAM!): From a MixIT teacher, SNOAR reaches 13.25 dB SI-SDR, within 0.43 dB of a supervised
topline, outperforms RemixIT on every metric, and has the lowest word error rate of all systems trained without
clean references.
Results (CHiME-3 real): SNOAR improves every reference-free quality metric and the word error rate of three
different teachers, whereas RemixIT falls below each of its teachers on every reference-free metric.
Click a spectrogram to play itSwitching tiles within an example keeps the playback positionClick inside a playing spectrogram to seekspace pause / resume
WHAM! (simulated, tt set)
Example 1
Input SNR -4.9 dB · SNOAR(MixIT) SI-SDRi +14.6 dB
Transcript: The hot fluid is circulated through a tube located in the lower tank of the radiator
Teacher
RemixIT student
SNOAR student
MixIT
Example 2
Input SNR -3.3 dB · SNOAR(MixIT) SI-SDRi +14.6 dB
Transcript: In certain cases the cards are given free to subscribers
Teacher
RemixIT student
SNOAR student
MixIT
Example 3
Input SNR -2.9 dB · SNOAR(MixIT) SI-SDRi +14.7 dB
Transcript: All of a sudden they came back and said We aren\'t buying at this yield
Teacher
RemixIT student
SNOAR student
MixIT
Example 4
Input SNR -2.1 dB · SNOAR(MixIT) SI-SDRi +14.6 dB
Transcript: He said such products would be marketed by other companies with experience in that business
Teacher
RemixIT student
SNOAR student
MixIT
Example 5
Input SNR +0.2 dB · SNOAR(MixIT) SI-SDRi +14.8 dB
Transcript: He has been told by the Justice Department that before long he may be indicted on criminal charges
Teacher
RemixIT student
SNOAR student
MixIT
Example 6
Input SNR +1.4 dB · SNOAR(MixIT) SI-SDRi +14.7 dB
Transcript: The Statue of Liberty and Ellis Island are within the New Jersey waters of New York Bay
Teacher
RemixIT student
SNOAR student
MixIT
CHiME-3 (real recordings, et05)
Bus · F05
Real recording, channel 5 · no clean reference exists
Transcript: Using a measure that counts the military among the employed the rate was unchanged at six point six percent last month
Teacher
RemixIT student
SNOAR student
MixIT, no clean speech
MixIT + WSJ0 synthetic
MixIT + LibriSpeech synthetic
Bus · M06
Real recording, channel 5 · no clean reference exists
Transcript: The may slump reported june twenty second came as a big surprise to most analysts and helped trigger a powerful bond rally that day
Teacher
RemixIT student
SNOAR student
MixIT, no clean speech
MixIT + WSJ0 synthetic
MixIT + LibriSpeech synthetic
Café · F05
Real recording, channel 5 · no clean reference exists
Transcript: Today ninety percent of the four billion dollars of b. a. s. f. sales in the u. s. is produced there
Teacher
RemixIT student
SNOAR student
MixIT, no clean speech
MixIT + WSJ0 synthetic
MixIT + LibriSpeech synthetic
Café · M05
Real recording, channel 5 · no clean reference exists
Transcript: The consensus was that a new piece of paper isn't required said one u. s. diplomat
Teacher
RemixIT student
SNOAR student
MixIT, no clean speech
MixIT + WSJ0 synthetic
MixIT + LibriSpeech synthetic
Pedestrian area · F06
Real recording, channel 5 · no clean reference exists
Transcript: Texaco has three choices a company adviser says
Teacher
RemixIT student
SNOAR student
MixIT, no clean speech
MixIT + WSJ0 synthetic
MixIT + LibriSpeech synthetic
Pedestrian area · M06
Real recording, channel 5 · no clean reference exists
Transcript: No one is making very much money on it acknowledges brian j. kelly chairman of bell atlantic's investment development unit
Teacher
RemixIT student
SNOAR student
MixIT, no clean speech
MixIT + WSJ0 synthetic
MixIT + LibriSpeech synthetic
Street · M06
Real recording, channel 5 · no clean reference exists
Transcript: Sony which lost points in previous sessions this week rebounded eighty to five thousand one hundred thirty
Teacher
RemixIT student
SNOAR student
MixIT, no clean speech
MixIT + WSJ0 synthetic
MixIT + LibriSpeech synthetic
Street · F06
Real recording, channel 5 · no clean reference exists
Transcript: But while the fed stands pat it is coming under increasing attack from both sides