# Acoustics for Speech ### Will Styler - LIGN 6 --- # “Noise” --- # /nɔɪz/ --- ### ... but that's not what speech actually is ---
--- ### Speech is acoustic - It's all sound - We're using speech gestures to produce an acoustic signal - Gestures without sound are unhelpful - Communication works fine on the phone --- ### Today's Plan - What is sound? - What are the important properties of sound? - How can we visualize them? --- # Yay! Acoustics!! --- ## Acoustics The study of waves ---
---
---
--- ## What is Sound? ---
---
--- ### Sound is compression and rarefaction in a medium - Sound needs something to travel in (like air or water) --- ### (Yes, your childhood is a lie)
--- ### Thinking of sound as waves is helpful - How do speakers work? - Why does clapping cause a sound, but waving your hand through the air doesn’t? - Why are gunshots loud? --- ### We're good at hearing sound - ... but we need to visualize it --- ### Visualizing Sound - Waveforms - Spectra - Spectrograms --- ## Waveform A horizontal cut through the wave showing the peaks and troughs over time - The height/strength of the wave is called its "amplitude"
---
--- ### Let's look at the sounds in this room right now --- ### Waveforms are well and good - ... and you can tell a lot from a waveform ---
---
--- ### ... but we'll want better information to process speech --- ## Frequency The speed with which a wave oscillates - Measured in Hertz (Hz), Cycles per second
--- ### Sound W - 100 Hz - Waveform
--- ### 200Hz - Waveform
--- ### Sound T - 500 Hz - Waveform
--- ### Sound F - 1000 Hz - Waveform
--- ### Frequency is important - Different phenomena produce sounds at different frequencies - Most things produce sounds with a mix of different frequencies, each at different amplitudes - Speech has *many* components at different frequencies --- ### Voice Pitch - Changing the "fundamental frequency" (f0) of your voice changes the perceived "pitch" of your voice - Higher frequency of laryngeal vibration == "higher pitch" - *Intonation is all about this fundamental frequency!* --- ### How can we visualize the frequencies which make up a more complex sound? --- ## Spectral Slice Using a Mathematical Process called a Fourier Transform which breaks a signal down into its component frequencies at a certain time - This creates a "spectral slice" or "FFT" ('Fast Fourier Transform') or "Power Spectrum" --- ## FFT from the middle of "Noise"
--- ### Fourier Transforms show us the component frequencies - This is *much* better information for analyzing speech - So, we're going to use them all the time here --- ### Let's walk through this process in more detail --- ### Sound W - 100 Hz - Waveform
--- ### Sound W - 100 Hz - Spectrum
--- ### Sound T - 500 Hz - Waveform
--- ### Sound T - 500 Hz - Spectrum
--- ### Sound F - 1000 Hz - Waveform
--- ### Sound F - 1000 Hz - Spectrum
--- ### What if we were to play these all at once? --- ### Sound WTF - Waveform
--- ### Sound WTF - Spectrum
--- ### That's much easier to cope with! --- ### Let's see how it works in the real world --- ### ... but what about time? - Spectra only show one 'moment' of the signal --- ### "Noise" - Waveform
--- ### "Noise" - Spectral Slice (FFT)
--- ### How can we bring time back into it? --- ### Let's try one approach --- ## Spectrogram Displays a series of FFTs, with peaks arrayed vertically, to show changes in frequency over time --- ### "Noise" - Waveform
--- ### "Noise" - Spectral Slice (FFT)
--- ### "Noise" - Spectrogram
--- ### Sound WTF - Waveform
--- ### Sound WTF - Spectrum
--- ### Sound WTF - Spectrogram
--- ### Let's have a bit of spectrogram fun --- ### You can have fun on your own - SpectrumView on iOS - https://musiclab.chromeexperiments.com/Spectrogram/ -
- Praat (which you should already have installed) --- ### We're not the only people using Spectrograms - Folks doing Sonar and Submarine Detection love them!
--- ## For next time - Play with some sound visualization tools - Get ready to visualize speech! ---
Thank you!