Intro to Auto Speech Recognition -- How ML Learns Speech-to-Text
Download as PPTX, PDF1 like613 views
Automatic speech recognition (ASR) is the technology that converts speech to written text. There are two main approaches: static systems that use acoustic, pronunciation, and language models sequentially; and end-to-end neural networks that use deep neural networks for feature extraction, acoustic modeling, and language modeling. Challenges for ASR systems include noise, variations in accents and ages, transferring learning across dialects, and operating locally on devices without internet.
More Related Content
Similar to Intro to Auto Speech Recognition -- How ML Learns Speech-to-Text (20)
![Project_Phase1_-_Literature_Review-1[1].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/projectphase1-literaturereview-11-231117183848-e100b33e-thumbnail.jpg?width=560&fit=bounds)
Intro to Auto Speech Recognition -- How ML Learns Speech-to-Text
- 2. What is Auto Speech Recognition (ASR)? Hello, how are you? Voice Speech Waveform Feature from Audio (e.g., Spectrogram) Auto ML Text Automatic Speech Recognition (ASR) The technology of converting speech to written form (called speech-to- text) which human can interpret the meaning of text. 1/7
- 3. How to Understand Speech: Approach by Educated Human • Reading Spectrogram source: Step by step through a spectrogram, https://www.youtube.com/watch?v=lfZ6XSRaRR8 2/7
- 4. How to Understand Speech: Approach by AI • Static ASR system Sound Waveform ↓ Acoustic Feature Extraction ↓ Acoustic Model ↓ Pronunciation Model ↓ Language Model ↓ Text • End-to-End Neural ASR System Sound Waveform ↓ Acoustic Feature Extraction ↓ Acoustic Model with DNN ↓ Language Model with DNN ↓ Text (Graves, Jaitley, 2014, “Towards End-to-End Speech Recognition with Recurrent Neural Networks”, ICML) (Chan et. al., 2016, “Listen, Attend and Spell: A Neural Network for LargeVocabulary Conversational Speech Recognition”, ICASSP) 3/7
- 5. Challenges in a System Using Only Acoustic Model Word “Probably” Dictionary Pronunciation pr aa b ax b l iy Actual pronunciations (many common ways to pronounce) p r aa b iy p r aw l uh p r aa l iy p aa b uh b liy p ow ih p aa iy p r ah b iy (Preethi Jyothi, 2017, “Automatic Speech Recognition - An Overview”) I • probably (p r aa b iy) • probability (p r aa b iy) play tennis. Language Model 4/7
- 6. BeyondText – Insights Insight • Topic Classification • Semantic Parsing and Question Answering • Customer Segmentation/ Prioritization • Summary Generation Voice Sound Waveform Feature Auto ML Text Insights! 5/7
- 7. Current Challenges in ASR • Noisy real-life conversion with multiple speakers • Robustness to variations in ages and accents • Integration of effort across multiple dialects with transfer learning • Embedded ASR system locally on mobile devices without internet connection • Bad channel conditions (intermittently dropping voice) (Preethi Jyothi, 2018, “State-of-the-Art in Speech Technologies”) 6/7
- 8. Great Study Materials on Speech Recognition atYouTube Deep Learning Lecture Series 1. “CS231N Winter 2016” Convolutional Neural Networks for Visual Recognition by Stanford University (Andrej Karpathy ver.), 16 videos 2. “CS231N Spring 2017” Convolutional Neural Networks for Visual Recognition by Stanford University, 16 videos 3. “CS224N Winter 2017” Natural Language Processing with Deep Learning by Stanford University, 19 videos Auto Speech Recognition Sessions 1. Automatic Speech Recognition - An Overview Presenter is Preethi Jyothi, IIT Bombay in Sep 2017 2. State-of-the-Art in Speech Technologies Presenter is Preethi Jyothi, IIT Bombay in Jan 2018 3. Lecture 2 | Word Vector Representations: word2vec Lecture 2 in Natural Language Processing with Deep Learning 4. Step by step through a spectrogram Lecturer is Andy McMillin, Clinical Associate Professor at Portland State University 7/7
Editor's Notes
- #3: The reason that the sample is discretized within a certain sample rate is that within around 25 mili-seconds, you speech signal is stationary. Starting from raw speech waveform, we generate tiny slices, which is called speech frames, each speech frame represents a feature.
- #4: Amy Costanza Smith
- #8: Academic Research Summit, which was co-organized by Microsoft Research, was held at the International Institute of Information Technology (IIIT) Hyderabad on the 24th and 25th of January 2018.