Dernière sortie
Call Jotting Using 80-dim Fbank Audio input CNN Feature Extraction Methods with RBM-DBN Pre-training HMM Temporal Smoothing and MLP Classifier Algorithms.
"Call Jotting Using 80-dim Fbank Audio Input CNN Feature Extraction Methods with RBM-DBN Pre-training HMM Temporal Smoothing and MLP Classifier AlgorithmsFunctions, Techniques, and Comparative AnalysisThis book presents a rigorous, first-principles analysis of every machine learning function, architectural component, loss criterion, and training technique used across 18 generations (V1-V18) of CallJots audio-plus-text jot extraction research - 765 experiments in total - delivering the most comprehensive comparative study of neural architectures for automatic speech jot classification ever published.
You will learn how CNN and TCN frontends process 80-band mel filterbank features, how RBM-DBN unsupervised pre-training initializes deep audio networks, how DNN-HMM hybrid architectures with Viterbi temporal decoding achieve state-of-the-art results, and why sequence-level ASR criteria (LF-MMI, sMBR) consistently degrade utterance-level classification performance by 10-12%. The V13 DNN-HMM hybrid - the current best pipeline at type_f1=0.7140 - combines a CNN audio frontend, RBM-DBN pre-training, and HMM late-fused with a Sentence-BERT text stream (all-MiniLM-L6-v2).
The book explains every design decision, from the mathematical formulation of Fbank extraction (Davis & Mermelstein, 1980) to focal loss with label smoothing (gamma=2.0, epsilon=0.1) for handling extreme class imbalance across five jot types: topic, decision, question, action_item, and commitment. Key technical coverage includes: Sentence-BERT embeddings for text stream encoding (Reimers & Gurevych, 2019); 80-dim mel-scaled filterbank features with 25ms window and 10ms hop; stacked Conv2D layers with batch normalization and ReLU for local time-frequency feature extraction; dilated causal convolutions in TCN (receptive field of 63 frames) versus standard CNN; 4-layer RBM-DBN pre-training converging via contrastive divergence; HMM with 5-state left-to-right topology and Viterbi decoding for temporal smoothing; late fusion of audio posteriors and text logits with tuned lambda=0.55; focal loss analysis showing +0.0963 F1 gain on rare action_item class; the label cycling problem and why data quality exceeds quantity; and why attention variants (V16, V17) fail to converge without self-supervised pre-training.
All experiments were conducted on RunPod RTX A4000 (16GB) with PyTorch 2.4.1+cu124 across LibriSpeech train-clean-100 and Switchboard (124, 794 labeled utterances). The book includes 14 figures, 7 tables, complete per-class analysis, error reduction waterfall charts, training dynamics curves, and an extensive literature review mapping every technique to its historical origins from DTW through wav2vec 2.0.
Essential reading for ML engineers, audio researchers, NLP practitioners, and anyone working on speech recognition, call transcription, turn-taking analysis, meeting summarization, or multi-modal (audio+text) neural classification systems. Every claim is backed by experimental data, every failure is documented with root-cause analysis, and every finding is positioned within the 50-year lineage of acoustic modeling research.
You will learn how CNN and TCN frontends process 80-band mel filterbank features, how RBM-DBN unsupervised pre-training initializes deep audio networks, how DNN-HMM hybrid architectures with Viterbi temporal decoding achieve state-of-the-art results, and why sequence-level ASR criteria (LF-MMI, sMBR) consistently degrade utterance-level classification performance by 10-12%. The V13 DNN-HMM hybrid - the current best pipeline at type_f1=0.7140 - combines a CNN audio frontend, RBM-DBN pre-training, and HMM late-fused with a Sentence-BERT text stream (all-MiniLM-L6-v2).
The book explains every design decision, from the mathematical formulation of Fbank extraction (Davis & Mermelstein, 1980) to focal loss with label smoothing (gamma=2.0, epsilon=0.1) for handling extreme class imbalance across five jot types: topic, decision, question, action_item, and commitment. Key technical coverage includes: Sentence-BERT embeddings for text stream encoding (Reimers & Gurevych, 2019); 80-dim mel-scaled filterbank features with 25ms window and 10ms hop; stacked Conv2D layers with batch normalization and ReLU for local time-frequency feature extraction; dilated causal convolutions in TCN (receptive field of 63 frames) versus standard CNN; 4-layer RBM-DBN pre-training converging via contrastive divergence; HMM with 5-state left-to-right topology and Viterbi decoding for temporal smoothing; late fusion of audio posteriors and text logits with tuned lambda=0.55; focal loss analysis showing +0.0963 F1 gain on rare action_item class; the label cycling problem and why data quality exceeds quantity; and why attention variants (V16, V17) fail to converge without self-supervised pre-training.
All experiments were conducted on RunPod RTX A4000 (16GB) with PyTorch 2.4.1+cu124 across LibriSpeech train-clean-100 and Switchboard (124, 794 labeled utterances). The book includes 14 figures, 7 tables, complete per-class analysis, error reduction waterfall charts, training dynamics curves, and an extensive literature review mapping every technique to its historical origins from DTW through wav2vec 2.0.
Essential reading for ML engineers, audio researchers, NLP practitioners, and anyone working on speech recognition, call transcription, turn-taking analysis, meeting summarization, or multi-modal (audio+text) neural classification systems. Every claim is backed by experimental data, every failure is documented with root-cause analysis, and every finding is positioned within the 50-year lineage of acoustic modeling research.
Les livres de Chris Sardius
Spectre Concord
E-book
8,49 €