C# Mel-Spectrogram 梅尔频谱

介绍

Main features

Philosophy of NWaves

效果

项目

代码

下载

C# Mel-Spectrogram 梅尔频谱

介绍

利用NWaves实现Mel-Spectrogram 梅尔频谱

NWaves github 地址：https://github.com/ar1st0crat/NWaves

NWaves is a .NET DSP library with a lot of audio processing functions.

Main features

major DSP transforms (FFT, DCT, MDCT, STFT, FWT, Hilbert, Hartley, Mellin, cepstral, Goertzel)
signal builders (sine, white/pink/red/Perlin noise, awgn, triangle, sawtooth, square, pulse, ramp, ADSR, wavetable)
basic LTI digital filters (moving average, comb, Savitzky-Golay, pre/de-emphasis, DC removal, RASTA)
FIR/IIR filtering (offline and online), zero-phase filtering
BiQuad filters (low-pass, high-pass, band-pass, notch, all-pass, peaking, shelving)
1-pole filters (low-pass, high-pass)
IIR filters (Bessel, Butterworth, Chebyshev I & II, Elliptic, Thiran)
basic operations (convolution, cross-correlation, rectification, amplification, fade / crossfade)
block convolution (overlap-add / overlap-save offline and online)
basic filter design & analysis (group delay, zeros/poles, BP, BR, HP from/to LP, SOS, combining filters)
state space representation of LTI filters
FIR filter design: frequency sampling, window-sinc, equiripple (Remez / Parks-McClellan)
IIR filter design: IirNotch / IirPeak / IirCombNotch / IirCombPeak
non-linear filters (median filter, distortion effects, bit crusher)
windowing functions (Hamming, Blackman, Hann, Gaussian, Kaiser, KBD, triangular, Lanczos, flat-top, Bartlett)
periodograms (Welch / Lomb-Scargle)
psychoacoustic filter banks (Mel, Bark, Critical Bands, ERB, octaves) and VTLN warping
customizable feature extraction (time-domain, spectral, MFCC, PNCC/SPNCC, LPC, LPCC, PLP, AMS)
preconfigured MFCC extractors: HTK (MFCC-FB24), Slaney (MFCC-FB40)
LPC conversions: LPC<->cepstrum, LPC<->LSF
feature post-processing (mean and variance normalization, adding deltas) and CSV serialization
spectral features (centroid, spread, flatness, entropy, rolloff, contrast, crest, decrease, noiseness, MPEG7)
harmonic features (harmonic centroid and spread, inharmonicity, tristimulus, odd-to-even ratio)
time-domain characteristics (rms, energy, zero-crossing rate, entropy)
pitch tracking (autocorrelation, YIN, ZCR + Schmitt trigger, HSS/HPS, cepstrum)
chromagram (chroma feature extractor)
time scale modification (phase vocoder, PV with identity phase locking, WSOLA, PaulStretch)
simple resampling, interpolation, decimation
bandlimited resampling
wavelets: haar, db, symlet, coiflet
polyphase filters
noise reduction (spectral subtraction, sciPy-style Wiener filtering)
sound effects (echo, tremolo, wahwah, phaser, chorus, vibrato, flanger, pitch shift, morphing, robotize, whisperize)
3D/Stereo audio (stereo panning, stereo and ping-pong delay, ITD-ILD, binaural panning)
envelope following
dynamics processing (limiter / compressor / expander / noise gate)
harmonic/percussive separation
Griffin-Lim algorithm
Karplus-Strong synthesis
PADSynth synthesis
adaptive filtering (LMS, NLMS, LMF, SignLMS, RLS)
simple modulation/demodulation (AM, ring, FM, PM)
simple audio playback and recording

Philosophy of NWaves

NWaves was initially intended for research, visualizing and teaching basics of DSP and sound programming.

Usually, DSP code is quite complicated and difficult to read, because it's full of optimizations (which is actually a very good thing). NWaves project aims in particular at achieving a tradeoff between good understandable code/design and satisfactory performance. Yet, the main purpose of this lib is to offer the DSP codebase that would be:

easy to read and understand
easy to incorporate into existing projects
easy to port to other programming languages and frameworks
even possibly treated as the DSP/audio textbook.

效果

项目

代码

using NWaves.Audio;
using NWaves.FeatureExtractors;
using NWaves.FeatureExtractors.Options;
using NWaves.Filters.Fda;
using NWaves.Signals;
using NWaves.Windows;
using System;
using System.IO;
using System.Text;
using System.Windows.Forms;

namespace C__Mel_Spectrogram_梅尔频谱
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }

        WaveFile waveContainer;
        StringBuilder sb = new StringBuilder();

        private void button1_Click(object sender, EventArgs e)
        {
            using (var stream = new FileStream("你好.wav", FileMode.Open))
            {
                waveContainer = new WaveFile(stream);
            }

            DiscreteSignal left = waveContainer[Channels.Left];
            //DiscreteSignal right = waveContainer[Channels.Right];

            //Mel-Spectrogram
            var fftSize = 1024;
            var hopSize = 512;
            var melCount = 16;
            int samplingRate = left.SamplingRate;

            var mfccExtractor = new FilterbankExtractor(
               new FilterbankOptions
               {
                   SamplingRate = samplingRate,
                   FrameSize = fftSize,
                   FftSize = fftSize,
                   HopSize = hopSize,
                   Window = WindowType.Hann,
                   FilterBank = FilterBanks.Triangular(fftSize, samplingRate, FilterBanks.MelBands(melCount, samplingRate)),
                   // if power = 1.0
                   // SpectrumType = SpectrumType.Magnitude
               });


            var mfccVectors = mfccExtractor.ParallelComputeFrom(left);

            sb.Clear();

            foreach (var item in mfccVectors)
            {
                sb.AppendLine(String.Join(",", item));
            }

            textBox1.Text = sb.ToString();
        }
    }
}