ESSENTIA CHOP SUITE

Real-time & Batch Audio Analysis for TouchDesigner

Five C++ CHOP plugins powered by Essentia exposing spectrum analysis, mel bands, MFCCs, pitch detection, key estimation, onset/BPM tracking, and EBU R128 loudness metering — all running natively inside TouchDesigner. Each operator supports both real-time per-frame analysis and offline batch processing of entire audio files.

WHAT IT DOES

Five Operators, Two Modes

Each analysis operator (Spectral, Tonal, Rhythm, Loudness) has a Mode parameter that switches between real-time per-frame analysis and offline batch processing. Every operator takes raw audio directly and runs its own FFT internally, in both modes.

Realtime — wire an Audio CHOP straight into any analyzer. Incoming audio accumulates in an internal ring buffer; each cook the operator windows and FFTs the latest full analysis window and outputs 1 sample per channel at TD's cook rate. Each operator uses the window its algorithm wants (Tonal resolves semitones, Rhythm keeps transients sharp) with no shared FFT node to configure.

Batch mode — When set to Batch, each operator processes an entire audio buffer (e.g. from a File In CHOP) on a background thread. Output contains one sample per analysis frame across the full file. Trigger analysis with the Compute pulse or enable Autocompute for automatic re-analysis when the input changes. Flipping Mode never requires rewiring: both modes read the same audio input.

New in v2.0 (breaking change) — in v1.x, Spectral, Tonal, and Rhythm consumed an Essentia Spectrum CHOP in realtime mode. In v2.0 they take raw audio directly in both modes. A Spectrum CHOP wired into an analyzer now raises a clear migration error instead of producing garbage; the fix is to delete the Spectrum node and connect the audio straight in. FFT settings moved onto each analyzer (page Analysis): Tonal's FFT Size defaults to auto (semitone-safe at any sample rate), Rhythm gets a realtime Window Size parameter (512–4096, default 1024), and operators warn whenever the window is shorter than the per-cook audio chunk (audio would be silently skipped).

Spectrum: raw data, not a display — Essentia Spectrum outputs a linear-bin FFT magnitude spectrum plus phase as a static sample buffer. The analyzers no longer consume it; use it for GLSL, resynthesis, or custom feature work. Linear bins look bottom-heavy when plotted directly — for on-screen spectral visualization, TouchDesigner's built-in Audio Spectrum CHOP provides a perceptually scaled output better suited for display.

Mono by design — The suite processes a single audio channel. Stereo analysis would multiply every output channel (e.g., mfcc0_L / mfcc0_R, spectral_centroid_L / spectral_centroid_R), making outputs unwieldy and harder to map in a visual context. For stereo-aware analysis, use a Select CHOP to pick each channel independently and run two separate analysis chains. For most audio-reactive scenarios, the recommended approach is to sum left and right with a Math CHOP (Combine Channels → Average) before feeding the analyzers — this preserves the full frequency content of both channels without phase cancellation artifacts. Channel choice measurably shifts results: fed a stereo CHOP, Batch mode analyzes channel 0 (left) only and warns — on the same file with identical settings, Tonal's batch key_strength reads 0.825 from the L+R average but 0.882 from the left channel alone (same detected key). State how you collapsed to mono whenever you compare analysis numbers.

GETTING STARTED

Installation

Pre-built DLLs — no compilation required. Drop them into TouchDesigner's plugin folder and restart.

Step 1 — Copy the DLLs

Place all 5 plugin DLLs into TouchDesigner's Plugins folder (or any subfolder — TD scans subdirectories):

EssentiaSpectrumCHOP.dll

EssentiaSpectralCHOP.dll

EssentiaTonalCHOP.dll

EssentiaRhythmCHOP.dll

EssentiaLoudnessCHOP.dll

Default plugin path:

C:/Users/<you>/Documents/Derivative/Plugins/Essentia/

Step 2 — Restart TouchDesigner

TouchDesigner loads plugin DLLs at startup. After copying the files, restart TD for the new operators to appear.

Step 3 — Add the operators

In the OP Create Dialog (Tab key), search for the operator names:

Essentia Spectrum Essentia Spectral Essentia Tonal Essentia Rhythm Essentia Loudness

Tip — Connect an Audio CHOP (Realtime) or File In CHOP (Batch) directly to any analysis operator; every operator takes raw audio in both modes. Essentia Spectrum is optional — add it only when you want the raw magnitude + phase spectrum itself. See the Signal Flow section for details.

OPERATORS

The Plugin Suite

Each operator is a standalone DLL loaded by TouchDesigner as a custom CHOP. Click any card to expand its full specification.

Essentia Spectrum OUTPUT

Computes the FFT from raw audio and outputs the raw result: spectrum (magnitude) and phase channels of fftSize/2+1 bins as a static sample buffer. For visualization, GLSL, resynthesis, or custom feature work — the analyzers run their own FFT and do not read from it.

Input

Audio CHOP (mono, first channel used)

Output

1 channel × 513 samples (at FFT 1024) — static indexed buffer, not time-domain

Processing

Reads latest fftSize samples → Windowing (Hann / Hamming / Triangular / Blackman-Harris) → Zero Padding (optional) → Essentia Spectrum algorithm

Parameters

FFT Size Hop Size Window Type Zero Padding

Essentia Spectral ANALYSIS

Computes spectral shape descriptors and mel-frequency band energies, with optional PCA reduction of the feature set. Supports Realtime (per-frame, own FFT) and Batch (full-file offline analysis). Each feature group can be independently toggled.

Input

Audio / File In CHOP (raw audio, both modes)

Output

Realtime: 1 sample per channel at frame rate | Batch: N samples (one per analysis frame). Channels: mfcc0–mfcc12, spectral_centroid, spectral_flux, spectral_rolloff, spectral_contrast0–spectral_contrast5, hfc, spectral_complexity, mel0–melN, pc0–pcN (if PCA enabled), pc_var0–pcN (if variance output enabled)

Features

MFCC (timbral fingerprint, 13 coefficients) — Centroid (brightness) — Flux (rate of spectral change) — Rolloff (85% energy frequency) — Contrast (peak-valley difference in 6 bands) — HFC (high-frequency content) — Complexity (number of spectral peaks) — Mel Bands (perceptual frequency band energies)

Parameters

Enable MFCC MFCC Count MFCC Low Freq MFCC High Freq Enable Centroid Enable Flux Flux Half Rectify Flux Norm Enable Rolloff Rolloff Cutoff Enable Contrast Contrast Bands Enable HFC HFC Type Enable Complexity Complexity Threshold Enable Mel Bands Mel Bands Count Mel Low Freq Mel High Freq Mel Freq Names Log Mel (dB Scale) FFT Size Window Type Enable PCA PCA Components Mode

Essentia Tonal ANALYSIS

Detects pitch, harmonic content, musical key, dissonance, and inharmonicity. Supports Realtime (per-frame with EMA smoothing) and Batch (full-file with global or windowed key detection). FFT Size defaults to auto: semitone-safe at any sample rate.

Input

Audio / File In CHOP (raw audio, both modes)

Output

Realtime: 1 sample per channel | Batch: N samples per channel. Channels: pitch, pitch_confidence, note_a–note_gs (12 bins with Musical Labels on), key, major_minor, key_strength, dissonance, inharmonicity

Features

Pitch (YinFFT algorithm, Hz + confidence) — HPCP (harmonic pitch class profile / chroma, 12/24/36 bins) — Key (key + major/minor + strength) — Dissonance (sensory roughness) — Inharmonicity (deviation from harmonic series)

Parameters

Pitch Algorithm HPCP Size Enable Pitch Pitch Min Freq Pitch Max Freq Pitch Tolerance Enable HPCP HPCP Harmonics Reference Freq HPCP Non-Linear HPCP Normalized Enable Key Key Profile Key Frames Peak Threshold Peak Max Freq Enable Dissonance Enable Inharmonicity Musical Labels Enable Pitch Note Smoothing Mode

Essentia Rhythm ANALYSIS

Detects onsets and estimates tempo. Realtime analyzes its own short window (default 1024) with TempoTapDegara BPM and Onsets-style adaptive thresholding. Batch uses RhythmExtractor2013 for full-file BPM and beat tracking.

Input

Audio / File In CHOP (raw audio, both modes)

Output

Realtime: 1 sample per channel | Batch: N samples per channel. 6 channels: onset (0/1 trigger), onset_strength, bpm, beat (0/1 trigger), beat_phase (0–1 sawtooth), beat_confidence

Processing

RT: OnsetDetection/SuperFlux → Onsets-style adaptive threshold → onset trigger. ODF history → TempoTapDegara → BPM + tick-anchored beat phase. Batch: RhythmExtractor2013 → BPM/beats + per-frame onset detection via Onsets algorithm.

Parameters

Onset Method Onset Sensitivity BPM Min BPM Max Window Size (RT) Window Type Rhythm Method Mode

Essentia Loudness METERING

EBU R128 loudness metering with momentary, short-term, and integrated measurements, plus RMS energy and zero-crossing rate. Supports Realtime (per-frame via ring buffer) and Batch (full-file offline analysis).

Input

Audio CHOP (mono) — both modes take raw audio directly

Output

Realtime: 1 sample per channel | Batch: N samples per channel. 7 channels: loudness (instantaneous dB), loudness_momentary, loudness_shortterm, loudness_integrated, dynamic_range, rms, zcr

Processing

Audio → ring buffer → frame dispatch → Essentia Loudness → dB conversion → sliding windows (momentary 400ms, short-term 3s) → EBU R128 two-pass gating for integrated. RMS and ZCR computed directly from the audio frame.

Parameters

Frame Size ZCR Threshold Mode

ARCHITECTURE

Signal Flow

One topology for both modes: audio wires straight into every operator, and each runs its own FFT internally with the window its algorithm wants. There is no shared FFT node and no ordering constraint between analyzers.

Audio Input

Analyzers (own FFT)

Loudness & Energy

Raw Spectrum Output

Why per-operator FFT — a shared spectrum forces one FFT size onto operators with conflicting needs: Tonal needs 4096 to resolve semitones, but feeding Rhythm a 4096 spectrum instead of 1024 cut onset-detection F1 from 0.47 to 0.13 on a measured 263-onset reference (long windows blunt the transients onset detection keys on). v2.0 removes the conflict — each operator gets the window its algorithm wants, on the same audio wire. The converted Rhythm at window 1024 reproduces the v1.x implementation's onset F1 exactly.

Migrating from v1.x — a Spectrum CHOP wired into Spectral, Tonal, or Rhythm raises an error in both modes ("As of v2.0 this operator analyzes raw audio and computes its own FFT — connect the audio directly"). Delete the Spectrum node from the analysis chain and wire the audio straight in. If you tuned an upstream Spectrum's FFT Size for Tonal, set Tonal's own FFT Size instead (or leave it on auto). Essentia Spectrum remains available as a raw magnitude + phase output.

CREATIVE APPLICATIONS

Output Use Cases

Every output channel has a purpose. This reference describes what each channel measures and suggests practical ways to use it inside TouchDesigner for audio-reactive visuals, installations, and live performance.

Essentia Spectrum OUTPUT

Channel	Range	What it does & how to use it
`spectrum`	0+ (fftSize/2+1 samples)	Raw FFT magnitude bins. Visualize as a bar graph, 3D terrain, or heatmap. Use as input for custom analysis. The lower bins represent bass, upper bins represent treble — slice with a Select CHOP to isolate frequency bands.

Essentia Spectral ANALYSIS

Channel	Range	What it does & how to use it
`mfcc0`–`mfcc12`	~−50 to 50	Timbral fingerprint. MFCCs capture the "texture" of a sound independent of pitch. Use them to cluster similar sounds, drive visual style transitions based on tonal character, or distinguish instruments. Feed into a Math CHOP for normalization, then map to shader uniforms.
`spectral_centroid`	0 – sr/2 Hz	Brightness. The "center of mass" of the spectrum. High values mean bright, shimmery sounds; low values mean warm, bassy tones. Map to color temperature (orange ↔ blue), particle speed, or lighting intensity.
`spectral_flux`	0+	Rate of spectral change. Spikes when the timbre shifts suddenly — a new instrument enters, a filter sweeps, or a transition occurs. Use to trigger visual scene changes, glitch effects, or novelty highlights.
`spectral_rolloff`	0 – sr/2 Hz	Energy distribution edge. The frequency below which 85% of spectral energy lies. Distinguishes bright mixes from dark ones. Control cutoff-style visual filters, blur intensity, or fog density.
`spectral_contrast0`–`5`	unbounded	Per-band tonal vs. noisy content. Six sub-bands measuring peak-to-valley ratio. Drive multi-layer visual intensities — assign each band to a separate geometry layer or glow ring for a multi-band reactive sculpture.
`hfc`	0+	High-frequency content. Sensitive to cymbals, hi-hats, and sibilance. Use for sparkle/shimmer particle effects, percussive high-end triggers, or treble-reactive lighting.
`spectral_complexity`	0+	Number of spectral peaks. Simple tones (sine, flute) score low; complex sounds (orchestra, noise) score high. Drive visual density — particle count, fractal detail, or geometric subdivision level.
`mel0`–`melN`	0+	Perceptual frequency band energies. Mel bands approximate human hearing — evenly spaced in perceived pitch, not linear frequency. Ideal for multi-band audio visualizers, frequency-mapped color gradients, per-band particle emitters, or as input features for ML classifiers.
`pc0`–`pcN`	unbounded	Principal components (PCA). Reduces all enabled spectral features into a compact set of decorrelated dimensions ordered by variance. `pc0` captures the dominant pattern of variation across your features. Map to position, color, or shape parameters for a low-dimensional "summary" of the audio's spectral character. Requires Enable PCA = On.
`pc_var0`–`pcN`	0 – 1	Variance ratios. How much of the total feature variance each principal component explains. Use to auto-weight visual mappings — components with higher variance carry more information. Requires Enable PCA and Variance Output = On.

Essentia Tonal ANALYSIS

Channel	Range	What it does & how to use it
`pitch`	0+ Hz	Fundamental frequency. Detected via YinFFT. Map to note names for generative music notation, control animation speed or vertical position with pitch, or track vocals for lip-sync effects.
`pitch_confidence`	0 – 1	Pitch reliability. Low during noise or silence, high for clear tonal content. Use as a gate — only apply pitch-driven effects when confidence exceeds a threshold. Crossfade between pitched and unpitched visual modes.
`note_a`–`note_gs`	0 – 1	Chroma / pitch class energy. 12 bins (one per semitone, A through G#) with Musical Labels on by default. Bin 0 = A (reference frequency). Visualize as a harmony wheel, map each note to a unique color, detect chord changes for scene transitions, or build a real-time piano-roll display. Set HPCP Size to 24 or 36 for finer resolution.
`key`	0 – 11	Musical key. Encoded integer (0=C, 1=C#, ... 11=B). Assign color palettes or scene themes per key. Drive generative art parameters that shift as the music modulates to a new key.
`major_minor`	0 or 1	Major or minor. 0 = major, 1 = minor. Use for mood — major keys → warm, bright, expansive palettes; minor keys → cool, dark, constrained palettes. Switch between two visual presets.
`key_strength`	0 – 1	Key confidence. Higher when the harmonic content strongly matches a key template. Blend strength into color saturation or confidence-gated transitions.
`dissonance`	0 – 1	Sensory roughness. Measures perceptual dissonance between spectral peaks. Map to visual chaos — glitch intensity, distortion amount, turbulence in fluid simulations, or camera shake.
`inharmonicity`	0 – 1	Deviation from harmonic series. Low for flutes and voices, high for bells and percussion. Distinguish melodic from percussive sources, or drive material textures (metallic vs. organic).

Essentia Rhythm ANALYSIS

Channel	Range	What it does & how to use it
`onset`	0 or 1	Transient trigger. Fires on detected attacks — drum hits, plucks, consonants. Use for particle bursts, flash/strobe triggers, camera cuts, step-sequenced events, or instantiating geometry.
`onset_strength`	0+	Continuous onset function. The raw detection signal before thresholding. Scale burst intensity or particle count by strength for velocity-sensitive triggers. Smooth with a Lag CHOP for an onset envelope.
`bpm`	BPM min – max	Estimated tempo. Sync LFO rates, animation cycle durations, or generative pattern timing to the music. Divide by 60 to get beats-per-second for direct use in Speed parameters.
`beat`	0 or 1	Beat trigger. Fires on the estimated beat grid. Use for rhythmic pulsing, beat-locked scene transitions, synchronized step-sequencing, or quantized color cycling.
`beat_phase`	0 – 1 (sawtooth)	Position within beat. Ramps linearly from 0 to 1 between beats. Feed into easing curves (Lookup CHOP) for smooth beat-synced animation — bouncing, breathing, pendulum swings, or rhythmic camera motion.
`beat_confidence`	0 – 1	Beat tracking reliability. Low during ambient or arrhythmic passages, high during steady beats. Use to fade in beat-synced effects only when tracking is stable, or crossfade to an onset-only mode when confidence drops.

Essentia Loudness METERING

Channel	Range	What it does & how to use it
`loudness`	dB (~−100 to 0)	Instantaneous perceived loudness. The fastest-responding level signal. Use for frame-level reactive scaling — VU-meter needle, geometry size, or opacity. Responds immediately to transients.
`loudness_momentary`	dB	EBU R128 momentary (400 ms). A smoothed level that tracks phrase-level dynamics without jitter. Use for dynamic scaling of visual elements, responsive but stable brightness, or gain-riding effects.
`loudness_shortterm`	dB	EBU R128 short-term (3 s). Captures the current "energy zone" of the track. Use for scene-level intensity — ambient lighting adjustments, macro-level background color, or fog density that follows the song's structure.
`loudness_integrated`	dB	Running gated average. EBU R128 integrated loudness with absolute + relative gating. Use for overall show level monitoring, normalization reference, or auto-gain to keep visuals consistent across tracks of different loudness.
`dynamic_range`	dB (0+)	Loudness swing over 3 s. Peak-to-valley of the short-term window. Detects builds and drops — high values mean the audio is moving between quiet and loud. Drive contrast-based transitions, tension/release mapping, or drop-triggered explosions.
`rms`	0 – 1	Root mean square energy. The classic "audio reactive" signal — simple, linear amplitude. Map directly to geometry scale, opacity, displacement amount, or any parameter that should pulse with the music's energy.
`zcr`	0 – 1	Zero-crossing rate. Measures how "noisy" vs. "tonal" the audio is. High ZCR = noise, percussion, sibilance. Low ZCR = clean pitched tones. Use to drive grain/static effects, distinguish drums from melody, or control visual roughness.

CONFIGURATION

Parameters Reference

Essentia Spectrum
Parameter	Type	Default	Options / Range	Description
FFT Size	Menu	1024	512 / 1024 / 2048 / 4096 / 8192 / 16384	Window size for FFT computation
Hop Size	Int	512	64 – 16384	Samples between analysis frames
Window Type	Menu	Blackman-Harris 62	Hann / Hamming / Triangular / Blackman-Harris 62/70/74/92	Window function applied before FFT
Zero Padding	Menu	None	None / Half FFT / Full FFT	Interpolates the spectrum for better frequency resolution

FFT Size & Quality Tradeoff

Larger FFT sizes improve frequency resolution (more bins, better at distinguishing close pitches) at the cost of time resolution (each frame covers more audio, smearing transients). This is exactly why each analyzer runs its own FFT: Tonal defaults to a large window (auto: 4096+), Rhythm to a short one (1024). For Spectrum itself the default 1024 is a good general-purpose balance; go bigger only when you need finer bins.

Window Type

Each window trades main-lobe width for side-lobe suppression. The default Blackman-Harris 62 follows Essentia's recommended analysis configuration; Hann is the lighter classic alternative. The higher Blackman-Harris variants offer progressively stronger side-lobe suppression (70/74/92 dB) at the cost of wider main lobes.

Zero Padding

Appends zeros to the windowed frame before FFT, which interpolates spectral bins without changing frequency resolution. This improves the accuracy of peak-based descriptors (centroid, rolloff, pitch) and produces smoother spectrum plots. “Half FFT” adds fftSize/2 zeros; “Full FFT” doubles the frame.

Essentia Spectral
Parameter	Type	Default	Description
Mode	Menu	Realtime	Realtime (per-frame from raw audio, own FFT) / Batch (full-file offline)
FFT Size	Menu	2048	Analysis window — applies in both modes (page Analysis)
Hop Size (Batch)	Int	1024	Samples between analysis frames — Batch only (Realtime analyzes the latest window once per cook)
Window Type	Menu	Blackman-Harris 62	Window function — applies in both modes
Compute (Batch)	Pulse	—	Trigger batch analysis — only visible in Batch mode
Autocompute (Batch)	Toggle	Off	Auto-recompute when input changes — only visible in Batch mode
Enable MFCC	Toggle	On	Enable/disable MFCC output channels
MFCC Count	Int	13	Number of MFCC coefficients (1–20)
MFCC Low Freq	Float	0 Hz	Lower frequency bound for MFCC mel filters
MFCC High Freq	Float	11000 Hz	Upper frequency bound for MFCC mel filters
Enable Centroid	Toggle	On	Enable spectral centroid
Enable Flux	Toggle	Off	Enable spectral flux
Flux Half Rectify	Toggle	Off	Only count energy increases (onset emphasis)
Flux Norm	Menu	L2	L1 or L2 norm for difference computation
Enable Rolloff	Toggle	Off	Enable spectral rolloff
Rolloff Cutoff	Float	0.85	Energy fraction threshold (0.5 = median, 0.85 = standard, 0.95 = brightness)
Enable Contrast	Toggle	Off	Enable spectral contrast
Contrast Bands	Menu	6	Number of octave sub-bands (4 / 6 / 8)
Enable HFC	Toggle	On	Enable high-frequency content
HFC Type	Menu	Masri	Masri / Jensen / Brossier — different HFC formulations
Enable Complexity	Toggle	On	Enable spectral complexity
Complexity Threshold	Float	0.005	Minimum peak magnitude to count (0–0.1)
Enable Mel Bands	Toggle	On	Enable mel band output channels
Mel Bands Count	Menu	40	24 / 40 / 60 / 80 / 128
Mel Low Freq	Float	0 Hz	Lower frequency bound for mel filters
Mel High Freq	Float	22050 Hz	Upper frequency bound for mel filters
Mel Freq Names	Toggle	On	Include frequency ranges in channel names
Log Mel (dB Scale)	Toggle	Off	Convert mel band output to dB scale
PCA PAGE
Enable PCA	Toggle	Off	Enable PCA dimensionality reduction — projects all enabled spectral features into a compact set of principal components
Components	Int	3	Number of principal components to output (2–16). Clamped to the number of enabled spectral channels
Window Size (RT)	Menu	512	Circular buffer size for covariance estimation (128 / 256 / 512 / 1024 / 2048 / 4096). Larger = more stable, smaller = more responsive
Update Rate (RT)	Int	1	How often to recompute eigenvectors per second (1–60 Hz). Lower values reduce CPU load
Variance Output	Toggle	Off	Append `pc_var0`–`pcN` channels showing the variance ratio explained by each component

PCA — Dimensionality Reduction

PCA reduces many correlated spectral features (MFCCs, mel bands, centroid, etc.) into a few decorrelated dimensions that capture the most variance. With 3 components you get a compact "spectral signature" suitable for driving 3D position, color (RGB), or clustering. Realtime mode uses a sliding window with throttled updates for stability; Batch mode computes PCA over the entire file. Sign-flip correction ensures temporal coherence between recomputations.

MFCC Frequency Bounds

The default 0–11000 Hz covers the full speech/music range. For voice-only analysis, narrow to 80–3400 Hz to exclude sub-bass and high-frequency noise. For full-band analysis, set High Freq to the Nyquist (sampleRate/2).

HFC Type

Masri weights by energy×frequency (default), Jensen by amplitude×frequency² (stronger high-frequency emphasis), Brossier by amplitude×frequency (linear). Jensen and Brossier respond more aggressively to transients in the upper spectrum.

Essentia Tonal
Parameter	Type	Default	Description
Mode	Menu	Realtime	Realtime (per-frame from raw audio, own FFT) / Batch (full-file offline)
FFT Size	Menu	auto	Both modes. Auto picks the smallest semitone-safe window for the incoming sample rate (4096 at 44.1/48 kHz, 8192 at 88.2/96 kHz); explicit smaller sizes warn
Hop Size (Batch)	Int	2048	Samples between analysis frames — Batch only
Compute (Batch)	Pulse	—	Trigger batch analysis
Autocompute (Batch)	Toggle	Off	Auto-recompute when input changes
Pitch Algorithm	Menu	YinFFT	YinFFT
HPCP Size	Menu	12	12 (default, 1 per semitone) / 24 / 36 bins
Enable Pitch	Toggle	On	Enable pitch detection
Pitch Min Freq	Float	20 Hz	Minimum detectable frequency (constrain to instrument range)
Pitch Max Freq	Float	22050 Hz	Maximum detectable frequency
Pitch Tolerance	Float	1.0	Peak detection strictness (lower = fewer octave errors, more unvoiced frames)
Enable HPCP	Toggle	On	Enable chroma output
HPCP Harmonics	Int	0	Harmonic contributions (0 = fundamental only, 3–5 for harmonic instruments)
Reference Freq	Float	440 Hz	Tuning reference (415 = Baroque, 432 = alternative, 440 = standard)
HPCP Non-Linear	Toggle	Off	Apply peak-sharpening post-processing
HPCP Normalized	Menu	Unit Max	Unit Max / Unit Sum / None
Enable Key	Toggle	On	Enable key detection
Key Frames (RT)	Int	8	HPCP frames to average for key detection (1–300) — Realtime only
Key Profile	Menu	Bgate	Bgate (default) / Temperley / Krumhansl / EDMA / Diatonic / Gomez
Peak Threshold	Float	0.00001	Minimum spectral peak magnitude — filters noise-floor peaks from HPCP/Key/Dissonance/Inharmonicity
Peak Max Freq	Float	3500 Hz	Upper frequency limit for spectral peak detection — tonal content lives below 3500 Hz
Enable Dissonance	Toggle	On	Enable dissonance output
Enable Inharmonicity	Toggle	On	Enable inharmonicity output
Musical Labels	Toggle	On	Use note names (A through G#) instead of indices for HPCP channels
Enable Pitch Note	Toggle	Off	Output pitch-to-note-class channel
Smoothing (RT)	Float	0.5	EMA smoothing coefficient (0 = none, 1 = maximum) — Realtime only
Key Mode (Batch)	Menu	Global	Global (single key for whole file) / Windowed (per-frame key) — Batch only
Key Window Size (Batch)	Int	8	HPCP frames for windowed key detection — Batch only, visible when Key Mode = Windowed

Recommended Settings

The defaults follow Essentia’s MusicExtractor recommendations. Here are per-use-case tweaks:

FFT Size is a correctness setting for key, not a latency knob. The spectrum has to separate adjacent semitones, so the requirement scales with sample rate: 4096 at 44.1/48 kHz, 8192 at 96 kHz. Auto picks the right one. Below it the key goes wrong with a confident-looking strength (measured on one track at otherwise identical settings: 4096 correct, 2048 a fifth away, 1024 elsewhere again, in both Realtime and Batch). The operator warns when you cross that line, so if you see “Spectrum too coarse for key/HPCP”, raise FFT Size rather than trusting the reading.

Key Detection (best accuracy): the defaults are already the accurate setting — the key stage runs its own chroma pinned to Essentia’s KeyExtractor configuration, so HPCP Size, HPCP Harmonics and HPCP Non-Linear do not affect the key output. Worth tuning: Key Profile (Bgate by default, matching Essentia), Key Frames = 30–60 in Realtime so the window spans whole chords rather than a fraction of one. For batch, use Global key mode unless tracking modulations.

Pitch Tracking: Pitch Algorithm = YinFFT (default). Constrain frequency range to your source — voice 80–800 Hz, guitar 80–1200 Hz, bass 30–300 Hz. Smoothing = 0.3 for responsiveness.

Chord / Harmony Analysis: HPCP Size = 36, HPCP Harmonics = 8, HPCP Non-Linear = On, Key Profile = EDMA (electronic/dance) or Temperley (pop/rock).

Live Visuals (fast RT): HPCP Size = 12 (default), Musical Labels = On (default), Smoothing = 0.5, Key Frames = 8 (default). Enable Pitch Note for a 0–11 note class output.

Key Profile

Different profiles are tuned for different genres. Bgate (default) is Essentia’s own default and the profile its KeyExtractor uses. Temperley is a well-tested general-purpose alternative. Krumhansl is the classical music theory standard. EDMA is designed for electronic/dance music. Diatonic is the simplest model. Gomez is optimized for guitar-heavy material.

Pitch Frequency Range

Constraining to instrument-appropriate bands eliminates octave errors. Common ranges: guitar 80–1200 Hz, voice 80–800 Hz, bass 30–300 Hz.

HPCP Harmonics

When set to 0 (default), only the fundamental contributes to chroma. Setting to 3–5 makes HPCP more robust for harmonic instruments (piano, guitar, voice) where overtones reinforce the pitch class. This affects the hpcp_* output channels only. Key detection is unaffected: it runs its own chroma fixed at 4 harmonics, matching Essentia’s KeyExtractor.

Peak Threshold & Max Freq

The default threshold (0.00001) and max frequency (3500 Hz) match Essentia’s MusicExtractor. These filter noise-floor peaks and limit analysis to the tonal range where pitch classes are meaningful. Increase threshold for noisier signals; raise max frequency above 3500 only for very high-pitched instruments.

Essentia Rhythm
Parameter	Type	Default	Description
Mode	Menu	Realtime	Realtime (per-frame from raw audio, own FFT) / Batch (full-file offline)
Window Size (RT)	Menu	1024	Realtime analysis window (512 / 1024 / 2048 / 4096). 1024 is the measured best default; the operator warns if the window is shorter than the per-cook audio chunk
Window Type	Menu	Blackman-Harris 62	Window function — applies in both modes
FFT Size (Batch)	Menu	2048	Window size for batch FFT — Batch only
Hop Size (Batch)	Int	512	Samples between analysis frames (~11.6 ms at 44100 Hz) — Batch only
Compute (Batch)	Pulse	—	Trigger batch analysis
Autocompute (Batch)	Toggle	Off	Auto-recompute when input changes
Onset Method	Menu	Complex	HFC / Complex / Flux / Mel Flux / RMS / SuperFlux
Onset Sensitivity	Float	0.5	0.0 (rare triggers) – 1.0 (frequent). In batch mode, maps to Onsets algorithm alpha and silence threshold
BPM Min / Max	Int	60 / 180	BPM search range (internally clamped to [40,180] / [60,250] for Essentia algorithms)
Rhythm Method (Batch)	Menu	Degara	Degara / Multi-Feature — RhythmExtractor2013 algorithm variant, Batch only

Onset Method

Complex (default) uses both magnitude and phase for the most accurate general-purpose onset detection. HFC emphasizes high-frequency transients, good for percussive material. Flux measures overall spectral change. Mel Flux applies mel-weighted spectral difference — more robust for harmonic/melodic content. RMS uses simple energy change — fast and reliable for broadband signals. SuperFlux uses TriangularBands + SuperFluxNovelty for music with soft/gradual onsets.

Beat Detection

Realtime: BPM estimation uses Essentia’s TempoTapDegara algorithm, run periodically (~1.5 s) on accumulated onset detection history. BPM is derived from median tick intervals; beat phase is anchored to tick positions for audio-synchronized animation. Batch: RhythmExtractor2013, which requires 44.1 kHz input — non-44.1 kHz audio is resampled internally (see the sample-rate callout below).

Best Quality Settings

Realtime: Onset Method = Complex, Window Size = 1024 (both defaults; 1024 was validated by a measured onset-F1 window sweep). Batch: Onset Method = Complex, Rhythm Method = Degara, FFT 2048, Hop 512 (all defaults). For both modes: set the narrowest BPM range that covers your material (e.g., 100–140 for house, 80–160 for pop). Use SuperFlux for music with soft/gradual onsets.

Batch BPM and Sample Rate

RhythmExtractor2013 requires 44.1 kHz input. Non-44.1 kHz audio is resampled internally (libsamplerate) before BPM extraction — verified against a native 44.1 kHz analysis of the same material, agreeing to two decimals. Onset detection is frame-based and rate-aware, so it never needed resampling. If you have batch BPM numbers from v1.1.8: that release shipped without the resampler, so BPM on non-44.1 kHz files came out scaled by roughly rate/44100 (about 9% high on 48 kHz files). Expect v2.0 to report different, correct values on the same files.

Analysis Window vs Cook Rate

Realtime Rhythm analyzes the latest Window Size samples once per cook (512–4096, default 1024). Two measured facts guide the choice: 1024 is the validated default (onset F1 0.461 on a 262-onset reference; 4096 collapses to 0.028 because long windows smear onset timing), and if the per-cook audio chunk exceeds the window, the excess is never analyzed — at 30 fps / 48 kHz a 1024 window skips 36% of the audio and detection degrades sharply. The operator warns whenever window < sampleRate / fps and the fix is either direction: raise Window Size or raise TD’s fps.

Essentia Loudness
Parameter	Type	Default	Description
Mode	Menu	Realtime	Realtime (per-frame via ring buffer) / Batch (full-file offline)
Compute (Batch)	Pulse	—	Trigger batch analysis
Autocompute (Batch)	Toggle	Off	Auto-recompute when input changes
Frame Size	Menu	1024	512 / 1024 / 2048
ZCR Threshold	Float	0	Dead-band around zero for ZCR (0–0.1). Increase to filter noise-floor chatter on quiet signals

BUILD FROM SOURCE

Build & Install

Prerequisites
Essentia static library (essentia.lib, MSVC x64) — see building-essentia.md
TouchDesigner C++ SDK headers (CHOP_CPlusPlusBase.h, CPlusPlus_Common.h)

Build

cd src

cmake -B build -G "Visual Studio 17 2022" -A x64

cmake --build build --config Release

Deploy

Copy all 5 DLLs from src/build/Release/ to your TouchDesigner Plugins folder:

C:/Users/<user>/Documents/Derivative/Plugins/Essentia/

Note — All operators appear in TouchDesigner's OP Create Dialog under their registered names. Restart TouchDesigner after copying new DLLs.