Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ freesound-data
*.DS_Store
freesound/local_settings.py
_docs/api/build/
_docs/api/descriptors/
.idea/
.env
.venv
Expand Down
19 changes: 18 additions & 1 deletion _docs/api/generate_analysis_documentation/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,25 @@
# Audio Descriptor Analysis Documentation

These scripts regenerate the audio-descriptor documentation and its plots.
Run them from inside this folder.

Everything starts from **`descriptors.csv`** — one row per descriptor.
Edit it whenever the descriptors change. The distribution plots also read the per-descriptor value
dumps from `../audio_descriptors_values/`.

### The scripts

| Script | What it does |
|---|---|
| `generate_plots.py` | Builds a distribution plot per descriptor, saved directly into `../source/_static/descriptors/` (where the per-descriptor docs pick them up automatically) |
| `generate_resources_descriptor_table.py` | Prints the descriptor table for the Sound Instance resource (paste into `../source/resources.rst`) |
| `generate_analysis_rst.py` | Prints the full analysis docs page code (and handles the plot display) |

# Instructions for updating descriptors documentation

1) Update `descriptors.csv``
2) Run `python generate_resources_descriptor_table.py` and copy the printed table to the corresponding place in ` ../source/resources.rst`
3) Run `python generate_analysis_rst.py > ../source/analysis_docs.rst` to update the analysis documentation page with new descriptors. This will add distribution images to the docs if these are present in the `../source/_static/descriptors/` folder. The first time a descriptor is deployed, it might need to be first added without a distribution image. When the descriptor is deployed in Freesound, then a command can be run to extract its values and generate the images. See next steps...
4) To create images for new descriptors (or update existing ones), the descriptor needs to be deployed in Freesound, and the its data exported to a JSON file with the command `export_audio_descriptor_stats --names xxx`. Then copy the resulting JSON file(s) here, and run the script `generate_plots.py` which will place new images in the correct location. Finally, re-run step 3) to update the docs. You can copy the descriptors with a command similar to `rsync -avz fs-cdn:/mnt/data/fsweb/audio_descriptors_values /home/ffont/Freesound/freesound/_docs/api/generate_analysis_documentation`

Note that to run `generate_plots.py` you'll need a python environment with 'seaborn', 'numpy' and 'matplotlib'
Note that to run `generate_plots.py` you'll need a python environment with 'seaborn', 'numpy' and 'matplotlib'
18 changes: 9 additions & 9 deletions _docs/api/generate_analysis_documentation/descriptors.csv
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ beat_count,rhythm.beats_count,-,integer,,,,,"Number of beats in the audio signal
beat_loudness,rhythm.beats_loudness,mean,numeric,advanced,,recheck terminology,,Spectral energy measured at the beat positions of the audio signal. It measures the strength of the rhythmic beats of the sound.,https://essentia.upf.edu/reference/streaming_BeatsLoudness.html
beat_times,rhythm.beats_position,VL,array[numeric],,,recheck it is sec,,"Beat timestamps (in seconds) for the audio signal, which can vary according to the amount (count) of beats identified in the audio.",http://essentia.upf.edu/documentation/reference/streaming_RhythmExtractor2013.html
boominess,ac_boominess,-,numeric,,,,0-100,"Boominess of the audio signal. A boomy sound is one that conveys a sense of loudness, depth and resonance.",
bpm,ac_tempo,-,integer,,rhythm.bpm,,,BPM value estimated by beat tracking algorithm.,https://en.wikipedia.org/wiki/Tempo
bpm,ac_tempo,-,integer,,rhythm.bpm,,,"BPM value of the audio, estimated by a beat tracking algorithm.",https://en.wikipedia.org/wiki/Tempo
bpm_confidence,ac_tempo_confidence,-,numeric,,,,0-1,Confidence score on how reliable the tempo (BPM) estimation is.,
brightness,ac_brightness,-,numeric,,,,0-100,Brightness of the audio signal. A bright sound is one that is clear/vibrant and/or contains significant high-pitched elements.,
decay_strength,sfx.strongdecay,-,numeric,advanced,,,,Rate at which the audio signal's energy decays (i.e. how quickly it decreases) after the initial attack. It is computed from a non-linear combination of the signal's energy and its temporal centroid (the balance point of the signal's absolute amplitude).,https://essentia.upf.edu/reference/streaming_StrongDecay.html
Expand All @@ -28,12 +28,12 @@ note_midi,ac_note_midi,-,integer,,,set to 0 if not note,,MIDI value correspondin
note_name,ac_note_name,-,string,,,,,"Pitch note name that includes one of the 12 western notes [""A"", ""A#"", ""B"", ""C"", ""C#"", ""D"", ""D#"", ""E"", ""F"", ""F#"", ""G"", ""G#""] and the octave number, e.g. ""A4"", ""E#7"". It is computed by the median of the estimated fundamental frequency.",
onset_count,rhythm.onset_count,-,integer,,,,,Number of detected onsets in the audio signal.,http://essentia.upf.edu/documentation/reference/streaming_OnsetRate.html
onset_times,rhythm.onset_times,VL,array[numeric],,,recheck it is sec,,"Timestamps for the detected onsets in the audio signal in seconds, which can vary according to the amount of onsets (computed by the onset_count descriptor).",http://essentia.upf.edu/documentation/reference/streaming_OnsetRate.html
pitch,lowlevel.pitch,mean,numeric,,"ac_note_frequency, sfx.pitch_max_to_total, sfx.pitch_min_to_total",,0-25000,"Mean (average) fundamental frequency derived from the audio signal, computed with the YinFFT algorithm.",http://essentia.upf.edu/documentation/reference/streaming_PitchYin.html
pitch_max,lowlevel.pitch,max,numeric,advanced,,,,Maximum fundamental frequency observed throughout the audio signal.,http://essentia.upf.edu/documentation/reference/streaming_PitchYin.html
pitch_min,lowlevel.pitch,min,numeric,advanced,,,,Minimum fundamental frequency observed throughout the audio signal.,http://essentia.upf.edu/documentation/reference/streaming_PitchYin.html
pitch_salience,lowlevel.pitch_salience,mean,numeric,,lowlevel.pitch_salience,,0-1,Pitch salience (i.e. tone sensation) given by the ratio of the highest auto correlation value of the spectrum to the non-shifted auto correlation value. Unpitched sounds and pure tones have value close to 0.,https://essentia.upf.edu/reference/streaming_PitchSalience.html
pitch,lowlevel.pitch,mean,numeric,,"ac_note_frequency, sfx.pitch_max_to_total, sfx.pitch_min_to_total",,0-25000,"Mean (average) fundamental frequency derived from the audio signal, computed with the YinFFT algorithm.",http://essentia.upf.edu/documentation/reference/streaming_PitchYinFFT.html
pitch_max,lowlevel.pitch,max,numeric,advanced,,,,Maximum fundamental frequency observed throughout the audio signal.,http://essentia.upf.edu/documentation/reference/streaming_PitchYinFFT.html
pitch_min,lowlevel.pitch,min,numeric,advanced,,,,Minimum fundamental frequency observed throughout the audio signal.,http://essentia.upf.edu/documentation/reference/streaming_PitchYinFFT.html
pitch_confidence,lowlevel.pitch_confidence,mean,numeric,,lowlevel.pitch_instantaneous_confidence,,0-1,"A measure of the confidence with which the pitch was estimated. If the output is near 1, there exist just one pitch in the mixture, an output near 0 indicates multiple, not distinguishable pitches.",https://essentia.upf.edu/reference/std_PitchYin.html
pitch_var,lowlevel.pitch,var,numeric,advanced,,,,Variance of the fundamental frequency of the audio signal.,http://essentia.upf.edu/documentation/reference/streaming_PitchYin.html
pitch_salience,lowlevel.pitch_salience,mean,numeric,,lowlevel.pitch_instantaneous_confidence,,0-1,Pitch salience (i.e. tone sensation) given by the ratio of the highest auto correlation value of the spectrum to the non-shifted auto correlation value. Unpitched sounds and pure tones have value close to 0.,http://essentia.upf.edu/documentation/reference/streaming_PitchYinFFT.html
pitch_var,lowlevel.pitch,var,numeric,advanced,,,,Variance of the fundamental frequency of the audio signal.,"http://essentia.upf.edu/documentation/reference/streaming_PitchYinFFT.html, https://en.wikipedia.org/wiki/Variance"
reverbness,ac_reverb,-,boolean,,,,,Whether the signal is reverberated or not.,https://en.wikipedia.org/wiki/Reverberation
roughness,ac_roughness,-,numeric,,,,0-100,Roughness of the audio signal. A rough sound is one that has an uneven or irregular sonic texture.,
sharpness,ac_sharpness,-,numeric,,,,0-100,Sharpness of the audio signal. A sharp sound is one that suggests it might cut if it were to take on physical form.,
Expand All @@ -46,13 +46,13 @@ spectral_energy,lowlevel.spectral_energy,mean,numeric,advanced,,,,Energy in the
spectral_entropy,lowlevel.spectral_entropy,mean,numeric,advanced,,,,"Shannon entropy in the frequency domain of the audio signal, measuring the unpredictability in the spectrum.",http://essentia.upf.edu/documentation/reference/streaming_Entropy.html
spectral_flatness,lowlevel.spectral_flatness_db,mean,numeric,advanced,,,0-1,"Flatness of the spectrum measured as the ratio of its geometric mean to its arithmetic mean (in dB). High values indicate a noise-like, flat spectrum with evenly distributed power, while low values indicate a tone-like, spiky spectrum with power concentrated in a few frequency bands.",http://essentia.upf.edu/documentation/reference/streaming_FlatnessDB.html
spectral_rolloff,lowlevel.spectral_rolloff,mean,numeric,advanced,,,0-25000,"Roll-off frequency of the spectrum, defined as the frequency under which some percentage (cutoff) of the total energy of the spectrum is contained. It can be used to distinguish between harmonic (below roll-off) and noisy sounds (above roll-off).",http://essentia.upf.edu/documentation/reference/streaming_RollOff.html
spectral_skewness,lowlevel.spectral_skewness,mean,numeric,advanced,lowlevel.barkbands_skewness,,,Skewness of the spectrum given its central moments. It measures how the values of the spectrum are dispersed around the mean and is a key indicator of the distribution's shape.,https://essentia.upf.edu/reference/streaming_CentralMoments.html
spectral_skewness,lowlevel.spectral_skewness,mean,numeric,advanced,lowlevel.barkbands_skewness,,,Skewness of the spectrum given its central moments. It measures how the values of the spectrum are dispersed around the mean and is a key indicator of the distribution's shape.,"https://essentia.upf.edu/reference/streaming_CentralMoments.html, https://en.wikipedia.org/wiki/Skewness"
spectral_spread,lowlevel.spectral_spread,mean,numeric,advanced,lowlevel.barkbands_spread,,,Spread (variance) of the spectrum given its central moments. It measures how the values of the spectrum are dispersed around the mean and is a key indicator of the distribution's shape.,https://essentia.upf.edu/reference/streaming_CentralMoments.html
start_time,lowlevel.startFrame,-,numeric,,,convert to sec,,"The moment at which sound begins in seconds, i.e. when the audio signal first rises above silence.",https://essentia.upf.edu/reference/streaming_StartStopSilence.html
temporal_centroid,ac_temporal_centroid,-,numeric,advanced,sfx.temporal_centroid,,0-1,"Temporal centroid of the audio signal, defined as the time point at which the temporal balancing position of the sound event energy.",http://essentia.upf.edu/documentation/reference/streaming_Centroid.html
temporal_centroid_ratio,sfx.tc_to_total,-,numeric,advanced,,,,"Ratio of the temporal centroid to the total length of the audio signal's envelope, which shows how the sound is balanced'. Values close to 0 indicate most of the energy is concentrated early (decrescendo or impulsive), while values close to 1 indicate energy concentrated late (crescendo).",http://essentia.upf.edu/documentation/reference/streaming_TCToTotal.html
temporal_centroid_ratio,sfx.tc_to_total,-,numeric,advanced,,,,"Ratio of the temporal centroid to the total length of the audio signal's envelope, which shows how the sound is 'balanced'. Values close to 0 indicate most of the energy is concentrated early (decrescendo or impulsive), while values close to 1 indicate energy concentrated late (crescendo).",http://essentia.upf.edu/documentation/reference/streaming_TCToTotal.html
temporal_decrease,sfx.temporal_decrease,mean,numeric,advanced,,,,"Overall decrease of the audio signal's amplitude over time, computed as the linear regression coefficient.",http://essentia.upf.edu/documentation/reference/streaming_Decrease.html
temporal_skewness,sfx.temporal_skewness,mean,numeric,advanced,,,,Skewness of the audio signal in the time domain given its central moments. It measures how the amplitude values of the signal are dispersed around the mean and is a key indicator of the distribution's shape.,https://essentia.upf.edu/reference/streaming_CentralMoments.html
temporal_skewness,sfx.temporal_skewness,mean,numeric,advanced,,,,Skewness of the audio signal in the time domain given its central moments. It measures how the amplitude values of the signal are dispersed around the mean and is a key indicator of the distribution's shape.,"https://essentia.upf.edu/reference/streaming_CentralMoments.html, https://en.wikipedia.org/wiki/Skewness"
temporal_spread,sfx.temporal_spread,mean,numeric,advanced,,,,Spread (variance) of the audio signal in the time domain given its central moments. It measures how the amplitude values of the signal are dispersed around the mean and is a key indicator of the distribution's shape.,https://essentia.upf.edu/reference/streaming_CentralMoments.html
tonality,ac_tonality,-,string,,"tonal.key_key, tonal.key_scale, tonal.chords_key, tonal.chords_scale",,,"Key (tonality) estimated by a key detection algorithm. The key name includes the root note of the scale, which is one of [""A"", ""A#"", ""B"", ""C"", ""C#"", ""D"", ""D#"", ""E"", ""F"", ""F#"", ""G"", ""G#""], and the scale mode, which is one of [""major"", ""minor""], e.g. ""C minor"", ""F# major"".",https://en.wikipedia.org/wiki/Key_(music)
tonality_confidence,ac_tonality_confidence,-,numeric,,tonal.key_strength,,0-1,Confidence score on how reliable the key estimation is (computed by the tonality descriptor).,
Expand Down
3 changes: 2 additions & 1 deletion _docs/api/generate_analysis_documentation/generate_plots.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import seaborn as sns
import numpy as np


DATA_FOLDER = "audio_descriptors_values" # json files for each descriptor extracted from DB
OUT_FOLDER = "../source/_static/descriptors"

Expand Down Expand Up @@ -155,4 +156,4 @@ def thousands_formatter(x, pos):
pool = [v[i] for v in values if len(v) > i]
label = f"{descriptor_name}-{i}"
print(f"\tDim. {i}")
plot_histogram(pool, label, OUT_FOLDER)
plot_histogram(pool, label, OUT_FOLDER)
2 changes: 1 addition & 1 deletion _docs/api/source/resources.rst
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ are marked with "yes" in the ``filtering`` column and are of type ``numeric`` or

.. _search-similar:

The 'similar_to' and 'simiarity_space' parameters
The 'similar_to' and 'similarity_space' parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

These parameters allow similarity-based search in order to retrieve sounds that are acoustically, semantically, or perceptually similar to a given reference sound.
Expand Down