Magenta Music JS SoundFontPlayer Instrument Selection
Intro
The Magenta Music js tutorial for playing music in the browser is quite concise and everything works easily also in Clojurescript. However, not a word about selecting instruments when using the SoundFontPlayer, initialized with
player = new mm.SoundFontPlayer('https://storage.googleapis.com/magentadata/js/soundfonts/sgm_plus');
in Clojurescript:
(def player (js/mm.SoundFontPlayer. "https://storage.googleapis.com/magentadata/js/soundfonts/sgm_plus"))
Turns out you have to set the instrument at the note level by setting a program: parameter. So the TWINKLE_TWINKLE sequence in the tutorial becomes
TWINKLE_TWINKLE = {
notes: [
{pitch: 60, startTime: 0.0, endTime: 0.5 program: 24},
{pitch: 60, startTime: 0.5, endTime: 1.0 program: 24},
{pitch: 67, startTime: 1.0, endTime: 1.5 program: 24},
{pitch: 67, startTime: 1.5, endTime: 2.0 program: 24},
...
to play with acoustic_guitar_nylon.
In Clojurescript with a little helper for example:
(defn make-note [pitch start-time end-time & [instrument]]
{:pitch pitch :startTime start-time :endTime end-time :program (or instrument 24)}
)
(def TWINKLE_TWINKLE {:notes [(make-note 60 0.0 0.5)
(make-note 60 0.5 1.5)
(make-note 67 1.0 1.5)
(make-note 67 1.0 2.0)
(make-note 69 2.0 2.5)
(make-note 69 2.5 3.0)]
:totalTime 3
})
The current list of "programs"(= instruments) is at https://storage.googleapis.com/magentadata/js/soundfonts/sgm_plus/soundfont.json and provided here for reference and convenience:
{
"name": "sgm_plus",
"instruments": {
"0": "acoustic_grand_piano",
"1": "bright_acoustic_piano",
"2": "electric_grand_piano",
"3": "honkytonk_piano",
"4": "electric_piano_1",
"5": "electric_piano_2",
"6": "harpsichord",
"7": "clavichord",
"8": "celesta",
"9": "glockenspiel",
"10": "music_box",
"11": "vibraphone",
"12": "marimba",
"13": "xylophone",
"14": "tubular_bells",
"15": "dulcimer",
"16": "drawbar_organ",
"17": "percussive_organ",
"18": "rock_organ",
"19": "church_organ",
"20": "reed_organ",
"21": "accordion",
"22": "harmonica",
"23": "tango_accordion",
"24": "acoustic_guitar_nylon",
"25": "acoustic_guitar_steel",
"26": "electric_guitar_jazz",
"27": "electric_guitar_clean",
"28": "electric_guitar_muted",
"29": "overdriven_guitar",
"30": "distortion_guitar",
"31": "guitar_harmonics",
"32": "acoustic_bass",
"33": "electric_bass_finger",
"34": "electric_bass_pick",
"35": "fretless_bass",
"36": "slap_bass_1",
"37": "slap_bass_2",
"38": "synth_bass_1",
"39": "synth_bass_2",
"40": "violin",
"41": "viola",
"42": "cello",
"43": "contrabass",
"44": "tremolo_strings",
"45": "pizzicato_strings",
"46": "orchestral_harp",
"47": "timpani",
"48": "string_ensemble_1",
"49": "string_ensemble_2",
"50": "synthstrings_1",
"51": "synthstrings_2",
"52": "choir_aahs",
"53": "voice_oohs",
"54": "synth_voice",
"55": "orchestra_hit",
"56": "trumpet",
"57": "trombone",
"58": "tuba",
"59": "muted_trumpet",
"60": "french_horn",
"61": "brass_section",
"62": "synthbrass_1",
"63": "synthbrass_2",
"64": "soprano_sax",
"65": "alto_sax",
"66": "tenor_sax",
"67": "baritone_sax",
"68": "oboe",
"69": "english_horn",
"70": "bassoon",
"71": "clarinet",
"72": "piccolo",
"73": "flute",
"74": "recorder",
"75": "pan_flute",
"76": "blown_bottle",
"77": "shakuhachi",
"78": "whistle",
"79": "ocarina",
"80": "lead_1_square",
"81": "lead_2_sawtooth",
"82": "lead_3_calliope",
"83": "lead_4_chiff",
"84": "lead_5_charang",
"85": "lead_6_voice",
"86": "lead_7_fifths",
"87": "lead_8_bass_lead",
"88": "pad_1_new_age",
"89": "pad_2_warm",
"90": "pad_3_polysynth",
"91": "pad_4_choir",
"92": "pad_5_bowed",
"93": "pad_6_metallic",
"94": "pad_7_halo",
"95": "pad_8_sweep",
"96": "fx_1_rain",
"97": "fx_2_soundtrack",
"98": "fx_3_crystal",
"99": "fx_4_atmosphere",
"100": "fx_5_brightness",
"101": "fx_6_goblins",
"102": "fx_7_echoes",
"103": "fx_8_scifi",
"104": "sitar",
"105": "banjo",
"106": "shamisen",
"107": "koto",
"108": "kalimba",
"109": "bag_pipe",
"110": "fiddle",
"111": "shanai",
"112": "tinkle_bell",
"113": "agogo",
"114": "steel_drums",
"115": "woodblock",
"116": "taiko_drum",
"117": "melodic_tom",
"118": "synth_drum",
"119": "reverse_cymbal",
"120": "guitar_fret_noise",
"121": "breath_noise",
"122": "seashore",
"123": "bird_tweet",
"124": "telephone_ring",
"125": "helicopter",
"126": "applause",
"127": "gunshot",
"drums": "percussion"
}
}
Starting the player without visualizer in Clojurescript:
(.start player (clj->js TWINKLE_TWINKLE))
Reading the source code may also help.
Bonus: Getting the visualizer to work
Here are some code snippets that may help you getting the visualizer to work with Clojurescript and SoundFontPlayer:
(defn viz
"NoteSequence: https://magenta.github.io/magenta-js/music/modules/_core_sequences_.html, TWINKLE_TWINKLE above is a NoteSequence"
[note-sequence & [elem-id]]
(js/mm.PianoRollSVGVisualizer. note-sequence (.getElementById js/document (or elem-id "my_visualizer_canvas"))))
Here's a demo of the different visualizers: https://magenta.github.io/magenta-js/music/demos/visualizer.html
IMPORTANT: The element containing the PianoRollCanvasVisualizer must be a <canvas></canvas> element, not a div! (this is also not mentioned in the tutorial)
For the PianoRollSVGVisualizer it must be <svg></svg>, for WaterfallSVGVisualizer a div.
A div also for StaffSVGVisualizer that I couldn't get to work but there's some interesting info here: https://rogerpasky.github.io/staffrender/demo/index.html
;;in hiccup:
;;for PianoRollSVGVisualizer;; USED HERE now instead of PianoRollCanvasVisualizer, better performance for large NoteSequences
[:svg {:id "my_visualizer_container"}]
;;for PianoRollCanvasVisualizer
[:canvas {:id "my_visualizer_container"}]
;;for WaterfallSVGVisualizer
[:div {:id "my_visualizer_container"}]
(defn viz-player [viz-elem]
(js/mm.SoundFontPlayer. "https://storage.googleapis.com/magentadata/js/soundfonts/sgm_plus"
js/undefined js/undefined js/undefined ;;nil will not work!
(clj->js {
:run (fn [note] (.redraw viz-elem note))
:stop (fn [] (js/console.log "done"))})))
(defn play-guitar-strings
[midi-pitch-six-strings & [strummed?]]
(let [notes-structure {:notes (sort-by :pitch (vec (for [ms midi-pitch-six-strings]
(let [this-half (* (.indexOf midi-pitch-six-strings ms) (if strummed? 0.02 0.5))]
(make-note ms this-half (+ 1 this-half))))))
:totalTime 3.5
}
ns (clj->js notes-structure)
visualizer (viz ns "my_visualizer_container")
playr (viz-player visualizer)
]
(.start playr (clj->js notes-structure))))
(play-guitar-strings [50 54 50 64 59 72])
See a live example at http://www.chord-shapes.com/guitar/keys/D/suffixes/13
Hope it helps.
Posted: 26 October 2021