Interaction via MIDI, SoundIn, external controllers

(
Server.default=s=Server.local;
s.boot;
)

MIDI

To access your MIDI devices you first initialise:

MIDIClient.init //should post a list of available devices

There may be more than one source and destination device, each containing different
input and output ports.

To react to incoming MIDI messages, the user sets up callback functions.

MIDIIn.connect(0,MIDIClient.sources[0]) //first number is port number, second is

device from sources list
//MIDIIn.connect //would work on its own but defaults to first port of first device
//MIDIIn.connectAll //connect to all attached input sources

Incoming MIDI messages can be easily handled through some callback functions in
MIDIIn. However, from SuperCollider 3.5, the use of MIDIFunc is much preferred.

First, the old way:

MIDIIn.noteOn= { arg src, chan, num, vel; [chan,num,vel / 127].postln; }; //set

up callback for MIDI Note On message

MIDI messages typically have a 7-bit (2**7) value range, so take on integers from 0
to 127. The vel/127 above converts from this range to a 0.0 to 1.0 range befitting
an amplitude control.

MIDIIn.control = { arg src, chan, num, val; [chan,num,val/127].postln; };

//control change messages have a 7 bit value

MIDIIn.bend = { arg src, chan, bend; [chan,bend/8192].postln; }; //pitch bend

has a 14 bit range and is a bipolar signal (so bend/8192 will remap the range to -
1.0 to 1.0)

See the MIDIIn help file for further message types.

[MIDIIn]

Examples:

//creating Synths with each new note on

(
SynthDef(\sound,{arg freq=440, amp=0.1;
 var saw, filter, env;

saw= Saw.ar(freq);
filter= Resonz.ar(saw,1000,0.1)*amp;
env= EnvGen.ar(Env([0,1,0],[0.01,0.1]),doneAction:2);

//dup(2) duplicates the mono signal onto two channels, giving instant stereo
middle panned output
Out.ar(0,(filter*env).dup(2))
}).add
)

//create one Synth for every new note, Synths will be of finite duration because of
the envelope
MIDIIn.noteOn = { arg src,chan, midinote, velocity; Synth(\sound,[\
freq,midinote.midicps,\amp,velocity/127.0]) };

//turn off again

MIDIIn.noteOn = nil;

Keeping track of active (held-down, sustained) notes can be a chore in MIDI. Here
is an example of doing this using an array with 128 slots, one for each possible
MIDI note.

//note the use of a gate; this will sustain until released

(
SynthDef(\sound,{arg freq=440, amp=0.1, gate=1;
var saw, filter, env;

saw= Saw.ar(freq);
filter= Resonz.ar(saw,1000,0.1)*amp;
env= EnvGen.ar(Env.asr(0.005,1.0,0.1),gate,doneAction:2);

Out.ar(0,(filter*env).dup(2))
}).add
)

(
var activenotes = nil!128; //make Array of 128 slots, initially with nil objects in
to represent nothing
var releasefunction = {|index|

//release existing note if present already

if(activenotes[index].notNil) {
activenotes[index].release; //will send gate=0
activenotes[index] = nil; //make sure now empty slot ready
}
 };

//create one Synth for every new note, with logic to check existing notes (though
not MIDI channel sensitive)
MIDIIn.noteOn = { arg src,chan, midinote, velocity;

"received".postln;

releasefunction.value(midinote);

//put active note in array; function above tidied any existing note on this
key
activenotes[midinote] =
Synth(\sound,[\freq,midinote.midicps,\amp,velocity/127.0]);

};

//must also look for note offs as indicated end of held note
MIDIIn.noteOff = { arg src,chan, midinote, velocity;

releasefunction.value(midinote);

};

//using control change for continuous variation; run one block/line at a time here
//no envelope this time, permanent sound
(
SynthDef(\sound,{arg freq=440, amp=0.1;
var saw, filter, env;

saw= Saw.ar(freq);

filter= Resonz.ar(saw,1000,0.1)*amp;

Out.ar(0,filter.dup(2))
}).add
)

a= Synth(\sound,[\freq,77,\amp,0.9]); //create running synth

//use the set message to update the control inputs of the running synth
MIDIIn.control = { arg src, chan, num, val; a.set(\amp, val/127) };

//when you're finished twiddling MIDI controllers

a.free;
For sending MIDI messages out see the MIDIOut help file
[MIDIOut]

WARNING: by default there is a long latency to messages sent out, to allow it to

match with other default latencies in the system.

m= MIDIOut(0); //quick way to access device 0, port 0

m.latency= 0.0; //use this to remove all latency and send messages immediately
m.noteOn(1,60,127); //arguments: channel, note, velocity
m.noteOff(1,60,0);

There are also some helper classes to allow you to more easily set up multiple
independent callbacks for the same type of message:

[MIDIFunc]
[MIDIdef]

To make a callback for when receiving note on messages:

MIDIIn.connect(0,MIDIClient.sources[1]) //second source device

(
b = MIDIFunc.noteOn({ |velocity, midipitch, channel|
   
[\velocity,velocity, \midinote,midipitch, \channel, channel].postln;

});
)

//make a separate callback, also for MIDI note on triggers

(
c = MIDIFunc.noteOn({ |velocity, midipitch, channel|
   
"note on!".postln;

});
)

//remove first callback and keep second

b.free;

//see the [Using MIDI] helpfile for more information

c.free; //remove second

//note that by default, cmd+period will remove any MIDIFuncs that haven't been made
permanent
SoundIn

To obtain an audio input stream, use the simple SoundIn UGen

{ SoundIn.ar([0,1],0.1) }.play; // stereo through patching from 2 inputs

to output

{ SoundIn.ar(1,0.1) }.play; // mono on input channel 1; won't work if you

don't have at least 2 inputs!

So it's easy to build effects processors for live audio:

(
{ //ring modulator
SinOsc.ar(MouseX.kr(0.001,110,'exponential' ))*SoundIn.ar(0,0.5)
}.play; // stereo through patching from input to output
)
SuperCollider comes with an amplitude tracker and pitch tracker for realtime audio

(
// use input amplitude to control Pulse amplitude - use headphones to prevent
feedback.
{
Pulse.ar(90, 0.3, Amplitude.kr(SoundIn.ar(0)))
}.play
)

You can threshold the input to avoid picking up background noise

(
{
var input,inputAmp,threshhold,gate;
var basefreq;

input = SoundIn.ar(0,0.1);
inputAmp = Amplitude.kr(input);
threshhold = 0.02; // noise gating threshold
gate = Lag.kr(inputAmp > threshhold, 0.01);
(input * gate)
}.play;
)

The Pitch follower has many input arguments, though you usually take the defaults
without worrying. It returns two outputs- the tracked frequency and a signal
indicating
whether it has locked onto any periodicity or not

Server.internal.boot; //if on a Mac you'll need to swap back to internal server for
using .scope- you can have both the internal and localhost server on at once, but
you might need to press the -> default button

//showing the outputs - K2A makes sure control rate signals are converted to audio
rate, because the final output of a Synth has to be audio rate
(
{
var freq, hasFreq;
# freq, hasFreq = Pitch.kr(SoundIn.ar(1,0.1));
[K2A.ar(freq*0.001), K2A.ar(hasFreq)]
}.scope
)

//detected fundamental frequency used to control some oscillators with allpass

reverberation
//Amplitude detector also used to make the control track the input more effectively
(
{
var in, amp, freq, hasFreq, out;
in = Mix.ar(SoundIn.ar([0,1]));
amp = Amplitude.kr(in, mul: 0.4);
# freq, hasFreq = Pitch.kr(in);
out = Mix.ar( LFTri.ar(freq * [0.5, 1, 2]) ) * amp;
6.do({
out = AllpassN.ar(out, 0.040, [0.040.rand,0.040.rand], 2)
});
out
}.play
)

//Also switch waveform based on hasFreq output

(
{
var in, amp, freq, hasFreq, out;

in = SoundIn.ar(1);
amp = Amplitude.kr(in, mul: 0.4);
# freq, hasFreq = Pitch.kr(in);

out=if(hasFreq,Pulse.ar(freq,0.5,0.1),SinOsc.ar(freq,0,0.1));

6.do({
out = AllpassN.ar(out, 0.040, [0.040.rand,0.040.rand], 2)
});
out
}.play
)
There are various other machine listening capabilities in SuperCollider. Machine
listening is getting the computer to extract perceptually and musically meaingful
attributes by analyzing an input sound.

Here are some onset detectors which might be helpful:

[Onsets]
[PV_HainsworthFoote]
[PV_JensenAndersen]

They rely on using the FFT UGen in the front end to go from time domain to
frequency domain. You can trust the code examples for now and we'll investigate FFT
properly later on (or explore the help file yourself).

Example triggering TGrains UGen:

//run this first

b = Buffer.read(s,Platform.resourceDir +/+"sounds/a11wlk01.wav");

//now this
(
{
var source, detect;

source= SoundIn.ar(0);

detect= Onsets.kr(FFT(LocalBuf(2048),source),0.1); //second argument is

detection threshold
//detect= PV_HainsworthFoote.ar(FFT(LocalBuf(2048),source), 1.0, 0.0, 0.7,
0.01);

TGrains.ar(2, detect, b, LFNoise0.kr(10,0.2,1.0), MouseX.kr(0,BufDur.kr(b)),

MouseY.kr(0.1,0.5), LFNoise0.kr(10,1.0), 0.5, 2);
}.play
)

RecordBuf

If you'd like to capture live sound, the RecordBuf UGen is your friend.
You need to set up a buffer to store the recorded sample data.

(
var b;

b=Buffer.alloc(s,44100,1); //1 second mono buffer allocated on local server

{
//continuously record in a loop, recording to the buffer we just declared
//each record cycle multiplies the old data
RecordBuf.ar(SoundIn.ar(0), b, 0, 1.0, MouseX.kr(0.0,1.0), 1, 1, 1);

//playback the captured buffer in a loop, backwards

PlayBuf.ar(1,b,MouseY.kr(0.0,-1.0), 1,0,1);
}.play;
)

You might sync captured buffers to tempo for dance music, and add refinements like
a user interface to choose when to rerecord the buffer...

There are also facilities for control from graphics tablets and joysticks:

[SC2DTabletSlider]
[HIDDeviceService]
[GeneralHID]

You might also like to try

[SerialPort] //serial port (via USB usually these days) for talking to
certain external devices

Another standard way is to communicate with other applications using Open Sound
Control, a network music protocol; we'll cover this in a later week in this course.