Apr 19 2009
Maestro 0.5b
I've given another last shot to the piano, it's working now. I removed proMIDI from it and implented ess r2, however, i don't have time to work on it anymore. I'm finishing the final report at the moment.
Apr 19 2009
I've given another last shot to the piano, it's working now. I removed proMIDI from it and implented ess r2, however, i don't have time to work on it anymore. I'm finishing the final report at the moment.
Apr 12 2009
I've revisted version 1.1 as the 1.2's Sine class is making it difficult to add on to at the moment. So, 1.1b is changing the pitch of a sine wave on the Y axis and changing the pitch of a triangle wave on the X axis. Here is the code:
import JMyron.*; import krister.Ess.*; import controlP5.*; JMyron m; //a camera object ControlP5 controlP5; AudioChannel myChannel; AudioChannel myChannel2; SineWave myWave; TriangleWave myWave2; float volume=1; int pitch=400; int release=10000; FadeIn myFadeIn; FadeOut myFadeOut; FadeIn myFadeIn2; FadeOut myFadeOut2; LowPass myLowPass; int r; int g; int b; int min_size = 50; int max_size=100; int sensitivity = 35; float objx = 160; float objy = 120; float objdestx = 160; float objdesty = 120; void setup(){ size(720,576); m = new JMyron();//make a new instance of the object m.start(width,height);//start a capture at 320x240 m.findGlobs(1);//disable the intelligence to speed up frame rate controlP5 = new ControlP5(this); /* controlP5.addSlider("sensitivity",0,255,40,10,10,10,100).setId(1); name, low val, high val, default val, x, y, width, height, */ controlP5.addSlider("sensitivity",0,255,sensitivity,10,10,10,100).setId(1); controlP5.addSlider("min size",0,800,min_size,60,10,10,100).setId(2); //controlP5.addSlider("max size",0,400,max_size,100,10,10,100).setId(3); //println("here are the globs:" + m.globCenters()); Ess.start(this); myChannel=new AudioChannel(); myChannel2=new AudioChannel(); myFadeIn=new FadeIn(); myFadeOut=new FadeOut(); myFadeIn2=new FadeIn(); myFadeOut2=new FadeOut(); } void draw(){ m.update();//update the camera view m.trackColor(r,g,b,sensitivity); m.minDensity(min_size); //m.maxDensity(max_size); int[] img = m.image(); //get the normal image of the camera loadPixels(); for(int i=0;i<width*height;i++){ //loop through all the pixels pixels[i] = img[i]; //draw each pixel to the screen } updatePixels(); stroke(0,255,0); int[][][] list = m.globEdgePoints(30);//get the outlines for(int i=0;i <list.length;i++){ beginShape(); if(list[i]!=null){ for(int ii=0;ii<list[i].length;ii++){ vertex(list[i][ii][0],list[i][ii][1]); } } endShape(); } int[][] centers = m.globCenters();//get the center points //draw all the dots while calculating the average. //println("Amount of globs: " + centers.length); float avX=0; float avY=0; for(int i=0;i<centers.length;i++){ fill(0); rect(centers[i][0],centers[i][1],1,1); avX += centers[i][0]; avY += centers[i][1]; } if(centers.length-1>0){ avX/=centers.length-1; avY/=centers.length-1; } //draw the average of all the points in red. fill(255,0,0); rect(avX,avY,5,5); //update the location of the thing on the screen. if(!(avX==0&&avY==0)&¢ers.length>0){ objdestx = avX; objdesty = avY; } objx += (objdestx-objx)/5.0f; objy += (objdesty-objy)/5.0f; fill(30,100,0); ellipseMode(CENTER); ellipse(objx,objy,25,25); //println("X of Glob:" + objx + "Y of Glob:" + objy); //println("the avx variable:" + avX); myWave=new SineWave(objy,volume); myWave2=new TriangleWave(objx,1); myChannel2.initChannel(myChannel2.frames(release)); myChannel.initChannel(myChannel2.frames(release)); myWave.generate(myChannel,5,myChannel.frames(release)); myWave2.generate(myChannel2,0,myChannel2.frames(release)); myFadeOut.filter(myChannel); myFadeIn.filter(myChannel); myFadeOut2.filter(myChannel2); myFadeIn2.filter(myChannel2); } void controlEvent(ControlEvent theEvent) { switch(theEvent.controller().id()) { case(1): sensitivity = (int)(theEvent.controller().value()); print("sensitivity changed to " + sensitivity + "\n"); break; case(2): min_size = (int)(theEvent.controller().value()); break; case(3): max_size = (int)(theEvent.controller().value()); break; } } if (mouseEvent.getClickCount() == 2) { mouseX = m.getX(); mouseY = m.getY(); println("The co-ordinates of the mouse click are:"); println("X:" + mouseX + " Y:" + mouseY); int c = get(mouseX, mouseY); r = int(red(c)); g = int(green(c)); b = int(blue(c)); println ("R: " + r + " G: " + g + " B: " + b + " Sens: " + sensitivity); myChannel.play(); myChannel2.play(); return; } } public void stop(){ Ess.stop(); m.stop();//stop the object super.stop(); }
Apr 12 2009
Added functionality:
Here's the code:
import JMyron.*; //import JMyron import krister.Ess.*; //import Ess import controlP5.*; //import controlP5 JMyron m; //create camera object ControlP5 controlP5; //create controlP5 object AudioStream speaker; //Create an audiostream object Sine mSine; //create a sine object from the sine class FadeIn myFadeIn; //create a fadein object FadeOut myFadeOut; //create a fadeout object boolean recording=false; AudioFile myFile; int bytesWritten; AudioStream myStream; SineWave myWave; AudioChannel myChannel; AudioChannel mySound; int r; //intialise the red color as an int int g; //initialise the green color as an int int b; //intialise the blue color as an int int min_size = 50; int max_size=100; int sensitivity = 50; float objx = 160; float objy = 120; float objdestx = 160; float objdesty = 120; float volume = 0.2; void setup(){ size(720,576); m = new JMyron();//make a new instance of the object m.start(width,height); //start a capture with the height and width m.findGlobs(1); //invoke the findGlobs method on the JMyron object controlP5 = new ControlP5(this); //initiate a controlP5 object controlP5.addSlider("sensitivity",0,255,sensitivity,10,10,10,100).setId(1); //add a slider to the controlP5 object that will change the sensitivty of trackcolor method of JMyron controlP5.addSlider("min size",0,800,min_size,60,10,10,100).setId(2); //add a slider to the control P5 object that will change the minimum size of the detectable pixels of the minDensity of JMyron controlP5.addSlider("volume",0,3,volume,120,10,10,100).setId(3); //add a slider to the control P5 object that will change the minimum size of the detectable pixels of the minDensity of JMyron Ess.start(this); //start the ess object speaker = new AudioStream(); //initiate a new AudioStram object mSine = new Sine(0, volume, speaker.sampleRate); //define a new since wave of set values myFile=new AudioFile(); speaker.start(); //start the new audioStream Object myChannel = new AudioChannel("drum.wav"); mySound = new AudioChannel("recording.wav"); } void draw(){ m.update(); //update the camera view m.trackColor(r,g,b,sensitivity); //track the rgb and sensitivity to track the rgb m.minDensity(min_size); //minimum pixels in the glob required to result in a box int[] img = m.image(); //get the normal image of the camera loadPixels(); //load the pixels for(int i=0;i<width*height;i++){ //loop through all the pixels pixels[i] = img[i]; //put all the values in the img array into the pixel array } updatePixels(); //update the pixels on screen stroke(r - 25,g-25,b-25); //set the stroke to the selected color and reduce the outline int[][][] list = m.globEdgePoints(30); //create a 3 dimensional array called list and place all the points desciribing the outlines of each glob for(int i=0;i <list.length;i++){ //loop through the list array of GlobEdgePoints beginShape(); //start recording the vertices for the shape if(list[i]!=null){ //if the list of globEdgePoints is not empty for(int ii=0;ii<list[i].length;ii++){ //create an int ii, loop through the list array of globEdgePoints vertex (list[i][ii][0],list[i][ii][1]); //draw the boxes for the globs points and connect them fill(r,g,b); //set the fill color of the glob area to the selected color } } endShape(); } int[][] centers = m.globCenters(); //create an array to find the center points off all the globs //draw all the dots while calculating the average. //println("Amount of globs: " + centers.length); float avX=0; //initialise a float integer avX float avY=0; //initialise a float integer avY for(int i=0;i<centers.length;i++){ //a for loop that runs for the amount of center points that exist fill(0); //give them a color of black rect(centers[i][0],centers[i][1],1,1); //draw the centers at the given array values and make them 1x1 pixel in size avX += centers[i][0]; //equal to avX = avX + centers[i][0] avY += centers[i][1]; //equal to avY = avY + centers[i][1] } if(centers.length-1>0){ //if the amount of centers -1 is greater than 0 avX/=centers.length-1; //equal to avX = avX / the length of the centers array - 1 avY/=centers.length-1; //equal to avY = avY / the length of the centers array - 1 } //draw the average of all the points in red. fill(255,0,0); rect(avX,avY,5,5); //update the location of the thing on the screen. if(!(avX==0&&avY==0)&¢ers.length>0){ //if the averages of the center points and are not at 0 and the centers array is greater than 0 objdestx = avX; //set objestx to avX objdesty = avY; //set objesty to avY } objx += (objdestx-objx)/5.0f; //equal to objx to objx + (objdestx-objx)/2.0f, f declares it a float objy += (objdesty-objy)/5.0f; //equal to objY to objy + (objdesty-objy)/2.0f fill(r*2,g*2,b*2); //the fill color is twice the value that was clicked on ellipseMode(CENTER); //center an elipse around that point ellipse(objx,objy,25,25); //the position of the glob is set to objx and objy, it's 5 pixels in height and width mSine.setFreq( map(objy, 0, width, 2880, 0) ); //map the frequency of the Sine to the Y axis based between 2880 and 0 of the intial freq //println("X of Glob:" + objx + "Y of Glob:" + objy); //println("the avx variable:" + avX); } void audioStreamWrite(AudioStream s) { mSine.generate(s); //generate the sine wave if (recording) { myFile.write(speaker); bytesWritten+=speaker.size*2; } } void controlEvent(ControlEvent theEvent) { //controlP5 controller stuff switch(theEvent.controller().id()) { case(1): sensitivity = (int)(theEvent.controller().value()); //set the sensitivity print("sensitivity changed to " + sensitivity + "\n"); break; case(2): min_size = (int)(theEvent.controller().value()); //set the minimum size of the pixels that we will track break; case(3): volume = (float)(theEvent.controller().value()); //set the volum break; } } if (mouseEvent.getClickCount() == 2) { //if they are 2 mouse clicks mouseX = m.getX(); //get the mouse horizontal value of the JMyron object mouseY = m.getY(); //get the mouse vertical value of the JMyron object println("The co-ordinates of the mouse click are:"); println("X:" + mouseX + " Y:" + mouseY); int c = get(mouseX, mouseY); //initialise c as the x and y co-ords of the mouse r = int(red(c)); //get the red int value of c g = int(green(c)); //get the green int value of c b = int(blue(c)); //get the blue int value of c println ("R: " + r + " G: " + g + " B: " + b + " Sens: " + sensitivity); } } void keyPressed() { if(key=='r') { if (recording) { // stop myFile.close(); println("Finished recording. "+bytesWritten+" bytes written."); } else { // start myFile.open(dataPath("recording.wav"),speaker.sampleRate,Ess.WRITE); bytesWritten=0; println("Recording started."); } recording=!recording; } if(key=='d') { println("drum!"); myChannel.play(); } if(key=='s') { println("Play recording"); mySound.play(); } } public void stop(){ m.stop(); //stop the object super.stop(); }
Sine Class
class Sine { private float freq, newFreq; private float amp; private float stepSize; private float step; Sine(float f, float a, float s) { freq = f; newFreq = freq; amp = a; stepSize = 1/s; step = 0; } void setFreq(float f) { newFreq = f; } boolean generate(AudioOutput ao) { float[] buff = new float[ao.buffer.length]; boolean alert = generate(buff); arraycopy(buff, 0, ao.buffer2, ao.buffer.length, buff.length); arraycopy(buff, 0, ao.buffer, 0, buff.length); return alert; } boolean generate(float[] buff) { boolean alert = freq != newFreq; for ( int i = 0; i < buff.length; i++ ) { buff[i] = amp*sin(freq*TWO_PI*step); if ( abs(freq-newFreq) < 0.1f ) freq = newFreq; else if ( freq < newFreq ) freq += 0.1f; else if ( freq > newFreq ) freq -= 0.1f; step += stepSize; if ( step > 1/freq ) step %= 1/freq; } return alert; } }
Apr 10 2009
After playing around with the last interaction example, a sound is generated if my hand goes over a button. I'm going to try and develop this with MIDI to create a virtual piano. Should be fun....
Apr 09 2009
Major breakthrough. I've got a digital thermin created, although it isn't very polished, it's working! The play method is placed in the mouose event method, and updates each time the average point is updated. The click is still occuring, however, I will try and change the priginal prototype to AudioStreams rather than channels to see if it'll sort that out.
Here's the code:
import JMyron.*; import krister.Ess.*; import controlP5.*; JMyron m;//a camera object ControlP5 controlP5; AudioChannel myChannel; SineWave myWave; float volume=1; int pitch=400; int release=10000; FadeIn myFadeIn; FadeOut myFadeOut; int r; int g; int b; int min_size = 50; int max_size=100; int sensitivity = 35; float objx = 160; float objy = 120; float objdestx = 160; float objdesty = 120; void setup(){ size(720,576); m = new JMyron();//make a new instance of the object m.start(width,height);//start a capture at 320x240 m.findGlobs(1);//disable the intelligence to speed up frame rate controlP5 = new ControlP5(this); /* controlP5.addSlider("sensitivity",0,255,40,10,10,10,100).setId(1); name, low val, high val, default val, x, y, width, height, */ controlP5.addSlider("sensitivity",0,255,sensitivity,10,10,10,100).setId(1); controlP5.addSlider("min size",0,800,min_size,60,10,10,100).setId(2); //controlP5.addSlider("max size",0,400,max_size,100,10,10,100).setId(3); //println("here are the globs:" + m.globCenters()); Ess.start(this); myChannel=new AudioChannel(); myFadeIn=new FadeIn(); myFadeOut=new FadeOut(); } void draw(){ m.update();//update the camera view m.trackColor(r,g,b,sensitivity); m.minDensity(min_size); //m.maxDensity(max_size); int[] img = m.image(); //get the normal image of the camera loadPixels(); for(int i=0;i<width*height;i++){ //loop through all the pixels pixels[i] = img[i]; //draw each pixel to the screen } updatePixels(); stroke(0,255,0); int[][][] list = m.globEdgePoints(30);//get the outlines for(int i=0;i <list.length;i++){ beginShape(); if(list[i]!=null){ for(int ii=0;ii<list[i].length;ii++){ vertex(list[i][ii][0],list[i][ii][1]); } } endShape(); } int[][] centers = m.globCenters();//get the center points //draw all the dots while calculating the average. //println("Amount of globs: " + centers.length); float avX=0; float avY=0; for(int i=0;i<centers.length;i++){ fill(0); rect(centers[i][0],centers[i][1],1,1); avX += centers[i][0]; avY += centers[i][1]; } if(centers.length-1>0){ avX/=centers.length-1; avY/=centers.length-1; } //draw the average of all the points in red. fill(255,0,0); rect(avX,avY,5,5); //update the location of the thing on the screen. if(!(avX==0&&avY==0)&¢ers.length>0){ objdestx = avX; objdesty = avY; } objx += (objdestx-objx)/10.0f; objy += (objdesty-objy)/10.0f; fill(30,100,0); ellipseMode(CENTER); ellipse(objx,objy,25,25); println("X of Glob:" + objx + "Y of Glob:" + objy); println("the avx variable:" + avX); myWave=new SineWave(objx,volume); myChannel.initChannel(myChannel.frames(release)); myWave.generate(myChannel,5,myChannel.frames(release)); myFadeOut.filter(myChannel); myFadeIn.filter(myChannel); } void controlEvent(ControlEvent theEvent) { switch(theEvent.controller().id()) { case(1): sensitivity = (int)(theEvent.controller().value()); print("sensitivity changed to " + sensitivity + "\n"); break; case(2): min_size = (int)(theEvent.controller().value()); break; case(3): max_size = (int)(theEvent.controller().value()); break; } } void mouseClicked(MouseEvent m) { if (mouseEvent.getClickCount() == 2) { mouseX = m.getX(); mouseY = m.getY(); println("The co-ordinates of the mouse click are:"); println("X:" + mouseX + " Y:" + mouseY); int c = get(mouseX, mouseY); r = int(red(c)); g = int(green(c)); b = int(blue(c)); println ("R: " + r + " G: " + g + " B: " + b + " Sens: " + sensitivity); myChannel.play(); return; } } public void stop(){ m.stop();//stop the object super.stop(); }
Apr 09 2009
For testing purposes, I have added the tracking of the pixels of a certain value and the average point of these. This allows me to grab the x and y co-ordinates of the average point an, hopefully, work the ess prototype around it.
Mar 26 2009
I've been looking around the processing forum and found a few tips to try out. Seemingly, it's better to use Streams in ess rather than channels, which is what I am using. Also, specific frequencies can be set rather than just the float values I've been using. More testing is needed on this one.
Mar 24 2009
Playing around with the fade functions with ess r2. It's smoother and working better (kinda). The fade function works but, if I click mid note, a click can be heard and the sine wave doesn't fade in. A loud pop is heard and then the sine wave plays for the full volume for about quarter of a second. I also got controlP5 to update the values when the slider is activated. this was due to not calling the background function in void draw. Here's the code so far:
import krister.Ess.*; import controlP5.*; ControlP5 controlP5; AudioChannel myChannel; SineWave myWave; float volume=0.5; int pitch=400; int release=1000; FadeIn myFadeIn; FadeOut myFadeOut; void setup() { //background(0); //frameRate(25); size(256,200); Ess.start(this); myChannel=new AudioChannel(); controlP5 = new ControlP5(this); controlP5.addSlider("Pitch",400,700,40,10,10,10,100).setId(1); controlP5.addSlider("Volume",0,2,40,70,10,10,100).setId(2); controlP5.addSlider("Release",1000,8000,40,120,10,10,100).setId(3); myFadeIn=new FadeIn(); myFadeOut=new FadeOut(); } void controlEvent (ControlEvent theEvent) { switch(theEvent.controller().id()) { case(1): pitch = (int)(theEvent.controller().value()); //print("the pitch changed to " + pitch + "\n"); break; case(2): volume = (float)(theEvent.controller().value()); //print("the VOLUME changed to " + volume + "\n"); break; case(3): release = (int)(theEvent.controller().value()); //print("the note length changed to " + release + "\n"); break; } myWave=new SineWave(pitch,volume); myChannel.initChannel(myChannel.frames(release)); myWave.generate(myChannel,5,myChannel.frames(release)); myFadeOut.filter(myChannel); myFadeIn.filter(myChannel); myChannel.play(); } void draw() { background(0); } public void stop() { Ess.stop(); super.stop(); }
Mar 23 2009
Had the meeting with Pat today and he advised me to look into fading the audio to get rid of the clicking sound, he also I should concentrate more on the audio side of things as JMyron has eaten up a lot of my time.
I had a quick look at the reference pages on ess r2 and there's a fade filter which may suffice.
Mar 22 2009
Got a simple throwaway prototype working for sound. ControlP5 is manipulating sine waves being generated in realtime. It's a little buggy, the sound is clicking a lot which may have something to do with the buffer, and controlP5's slider values are not updating on the screen. Here's the code so far:
import krister.Ess.*; import controlP5.*; ControlP5 controlP5; AudioChannel myChannel; SineWave myWave; float volume=0.5; int pitch=400; int release=1000; void setup() { background(0); size(256,200); Ess.start(this); myChannel=new AudioChannel(); controlP5 = new ControlP5(this); controlP5.addSlider("Pitch",400,700,40,10,10,10,100).setId(1); controlP5.addSlider("Volume",0,2,40,70,10,10,100).setId(2); controlP5.addSlider("Release",1000,8000,40,120,10,10,100).setId(3); } void controlEvent (ControlEvent theEvent) { switch(theEvent.controller().id()) { case(1): pitch = (int)(theEvent.controller().value()); //print("the pitch changed to " + pitch + "\n"); break; case(2): volume = (float)(theEvent.controller().value()); //print("the VOLUME changed to " + volume + "\n"); break; case(3): release = (int)(theEvent.controller().value()); //print("the note length changed to " + release + "\n"); break; } myWave=new SineWave(pitch,volume); myChannel.initChannel(myChannel.frames(release)); myWave.generate(myChannel,5,myChannel.frames(release)); myChannel.play(); } void draw() { controlP5.update(); } public void stop() { Ess.stop(); super.stop(); }