Building on my PCOMP Midterm
The Ripple Visual from my Midterm
I wanted to build on my midterm project and make a VJ box with a larger interface and more functionality. My goal was to have a strong p5 visualization that had more effects rather than simply changing color or frame rate. While brainstorming, I had wanted to the visual to be sound reactive as well as reacting to the device interaction. Beyond this, I wanted to have a website of at least three visual options that someone could choose from. Those visuals would vary in use, where one would simply be for having on in the background while music was playing, while another would be very interactive and the user would have to make effects occur with the music on their own, and the last would have a video component that performers coudl have as a backdrop. Beyond this, I could even put the circuit diagram on the website, as well as an instructable on box construction if someone wanted to create their own version of the project. Throughout the process of creating the box and coding the visual, some of these things were lost.
I started by creating three visuals the resonated with me, and initially used keypress functions to simulate box interacting, as I had alread done the serial communication code necessary for connecting this to an arduino, and I knew I could replicate it.
soundStorm
I liked the idea doing something with a thunderstorm, because I thought lightning and rain would be cool effects to see while music was playing. I started searching for inspiration code, and came across these three examples/sketches: Mouse Ripple, Generative Lightning, and rain. I took this code and tweaked it so that it would have variable changes and could function as an effect that is called based on key press (or button press for serial communication). I wrote the code for the bubble clouds based on a bouncyBalls homework assignment in ICM.
soundWaves
For this sketch, which isn't retained in the final version, I just wanted something simple and found two example of people working with sound waves. I added them both to this sketch and tweaked positioning, number, color, and opacity.
videoKill
This sketch was to be the one that incorporated the video aspect. I was really into the colors and video capture from my pixel project so I wanted to find a way to get video of myself with no background, which I could tweak the pixel colors of. I found this great source code for ML5 body segmentation. I thought it would be could to be able to change this into a grid, and have the grid number be one of the variable changes on the box interface, so I added that grid number as a variable to the code, as well as made two of the pixel colors chnage to a variable so the colors of the video would change as well. I wanted the box to have some general effects as well, so I added variables for blur, grain, and frame rate. The film grain code I found here.
Me in the pixels project!
Me in the video capture!
Arduino IDE & Fabrication
Once I had mostly finished making these sketches, I had to start fabricating the box and updating my original arduino code. This interface has two rotary encoders, fours slide potentiometers, 3 regular potentiometers, and six buttons. For the box fabrication, I expanded the design of my midterm box so that it was much larger. I updated my arduino code so that it had the proper number of inputs, created a basic version of the circuit, and set up soundStorm so that it could check the serial communication and make sure that everything was reacting properly.
Laser Cutting Box Walls
Base of Box Together
Checking Input Fits
Circuit
The Circuit Works!
Playtesting & Changing Code
After user and playtesting the circuit on the three different visuals, I gathered that people seemed to resonate with the body segmentation more than the other two. Some even suggested that I only focus on one. I agreed, because it felt like having to go to the computer to change the visuals kind of took people out of the experience of really getting to play with the box. I also didn't love how the buttons would only do these effects that would just diappear, and you'd have to keep pressing them to see something show up. It didn't feel impactful. From here, I decided to pair back and focus on one visual and really beef it up. I basically placed the soundStorm effects into the videoKill sketch. ML5 seems to not be able to function in a sketch while it is sound reactive, so I had to abandon that aspect of the project. From here, rather than having the effects be one off on the press of a button, I wanted them to just turn on and off. Furthermore, in order for all the buttons on the box to have a use, I had to create two new effects to fill the empty space. My classmate had sent me a cool delay effect on a grid of video captures, and when I got it working in the greater sketch the color changes over time were really fantastic. I also added this super cool laser sketch, which I used Chat GPT to help me get it moving. To be clear, with how long the code is, I did use Chat GPT to help organize and comment the code out so it's easier to understand. Before I had done that, everything was all over the place and it was very difficult to find things so I could change them.
Finishing Fabrication
While changing the code, I was also painting the box and putting the circuit together inside of it so that people could really get a feel for how the different inputs would function. Another classmate had given me the idea of putting vinyls on the rotary encoders so that it really felt like spinning a record to mix up visuals. I thought that was cool, so I got smaller vinyl record coasters and dremeled them to fit as rotary encoder caps.
in the spray paint booth
galaxy painted box
galaxy painted box cover
mess of soldered wires/inputs
Full Fabrication
Execution
Here's a video of the project working, as well as the final source code!