There’s an unspoken tenant of the maker movement that demands: calls for bespoke engineering work from friends should always be answered. Maker projects for friends pay dividends in net-happiness injected into the world.
My ‘ol pal Ben (A.K.A Bensbeendead.) knows this kind of work is a favorite of mine. So when he asked, of course I jumped at the opportunity to design and manufacture some “elbows” that mount his laptop and controller atop RGB stage lighting.
This post is one in a series about GANce
Close-readers, twitter-followers and corporeal-comrades will have already beheld the good news that Won Pound by Won Pound has been released! This is Won’s second album-length project (first of course being Post Space released in 2018), and graces listener’s ears courtesy of Minaret Records, a California jazz label.
The record is accompanied by an album-length music video synthesized with GANce, containing a completely unique video for each track. These 93960 frames have been the ultimate goal of this project since it’s inception, and serve as the best demonstration as to what GANce can do. Within the video (linked below), the video for ‘buzzz’ is a personal favorite, demonstrating the three core elements of a projection file blend:
This post is one in a series about GANce
As it stood, the three main features that would comprise the upcoming collaboration with Won Pound (slated for release mid-April) were:
As detailed in the previous post. The effect of these three features can be seen in this demo:
Knowing we had enough runway to add another large feature to the project, and feeling particularly inspired following a visit to Clifford Ross’ exhibit at the Portland Museum of Art, I began exploring the relationship between the projection source video and the output images synthesized by the network.
This post is one in a series about GANce
In collaboration with Won Pound for his forthcoming album release via minaret records I was recently commissioned to lead an expedition into latent space, encountering intelligences of my own haphazard creation.
A word of warning:
This and subsequent posts as well as the GitHub etc. should be considered toy projects. Development thus far has been results-oriented, with my git HEAD following the confusing and exciting. The goal was to make interesting artistic assets for Won’s release, with as little bandwidth as possible devoted to overthinking the engineering side. This is a fun role-reversal, typically the things that leave my studio look more like brushes than paintings. In publishing this work, the expected outcome is also inverted from my typical desire to share engineering techniques and methods; I hope my sharing the results shifts your perspective on the possible ways to bushwhack through latent space.
So, with that out of the way the following post is a summary of development progress thus far. Here’s a demo:
There are a few repositories associated with this work:
If you’re uninterested in the hardware/software configurations for image capture and GPU work, you should skip to Synthesizing Images.
Ever wanted to have multiple different sound files playing on different output devices attached to a host computer? Say you’re writing a DJing application where you want one mix for headphones and one for the speakers. Or you’re doing some sort of kiosk or art installation where you have many sets of speakers that need to all be playing their own sound file but the whole thing needs to be synchronized. This would even be cool for something like an escape room.
The ladder example is where I needed this bit of code. I’ve been working with interdisciplinary artist Sara Dittrich on a few projects recently and she asked if I could come up with a way to play 8 different mono sound files on 8 different loudspeakers. Here’s a video of the whole setup in action, and an explanation of the project:
I’ve wrapped up all of the code for the art installation project, and that can be found in a github repo here. It includes the startup functionality etc. If you’re interested in recreating the video above, that repo would be a good starting place. The following is a list of the parts used to make that build happen:
It is worth it to give a simple example of how to play multiple files on multiple audio devices using python. I couldn’t find an examples on how to do this online and had to spend some time experimenting to make it all come together. Hopefully this saves you the trouble.
To install sounddevice on my Raspberry Pi, I had to run the following commands:
sudo apt-get install python3-pip python3-numpy libportaudio2 libsndfile1 libffi-dev python3 -m pip install sounddevice soundfile
For this example, let’s say there are 4 audio files in the same directory as multi.py , so the directory looks like this:
multi_audio/ ├── 1.wav ├── 2.wav ├── 3.wav ├── 4.wav └── multi.py
The code is based on the sounddevice library for python, whose documentation is pretty sparse. This script will find the audio files, and then play them on as many devices as there are attached. For example, if you have 3 sound devices it will play 1.wav, 2.wav and 3.wav on devices 1-3. If you have any questions, feel free to ask:
"""
multi.py, uses the sounddevice library to play multiple audio files to multiple output devices at the same time
Written by Devon Bray (dev@esologic.com)
"""
import sounddevice
import soundfile
import threading
import os
DATA_TYPE = "float32"
def load_sound_file_into_memory(path):
"""
Get the in-memory version of a given path to a wav file
:param path: wav file to be loaded
:return: audio_data, a 2D numpy array
"""
audio_data, _ = soundfile.read(path, dtype=DATA_TYPE)
return audio_data
def get_device_number_if_usb_soundcard(index_info):
"""
Given a device dict, return True if the device is one of our USB sound cards and False if otherwise
:param index_info: a device info dict from PyAudio.
:return: True if usb sound card, False if otherwise
"""
index, info = index_info
if "USB Audio Device" in info["name"]:
return index
return False
def play_wav_on_index(audio_data, stream_object):
"""
Play an audio file given as the result of `load_sound_file_into_memory`
:param audio_data: A two-dimensional NumPy array
:param stream_object: a sounddevice.OutputStream object that will immediately start playing any data written to it.
:return: None, returns when the data has all been consumed
"""
stream_object.write(audio_data)
def create_running_output_stream(index):
"""
Create an sounddevice.OutputStream that writes to the device specified by index that is ready to be written to.
You can immediately call `write` on this object with data and it will play on the device.
:param index: the device index of the audio device to write to
:return: a started sounddevice.OutputStream object ready to be written to
"""
output = sounddevice.OutputStream(
device=index,
dtype=DATA_TYPE
)
output.start()
return output
if __name__ == "__main__":
def good_filepath(path):
"""
Macro for returning false if the file is not a non-hidden wav file
:param path: path to the file
:return: true if a non-hidden wav, false if not a wav or hidden
"""
return str(path).endswith(".wav") and (not str(path).startswith("."))
cwd = os.getcwd()
sound_file_paths = [
os.path.join(cwd, path) for path in sorted(filter(lambda path: good_filepath(path), os.listdir(cwd)))
]
print("Discovered the following .wav files:", sound_file_paths)
files = [load_sound_file_into_memory(path) for path in sound_file_paths]
print("Files loaded into memory, Looking for USB devices.")
usb_sound_card_indices = list(filter(lambda x: x is not False,
map(get_device_number_if_usb_soundcard,
[index_info for index_info in enumerate(sounddevice.query_devices())])))
print("Discovered the following usb sound devices", usb_sound_card_indices)
streams = [create_running_output_stream(index) for index in usb_sound_card_indices]
running = True
if not len(streams) > 0:
running = False
print("No audio devices found, stopping")
if not len(files) > 0:
running = False
print("No sound files found, stopping")
while running:
print("Playing files")
threads = [threading.Thread(target=play_wav_on_index, args=[file_path, stream])
for file_path, stream in zip(files, streams)]
try:
for thread in threads:
thread.start()
for thread, device_index in zip(threads, usb_sound_card_indices):
print("Waiting for device", device_index, "to finish")
thread.join()
except KeyboardInterrupt:
running = False
print("Stopping stream")
for stream in streams:
stream.abort(ignore_errors=True)
stream.close()
print("Streams stopped")
print("Bye.")
Here are some more photos of the build:
Here’s a video:
Here are some images:
This is a quick design I came up with rather than spend the $20 on amazon. You can get the STL files on thingiverse here.
I wrote a page summarizing my HackMIT 2016 Project, check it out here!
When trying to charge a cellphone via the USB port on the amplifier, I blew a power resistor and the 5v regulator on the amp. The following images show the repair process:
Here’s a video:
Not much further explanation needed for this post. The design process for delivering power to all components of the system is complete. The topology is pretty simple:
12v Battery -> switch on positive rail -> 5v Switching Regulator & 12v rail -> Amp & 5v Linear Regulator -> Arduino
If you have any questions, please let me know.
Here is a video of the whole thing working:
The whole system works! If you look at this post, which basically shows the whole thing can be battery powered as well. The following photos show the way the cable is run out the back of the housing. Both sets of the 3 wires are tied together and then to the output of the digital pot:
Contrary to what I thought, the 10k ohm digital pot can change the volume just fine! To be safe though, I ordered an SPI digital Pot that can do 50k ohm resistance.
Here is the new working version of the code as well:
#include <SPI.h> //for using the digital pot
const int slaveSelectPin = 10; //for SPI, from example code
//Shift Register Setup, taken from www.bildr.og
int SER_Pin = 6; //pin 14 on the 75HC595
int RCLK_Pin = 7; //pin 12 on the 75HC595
int SRCLK_Pin = 8; //pin 11 on the 75HC595
#define number_of_74hc595s 4 //How many of the shift registers - change this
#define numOfRegisterPins number_of_74hc595s * 8
boolean registers[numOfRegisterPins];
int IR_rangefinder = 0; //The pin attached to the rangefinder
int detect_led = 2;
int setlevelMode0_led = 3;
int serialDebug_switch = 4; //Throughout the program, this switch enables or disables all serial output
int check_val = 3; //The arbitrary position above the sensor that indicates a "check" - the position that must be held in order to change the volume
int cycle_delay = 30; //a universal delay time for refreshing the check functions
void setup(){
//shift register setup
pinMode(SER_Pin, OUTPUT);
pinMode(RCLK_Pin, OUTPUT);
pinMode(SRCLK_Pin, OUTPUT);
clearRegisters();
writeRegisters();
//spi setup
pinMode (slaveSelectPin, OUTPUT);
SPI.begin();
//general LED's
pinMode(detect_led, OUTPUT);
pinMode(setlevelMode0_led, OUTPUT);
pinMode(serialDebug_switch, INPUT);
//Serial setup
Serial.begin(9600);
}
//integers to remember values off of the IR sensor.
int prelevel_0 = 0;
int prelevel_1 = 0;
int prelevel_2 = 0;
int prelevel_3 = 0;
int prelevel_4 = 0;
int prelevel_5 = 0;
int prelevel_6 = 0;
int prelevel_7 = 0;
int prelevel_8 = 0;
int prelevel_9 = 0;
int pre_positions[10] = {prelevel_0, prelevel_1, prelevel_2, prelevel_3, prelevel_4, prelevel_5, prelevel_6, prelevel_7, prelevel_8, prelevel_9}; // an array holding the positions
void loop(){
/*
The system works by sampling the sensor a number of times. It puts these values into an array.
Once all sample have been made, each value is compared to a like value. If every && evaluates to true, this means whatever object above the sensor has been there for the "cycle_delay" * the number of comparisions made.
It will confirm that the user wants their hand the be there and it was not acciential.
Think of the following loop as the ambient mode, the user can't adjust the volume from here, but they can enter the mode where they can adjust the volume.
It has much less precision by design.
*/
for(int i = 0; i <= 9; i = i + 1){
writebargraph(0,map(analogRead(IR_rangefinder),20,600,0,9));
pre_positions[i] = map(analogRead(IR_rangefinder),20,600,0,9);
if(pre_positions[i] == check_val){
if(digitalRead(serialDebug_switch) == HIGH){
Serial.println("Check Detected");
}
digitalWrite(detect_led, HIGH);
}
else {
digitalWrite(detect_led, LOW);
}
delay(cycle_delay);
}
if(digitalRead(serialDebug_switch) == HIGH){
for(int i = 0; i <= 9; i = i + 1){
Serial.print(pre_positions[i]);
Serial.print(",");
}
}
//Once it has been determined that the object above the sensor has been there for a long enough time, the system enters the secondary level set mode.
if (pre_positions[0] == check_val && pre_positions[1] == check_val && pre_positions[2] == check_val && pre_positions[3] == check_val && pre_positions[4] == check_val && pre_positions[5] == check_val && pre_positions[6] == check_val && pre_positions[7] == check_val && pre_positions[8] == check_val && pre_positions[9] == check_val ){
if(digitalRead(serialDebug_switch) == HIGH){
Serial.print(" - Pre Level Set");
Serial.println("");
}
delay(500);
setlevel();
delay(500);
}
else {
if(digitalRead(serialDebug_switch) == HIGH){
Serial.println(" - No Set");
}
}
}
void setlevel(){
/*
Very similar to the above topology. This version is much more precise, and has 30 comparison samples as opposed to 10.
It also writes to the digital potentiometer as well at the the same time as the bar graph.
*/
int level0 = 0;
int level1 = 0;
int level2 = 0;
int level3 = 0;
int level4 = 0;
int level5 = 0;
int level6 = 0;
int level7 = 0;
int level8 = 0;
int level9 = 0;
int level10 = 0;
int level11 = 0;
int level12 = 0;
int level13 = 0;
int level14 = 0;
int level15 = 0;
int level16 = 0;
int level17 = 0;
int level18 = 0;
int level19 = 0;
int level20 = 0;
int level21 = 0;
int level22 = 0;
int level23 = 0;
int level24 = 0;
int level25 = 0;
int level26 = 0;
int level27 = 0;
int level28 = 0;
int level29 = 0;
int positions[30] = { level0, level1, level2, level3, level4, level5, level6, level7, level8, level9, level10, level11, level12, level13, level14, level15, level16, level17, level18, level19, level20, level21, level22, level23, level24, level25, level26, level27, level28, level29};
digitalWrite(setlevelMode0_led, LOW);
boolean seeking = true;
while(seeking == true){
for(int i = 0; i <= 29; i = i + 1){
writebargraph(1,map(analogRead(IR_rangefinder),20,600,0,19));
digitalpot(map(analogRead(IR_rangefinder),20,600,0,255)); //Writes to digital pot
positions[i] = map(analogRead(IR_rangefinder),20,600,0,19);
if(digitalRead(serialDebug_switch) == HIGH){
Serial.print(positions[i]);
Serial.print(",");
}
delay(cycle_delay);
}
//Instead of comparing to a predetermined value, it compares it to the first value sampled. If this if statement is true, it means the users hand has stopped moving, indicating they would like to set the volume at that position.
if (positions[0] == positions[0] && positions[1] == positions[0] && positions[2] == positions[0] && positions[3] == positions[0] && positions[4] == positions[0] && positions[5] == positions[0] && positions[6] == positions[0] && positions[7] == positions[0] && positions[8] == positions[0] && positions[9] == positions[0] && positions[10] == positions[0] && positions[11] == positions[0] && positions[12] == positions[0] && positions[13] == positions[0] && positions[14] == positions[0] && positions[15] == positions[0] ){
if(digitalRead(serialDebug_switch) == HIGH){
Serial.print(" - Level Set");
}
digitalWrite(setlevelMode0_led, HIGH);
seeking = false; //Stops the loop and holds the last value on the bar graph and digital pot.
}
else {
if(digitalRead(serialDebug_switch) == HIGH){
Serial.print(" - No Set");
}
digitalWrite(setlevelMode0_led, LOW);
}
if(digitalRead(serialDebug_switch) == HIGH){
Serial.println("");
}
}
}
//This function will write to the shift registers -> the bar graph. It will write all of the values below the one specified HIGH and all above LOW. It also allows multiple sets of bar graphs
void writebargraph(int set, int led){
if(set == 0){
for(int i = 0; i <= 9; i = i + 1){
if(i <= led){
setRegisterPin(i, HIGH);
writeRegisters();
}
else if(i > led){
setRegisterPin(i, LOW);
writeRegisters();
}
}
}
if(set == 1){
for(int k = 10; k <= 29; k = k + 1){
if(k <= 10 + led){
setRegisterPin(k, HIGH);
writeRegisters();
}
else if(k > 10 + led){
setRegisterPin(k, LOW);
writeRegisters();
}
}
}
}
//A very simple function to write values to the Digital Pot
void digitalpot(int value){
digitalWrite(slaveSelectPin,LOW);
SPI.transfer(0); // enables the chip
SPI.transfer(value);
digitalWrite(slaveSelectPin,HIGH);
}
//SHIFT REGISTER FUNCTIONS.
//set all register pins to LOW
void clearRegisters(){
for(int i = numOfRegisterPins - 1; i >= 0; i--){
registers[i] = LOW;
}
}
//Set and display registers
//Only call AFTER all values are set how you would like (slow otherwise)
void writeRegisters(){
digitalWrite(RCLK_Pin, LOW);
for(int i = numOfRegisterPins - 1; i >= 0; i--){
digitalWrite(SRCLK_Pin, LOW);
int val = registers[i];
digitalWrite(SER_Pin, val);
digitalWrite(SRCLK_Pin, HIGH);
}
digitalWrite(RCLK_Pin, HIGH);
}
//set an individual pin HIGH or LOW
void setRegisterPin(int index, int value){
registers[index] = value;
}
The only difference between this one and the last version I posted was the height of the check value. I made it further away from the sensor.
Before the code is “finished” I would like to add a few things. The first being an averaging loop in the raw input ; instead of just using variations of map(analogRead(IR_rangefinder),20,600,0,9); each time, I’d like to maybe write my own function that is more general for assigning comparison. The downside to this however is that it may slow things down and the top priority with this project is keeping it fast and accurate.