Hardware for Engineering Stream

My beloved blog post still sits on the throne as the most effective format for engineering projects. To me, Inlining code, photographs, CAD models and schematics in an interactive way trumps other mediums. This level of interactivity closes the gap between reader and material, allowing an independent relationship with the subject outside of the story being told by the author.

Working on stream to an online audience has a similar effect, the unedited and interactive format yielding a real understanding of the creators process and technique.

For a while there, I’d settled into a nice habit of broadcasting project development live on twitch. Two moves later, things have settled down enough in my personal life that I feel it’s time to try to get back into this habit.

Before we get started again, I took some time to improve the ergonomics (both hardware and software) of my stream setup. The following documents a few smaller projects, all in service of these upcoming broadcasts.

Continue reading →

Won Pound by Won Pound is released!

This post is one in a series about GANce

Close-readers, twitter-followers and corporeal-comrades will have already beheld the good news that Won Pound by Won Pound has been released! This is Won’s second album-length project (first of course being Post Space released in 2018), and graces listener’s ears courtesy of Minaret Records, a California jazz label.

The record is accompanied by an album-length music video synthesized with GANce, containing a completely unique video for each track. These 93960 frames have been the ultimate goal of this project since it’s inception, and serve as the best demonstration as to what GANce can do. Within the video (linked below), the video for ‘buzzz’ is a personal favorite, demonstrating the three core elements of a projection file blend:

Continue reading →

GANce Overlays

This post is one in a series about GANce

As it stood, the three main features that would comprise the upcoming collaboration with Won Pound (slated for release mid-April) were:

  • Projection Files (using a styleGAN2 network to project each of the individual frames in a source video, resulting in a series of latent vectors that can be manipulated and fed back into the network to create synthetic videos)
  • Audio Blending (using alpha compositing to combine a frequency domain representation of an audio signal with a series of projected vectors)
  • Network Switching (feeding the same latent vector into multiple networks produced in the same training run, resulting in visually similar results)

As detailed in the previous post. The effect of these three features can be seen in this demo:

 

Knowing we had enough runway to add another large feature to the project, and feeling particularly inspired following a visit to Clifford Ross’ exhibit at the Portland Museum of Art, I began exploring the relationship between the projection source video and the output images synthesized by the network.

Continue reading →

Introducing GANce

This post is one in a series about GANce

In collaboration with Won Pound for his forthcoming album release via minaret records I was recently commissioned to lead an expedition into latent space, encountering intelligences of my own haphazard creation.

A word of warning:

This and subsequent posts as well as the GitHub etc. should be considered toy projects. Development thus far has been results-oriented, with my git HEAD following the confusing and exciting. The goal was to make interesting artistic assets for Won’s release, with as little bandwidth as possible devoted to overthinking the engineering side. This is a fun role-reversal, typically the things that leave my studio look more like brushes than paintings. In publishing this work, the expected outcome is also inverted from my typical desire to share engineering techniques and methods; I hope my sharing the results shifts your perspective on the possible ways to bushwhack through latent space.

So, with that out of the way the following post is a summary of development progress thus far. Here’s a demo:

There are a few repositories associated with this work:

  • GANce, the tool that creates the output images seen throughout this post.
  • Pitraiture, the utility to capture portraits for training.

If you’re uninterested in the hardware/software configurations for image capture and GPU work, you should skip to Synthesizing Images.

Continue reading →

Forcing a screen resolution of an Ubuntu guest OS in VirtualBox

I figured that doing this would be trivial but turns out it took a little work:

I’m trying to emulate an official 7″ Raspberry Pi Touch Display in a VM, so for this post the target resolution is 800 x 480. If you want to change it to another resolution swap in yours for the rest of this guide.

First, make sure Auto-resize Guest Display is deselected in Virtualbox:

Run the following command in your terminal:

The output should look something the the following, starting with Modeline

Copy the text after Modeline so in this case it would be

And paste it after the following command:

NOTE! You may want to change the 800x480_60.00 to something without an underscore in it, it was causing problems on my system. I changed it to pidisplay. The resulting command for this example is:

You should be able to run the above command without error. Next, run:

You’ll be greeted with output similar to this. Note the name of the display device, in this case VGA-1.

With that output name, enter the following two commands:

After running that second command, the window should jump to it’s new resolution! You’re done!

Automatically run Electron application at reboot on Raspberry Pi

Here is a quick  way to have an application built on electron run at boot on a Raspberry Pi. This worked for me running Raspian Stretch with Desktop.

Edit /home/pi/.config/lxsession/LXDE-pi/autostart with nano:

Add the following line:

The file should now look somewhat like this:

Save and exit nano and reboot. Your app should open after the desktop environment loads. Yay!

If you want to be able to get access to the terminal output of your application, install screen with:

And then swap:

For:

In the above code snippets.

After the pi boots, you can run screen -list to see what screens are available to attach to then attach to yours with screen -r yourscreen. Here’s an example:

Press enter, and then see your terminal output.
For more info on how to use screen, check out this link:

https://www.gnu.org/software/screen/manual/screen.html

Electron cannot be started from an SSH session

Update: If you run export DISPLAY=:0 in the terminal prior to npm start, the application runs just fine on the remote device. Thank you Alex!

https://twitter.com/alexbragdon/status/915601277752573954


In working on an project for work, I have figured out the hard way that Electron has to be started from a terminal session on your target device (ie the computer it is to be viewed on). I am developing an embedded system based on the Raspberry Pi that does not take user input but displays information on a screen.

Upon downloading the electron-quick-start example, everything installs correctly without error and can be done remotely via SSH. Upon running with npm start, the following error is thrown.

I spent most of the evening trying to debug npm ERR! code ELIFECYCLE to no avail. On a lark, I connected a keyboard to the device and ran npm start and it ran without error. Sigh.

The remote development alternative for doing this is to use Remote Desktop Connection a client comes bundled in with windows. The software can be installed on the remote system (the Raspberry Pi) using apt-get install xrdp. Upon connecting, opening up a shell in the RDP client, and running npm start, the example application works just fine.

StripPi – Software Demo, Roadkill Electronics

I’m constantly loosing the remote for my RGB LED strip lights, and I have a few days for spring break, time to get hacking. Here’s a demo and explanation video:

I don’t mention it in the video, but the cool part of this project is how the different processes communicate with each other. Rather than interacting with the different processes through pipes, or something like stdin, I’ve decided to use a TCP websocket server:

StripPi High Level Diagram

Processes on the device send RGB values to the Strip Server via a TCP packet. This very very easy to implement, and almost all of the hard work is taken care of via the socketserver  module included in python3. This also allows for interactions with that main process (the StripPi Server process) to take place off of the Raspberry Pi as well. I plan on writing an Alexa integration for this project moving forward, and this should make that a lot easier.

The analog to digital conversion is handled by an MCP3008, exactly the same way as I did it here.

Thanks for reading, more soon.

Raspberry Pi Digital Hourglass

Trying to get the most out of a day has been big theme of my life lately, as I’m sure it is for many people. I’ve found that I always manage my time better when things are urgent; I’m considerably more productive when I have to be.

I want an ascetically pleasing way to be able to represent how much time is left in the day at a granular scale, like an hourglass. Watching individual seconds disappear will look cool and (hopefully) create that sense of urgency that I want to induce.

Technically, this is a really simple thing to accomplish thanks to python and pygame. Here’s a video of a proof of concept running on my laptop:

At the start of each day, the display is filled with squares at random locations, with a random color. As each second elapses, a square will vanish.

To make it easier to see for the video, I’ve made the squares much bigger than they will actually be for the final build. This is what the display looks like with the squares at their actual size:

The code really really simple, like less than 100 lines simple. Here’s how it works:

Here’s the version of the code running on my computer in the video:

Let’s walk through some of the design decisions of this code. The first thing that’s worth talking about is how the data for the squares is handled:

It’s just an object with no methods, and on initialization, all the parameters of the square (location and color) are generated randomly as opposed to just floating the raw numbers in arrays around (even though that’s basically what is happening). This let’s us fill the squares array very easily later on in the file here:

and here:

When it comes time to draw these squares, it also makes that pretty intuitive:

Again, very simple stuff, but worth it to talk about.

I’ll be back at my place that has the Raspberry Pi and display I would like to use for this project, so more on this then.

Thanks for reading!

Creature Capture | Variable Video Capture Length Code & Testing, Frame Rate Issues

So I’ve been working a lot in the past day in ironing out part of the night side loop (loop 3 in this diagram). Basically, it starts recording based on an input from a sensor and continues to record until these inputs stop occurring.

My test code looks like this

The interesting functions at work here are the following:

FilmDurationTrigger() Takes the period of time that will be filmed, in this example, it’s 5 seconds just to conserve time, but in application it will be 20 seconds. This code will pause for the input time, and continue to be paused upon inputs from GetContinueTrigger(). This delay allows the code to continue filming until there are no inputs.

In this example, GetContinueTrigger() returns a Boolean if a random event occurs, but in application it will return a Boolean based on the status of a motion detector.

I ran two tests, both of them produced separate results. The first one created a 10 second long video:

And the second created a 15 second long video:

These two test shows that variable capture length functionality works! As a note, the actual times on the output video varies from the amount of time that it’s designed to record for. This is because the variable frame rate nature of the video coming out of the camera module, it causes the videos to come out a little short, but they still contain all the frames of the amount of time desired to record, just scaled slightly by frame rate error.