VisPy is a Python library for interactive scientific visualization that is designed to be fast, scalable, and easy to use.
While looking for a near real time data visualization alternative to the venerable matplotlib, I came across this jaw dropping demo:
Absolutely insane, achieving that kind of performance in python is amazing to say the least. This demo in particular seems like it would be more likely to come from a pygame application at the least, but looks more like it would be a Unity project.
The VisPy project is massive, but luckily, there is a set of really good examples included in the repo. Reminds me of the Arduino standard library in this way. After through all of running these, I didn’t find exactly what I was looking for.
For how simple the finished product looks, the learning curve on the way there was surprisingly steep. Hopefully this post saves you some time.
Bitbucket is great for hosting private git repos. Turns out, it can also be used to turn those repos into python packages that you can integrate into your projects with pip. This took a bit of trial and effort to make happen, let me know if there is anything additional you had to do to get things working on your end and I can add them to the guide.
Background
This whole process is built on pip’s ability to install packages from common VCS’s using SSH keys for access credentials. The syntax for doing that looks like this:
Cloning ssh://git@bitbucket.org/esologic/sample_project.git to ./pip-sjec1gbh-build
git@bitbucket.org:Permission denied(publickey).
fatal:Could notread from remote repository.
Please make sure you have the correct access rights
andthe repository exists.
Command"git clone -q ssh://git@bitbucket.org/esologic/sample_project.git /tmp/pip-sjec1gbh-build"failed with error code128inNone
No dice. It didn’t work because our development environment isn’t configured correctly. Let’s get started with the guide.
Using private repo packages locally
Note: I’m on ubuntu 18.04, but I will leave Windows notes in each step if applicable.
Step 1: Make sure your repo CAN be installed as a python package
The key here is a proper setup.py file. Here are best the best set of docs I’ve found on how to make this file.
You can also look at the test repo for this project (https://bitbucket.org/esologic/sample_project/src/master/), it contains an example setup.py. This repo will also be the standard example for this post.
To make sure things are working correctly, you can try installing the package into your local python environment, or into a virtual one like I’m doing. Using sample_project as an example, we can do this like so:
Ifthey can get you asking the wrong questions,they don'thave toworry about answers.
>>>
If your package behaves as expected when installed like this locally, you’re all set to push the changes to your bitbucket repo and continue with the rest of the guide.
Step 2: Create SSH keys and add them to bitbucket
Note: at a few places in this step I use my own email as a reference, dev@esologic.com. Make sure whenever you see that, to substitute email address associated with your bitbucket account.
If you already have ssh keys created on your computer or whatever you’re developing on, they should be located at ~/.ssh. If you don’t see both id_rsa and id_rsa.pub files in that directory, create them with:
1
ssh-keygen-mPEM-trsa-C"dev@esologic.com"
Leave passphrase blank.
Now, copy the contents of ~/.ssh/id_rsa.pub to bitbucket. The following images should walk you through the steps, make sure to give the key a memorable name.
Now, the ssh key of whatever dev environment you’re on is added to bitbucket.
Windows steps to create ssh keys
I followed these two (1, 2) guides to create ssh keys on windows.
The short version goes something like this:
1
2
3
4
$ssh-keygen-mPEM-trsa-C"dev@esologic.com"-Emd5
$cdC:\Users\dev\.ssh
$ssh-add id_rsa
$ssh-Tgit@bitbucket.org
Then follow the step above to add the keys to your bitbucket account.
Step 3: Make sure your account can read from the private repo with your python package
This is a simple, but a trap for young players. Make sure the account you’re trying to install the module with has at least read settings on the repo.
Since the Devon account is an owner of the repo, it will be allowed to read from the repo. The account ci_bot will also be able to read from the repo because it has read permissions.
Step 4: Install the package from bitbucket
With the bitbucket repo permissions set, and your SSH key added to your bitbucket account, you should be able to re-run the installation command from earlier and use the package.
Ifthey can get you asking the wrong questions,they don'thave toworry about answers.
>>>
Fantastic! Remember, your pip command git+ssh://git@bitbucket.org/esologic/sample_project.git will be different for your package. It will look something like this: git+ssh://git@bitbucket.org/{your username}/{your project}.git.
Any user that you give read permissions to on the repo will be able to install your package as well. This includes a machine user, so your CI builds can use your private package as well, which I’ll show you how to do next.
Using private repo packages in circleci
Bitbucket and circleci go together like peanut butter and chocolate. Adding CI to a bitbucket project is made fast and easy using circleci.
And then install your project like you did before. The package should install no problem, and you should see the same output as step 4.
Step 8: Set the $KEY environment variable in circleci
We now want to make the private key we made for our ci bot (~/.ssh/ci_bot_keys/id_rsa) available to the circle build process.
The only tricky part here is that the private key will contain newlines. For simplicity, we can replace them with underscores, and add the newlines back in the circle build.
Copy the output of this command to your clipboard:
The output ends after -----END RSA PRIVATE KEY-----_ in case your terminal doesn’t wrap correctly.
Now we need to set this value to the env var $KEY in the circleci build that we are trying to use our private package (sample_project) in.
Click the gear on the project page for your project in circle. For me, this brought me to https://circleci.com/bb/esologic/crossbow/edit, where crossbow is the name of my project.
Go to build settings -> Environment Variables and then set the variable like so:
Now that the variable is set, we need to change our circle config to use it.
Step 9: Add the step to your /.circleci/config.yml file
This does the same thing that we just tried locally, but in circle.
You have to make sure that the export GIT_SSH_COMMAND step happens in the same step as any pip commands. Your full dependencies installation circle step may look something like this:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
-run:
name:Install Dependencies
command:|
# Give us access to private repos
export KEY_PATH=tmp_id_rsa
echo-e"${KEY//_/\\n}">$KEY_PATH
chmod600$KEY_PATH
export SSH_AUTH_SOCK=none
export GIT_SSH_COMMAND='ssh -i $KEY_PATH'
python3-mvenv venv
.venv/bin/activate
pip install-r./requirements.txtts.txt
Make sure you select a circle image that has a git version of 2.17.0 or later, or this step will fail without an explanation. I found that the python image of circleci/python:3.7-buster worked when testing.
Try running your job, with this step added, it should be able to pull the package from your private repo. Let me know if you run into issues and I can try to help you out. Maybe donate the money you saved on hosting fees to me via paypal? 🤷💖
Check out this comment for some tweaks to this guide to support the latest version of the tools!
Panelization is the process of taking two or more PCB designs and combining them using tabs or v-scores that you would then separate into individual boards once they come back from manufacturing. It’s a way to get more than one design made in a single order.
There are a few forum posts or other snippets on how to accomplish this out there already, but not a real guide. For my own sake, this is how you can do this panelization using all free tools. Here are some photos of a board I had fabricated by OSH Park using this panelization method:
I implement this technique whenever I’m creating closely-related PCBs.
The design highlighted in this blog post is a transmitter/receiver pair, meaning that there would never be a transmitter without a receiver, or vice-versa.
Design is made simple by doing the layouts individually, and manufacturing is made simple by getting them made as a single board, not having to coordinate multiple orders. Let’s get started with the guide.
1. Download The Tools
You probably already have KiCAD. Next, make sure to download GerberPanelizer by This is not Rocket Science (site link) from GitHub. This guide uses the 2018-08-10 snapshot release.
2. Export your designs from KiCAD
Your designs have to be completely ready for production before starting this process. Components placed, tracks laid, zones poured etc. It is very “one-way” in that it is impossible to update an already panelized design once it has been exported.
Here’s one of the designs that will be added to the panel.
You’ll want to add a grid origin that is really close to your design. In KiCAD, select place → grid origin to do this. I am putting it in the top left hand corner of the board.
Grid origin placed
In pcbnew, select file → plot to adjust the gerber export settings.
Make sure Output directory is set to an empty directory somewhere on your disk. In this example, it’s set to tx-gerbers.
Check Use auxiliary axis as origin
Check Use Protel filename extensions
*Optional* Since I’m not using them in this design, I’ve unchecked F.Paste and B.Paste.
And then click Plot.
You should be greeted with a directory of files with dissimilar extensions:
These settings will automatically be set to match the previous export, but make sure the output folder and the drill origin match the previous settings. Mine looked like this:
Here is my resulting output directory with all of the files:
3. Modify the exported files
This step is weird. You need to change the extension of all .gm1 files to .gko. For this example, flail-tx-kicad-Edge_Cuts.gm1 needs to be renamed to flail-tx-kicad-Edge_Cuts.gko as this is what GerberPanelizer expects. Here is my resulting directory:
Notice the .gko file
4. Load the designs into Gerber Panelizer
Open up GerberPanelizer, you will be greeted with this screen:
Select file → new to create a new project. Next, select board placement → add gerber folder and navigate to the output folder from KiCAD. In this example, it was tx-gerbers.
You should be seeing something like this:
Where is the board?! Select board placement → autopack: native and your design will leap into view:
Now, re-do the guide up until this point for however many unique designs you want to add to this panel. If you want to duplicate your design multiple times in the same panel, you can add an instance by right clicking on the instance in the right hand view and then clicking add instance.
5. Arrange designs and add tabs
Since you’ve been hitting board placement → autopack: native after each board add, your designs should be properly arranged at this point. You can manually move the designs by clicking and dragging them, but I’ve found that using the autopack works really really well. Here’s what my design looks like at this point:
To join the designs together, you need to add breaktabs.
Select breaktabs → insert breaktab, and a small red circle will appear in the top left hand corner of the workspace:
Click and drag the tab between the two designs. Make sure black dots appear on either edge of the design:
Continue to add tabs in the same manner until the text turns a bright green color, this lets you know that the boards will be secured.
There is no way to automatically add the proper tabs, so make sure you use your best judgement.
Now we’re ready to export!
6. Export the panelized design
It’s a good idea to first save the design in GerberPanelizer so you can edit the layout later without having to start from scratch. Once you export the final merged gerber files, they cannot be edited or re-arranged. Select file →save as to save the project.
Now to export the gerbers.
Again, in GerberPanelizer, select file → export merged gerbers and choose an empty output directory. The directory has to be empty because you typically send a zip archive of all gerbers to the manufacturer to get made, and this zip archive should just include this export. You should see this window pop up:
The contents of the merged output directory should look like this:
The merged output directory will include several image renderings of your merged designs, this is a great first check to make sure that everything went well.
Looks good! However before you send any critical designs off for manufacturing it’s best practice to visually inspect the layers with a gerber viewer. Save the merged output directory as a .zip file.
7. Verify using GerbView
KiCAD ships with a program called GerbView to inspect gerber files. Open that gerbview and then open your zipped merged output directory with file → open zip archive file.
There will be an error message which you can ignore.
You should see something like this:
There’s the design as we expect it, you can uncheck the different layers on the right pane just like in pcbnew to inspect them one by one. I’ve uploaded this design to oshpark (a domestic PCB fab service) to see if their preview also looks correct and again, there are no problems.
You’re now ready to send your panelized designs out for manufacturing. Congrats!
8. Wrap up
Thanks for reading! Did this guide work for you? Let me know in the comments below this post.
Note: This is confirmed to work with KiCAD 4 and 5.
Ever wanted to have multiple different sound files playing on different output devices attached to a host computer? Say you’re writing a DJing application where you want one mix for headphones and one for the speakers. Or you’re doing some sort of kiosk or art installation where you have many sets of speakers that need to all be playing their own sound file but the whole thing needs to be synchronized. This would even be cool for something like an escape room.
The ladder example is where I needed this bit of code. I’ve been working with interdisciplinary artist Sara Dittrich on a few projects recently and she asked if I could come up with a way to play 8 different mono sound files on 8 different loudspeakers. Here’s a video of the whole setup in action, and an explanation of the project:
I’ve wrapped up all of the code for the art installation project, and that can be found in a github repo here. It includes the startup functionality etc. If you’re interested in recreating the video above, that repo would be a good starting place. The following is a list of the parts used to make that build happen:
It is worth it to give a simple example of how to play multiple files on multiple audio devices using python. I couldn’t find an examples on how to do this online and had to spend some time experimenting to make it all come together. Hopefully this saves you the trouble.
To install sounddevice on my Raspberry Pi, I had to run the following commands:
For this example, let’s say there are 4 audio files in the same directory as multi.py , so the directory looks like this:
1
2
3
4
5
6
multi_audio/
├── 1.wav
├── 2.wav
├── 3.wav
├── 4.wav
└── multi.py
The code is based on the sounddevice library for python, whose documentation is pretty sparse. This script will find the audio files, and then play them on as many devices as there are attached. For example, if you have 3 sound devices it will play 1.wav, 2.wav and 3.wav on devices 1-3. If you have any questions, feel free to ask:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
"""
multi.py, uses the sounddevice library to play multiple audio files to multiple output devices at the same time
Written by Devon Bray (dev@esologic.com)
"""
import sounddevice
import soundfile
import threading
import os
DATA_TYPE="float32"
def load_sound_file_into_memory(path):
"""
Get the in-memory version of a given path to a wav file
I work with addressable LEDs a lot. For all that they’re great for, they’re kind of hard to debug when you have a lot of them connected up at once. This is especially apparent when you have many small single modules in hard to reach spaces.
Here’s my solution:
This lets me set the color and number of LEDs in a strip, and then displays a color pattern. This way I can tell if an LED has become disconnected in a strip, or if a channel inside a particular has died.
Features
Select LED type with the type switch, 4 positions
Can test up to 400 LEDs at a time, if you can find a worthy power supply
3 Test modes
RGB – 1 second red, 1 second green, 1 second blue
HUE – Lock strip at HSV (x, 255, 255) and x loops from 0-255
WHTE – Set the strip to RGB(255, 255, 255)
Count and Mode are saved into eeprom, so you don’t have to keep resetting the strip if it powers off
Wall mount fittings
Design Explanation
All of the raw code solidworks, and KiCAD have been posted on my github. You can look at the 3D models on thingiverse as well.
Mechanical
Here are a couple of quick renders of the assembly design:
The screw mount behind the pushbuttons is extended to be able to support the pressure without flexing: I added a ridge so you can grab onto something as you interact with the switches / buttons.
Electronics
Here’s the circuit:
There really isn’t a lot going on here, the parts are probably the coolest part of the project. The 5V jack is a 6mm DC barrel jack, the pushbuttons are illuminated 16mm pushbuttons from adafruit, the on/off switch is a locking toggle switch, and the 4 position rotary switch can be found here.
I wired up the circuit on a spare piece of perfboard.
The LED driving part of the code is based on FastLED, a beautiful library for driving these types of addressable LEDs.
The rest of the code is mostly just a hardware UI problem, and isn’t all that interesting. LED count “ramps” as you hold the button down. The longer you hold the button, the faster the
Wrap up
That’s pretty much it! I’ve already gotten some use out of this tool and have found great satisfaction in taking the time to make it look nice as it will be a permanent addition to my lab.
I’ll post any updates I make to this project as edits to the top of this post.
Thanks for reading, and here are a few more photos:
Python uses a Global Interpreter Lock to make sure that memory shared between threads isn’t corrupted. This is a design choice of the language that has it’s pros and cons. One of these cons is that in multi-threaded applications where at least one thread applies a large load to the CPU, all other threads will slow down as well.
For multi-threaded Python applications that are at least somewhat time-sensitive, you should use Processes over Threads.
Experiment
I wrote a simple python script to show this phenomenon. Let’s take a look.
Increment Function
Python
1
2
3
4
5
6
7
defincrement(running_flag,count_value):
c=0
whileTrue:
ifnotrunning_flag.value:
break
count_value.value=c# setting a Value is atomic
c+=1
The core is this increment function. It takes in a Value and then sets it over and over, increment each loop, until the running_flag is set to false. The value of count_value is what is graphed later on, and is the measure of how fast things are going.
The other important bit is the load function:
Load Function
Python
1
2
3
4
5
6
defload(running_flag):
z=10
whileTrue:
ifnotrunning_flag.value:
break
z=z*z
Like increment, load is the target of a thread or process. The z variable quickly becomes large and computing the loop becomes difficult quickly.
The rest of the code is just a way to have different combinations of increment and load running at the same time for varying amounts of time.
Result
The graph really tells the story. Without the load thread running, the process and thread versions of increment run at essentially the same rate. When the load thread is running, increment in a thread grinds to a halt compared to the process which is unaffected.
That’s all! I’ve pasted the full source below so you can try the experiment yourself.
Full Source
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
frommultiprocessingimportProcess,Value
fromthreadingimportThread
fromtimeimportsleep
fromctypesimportc_bool,c_longdouble
defincrement(running_flag,count_value):
"""
Increment the value in count_value as quickly as possible. If running_flag is set to false, break out of the loop
:param running_flag: a multiprocessing.Value boolean
:param count_value: a multiprocessing.Value Long Double
"""
c=0
whileTrue:
ifnotrunning_flag.value:
break
count_value.value=c# setting a Value is atomic
c+=1
defload(running_flag):
"""
Apply a load to the CPU. If running_flag is set to false, break out of the loop
:param running_flag: a multiprocessing.Value boolean
"""
z=10
whileTrue:
ifnotrunning_flag.value:
break
z=z*z
defmct(target,flag,value):
"""
Returns a lambda that can be called to get a thread to increment a increment using a thread
Ever wanted to measure the frequency of a square wave using an Arduino? There are a couple of good solutions out of there to do this, but not all of them had the capacity to do multiple inputs. I couldn’t find this quickly so here is my solution.
Here’s the link to the code if you want to skip ahead. The code uses interrupts and doesn’t use any kind of delaying so it’s good for giant state-machine applications. My application for this is measuring signals from 10Hz-100Hz in which this can measure within 1% error. The absolute limits of the code are 1Hz-50KHz.
This project is on GitHub if you want to send a pull request to make improvements.
Setup
For testing, I wrote a simple function generator and uploaded it to a separate arduino. It outputs a pulse train with periods of 10ms (100Hz) and 5ms (200Hz) on pins 2 and 3. I attached LEDs and their resistors for debugging.
Pins 2 and 3 on the function generator to pins 2 and 3 on the frequency counter.
The code for this simple function generator is here:
This code will work fine in a stateless application, because there are no delay statements (which some other frequency counters I’ve seen online use). It’s a little bit complicated, send me a pull request if you can refactor it to be cleaner.
* I've written most of the important notes as comments in the source, but a couple more details:
*
* - The important data is stored in `period_averages_ms` and `frequency_averages_hz`. You address them using the indices defined at the top of the file. These arrays get updated each time `compute_counts()` is called. Keep it `compute_counts()` somewhere in the main() loop.
*
* - You could easily add more frequencies, you just have to `NUMSIGS`, make a specific ISR, and another `attachInterrupt` line in setup()
*
* - It uses [interrupts](https://playground.arduino.cc/Code/Interrupts) which might not be right for your proejct, but normally shouldn't get in the way of too much stuff.
*
* - If the ISR hasn't seen a new edge in 1000000us, both `period_averages_ms[p_index]` and `frequency_averages_hz[p_index]` will be set to zero!
* - This means that slowest frequency that this code can detect is 1hz!
*
*/
intfreq_pin_1=2;// the pin connected to the first signal, must be an interrupt pin! See the arduino docs
intfreq_pin_2=3;// the pin connected to the second signal, must be an interrupt pin! See the arduino docs
#define BUFFSIZE 100 // a rolling average of the frequency/period is computed, and this is the size of that buffer
#define NUMSIGS 2
#define FREQ1INDEX 0
#define FREQ2INDEX 1
volatileintperiod_buffer_indices[NUMSIGS]={0};// the location of the index for adding to the rolling buffer average
volatileunsignedlongperiod_buffers[NUMSIGS][BUFFSIZE]={0};// the buffers
volatileunsignedlongprevious_edge_times_us[NUMSIGS]={0};// the time that the previous edge came in in microseconds
volatilefloatperiod_averages_ms[NUMSIGS]={0};// the period time of a given signal in milliseconds
volatilefloatfrequency_averages_hz[NUMSIGS]={0};// the frequency of a given signal in hertz
volatileboolperiod_buffer_locked[NUMSIGS]={false};// spin locks for the different buffers
voidsetup(){
Serial.begin(9600);
// the pins must be mapped to their ISRs
pinMode(freq_pin_1,INPUT_PULLUP);
attachInterrupt(digitalPinToInterrupt(freq_pin_1),new_freq1_edge,RISING);// you could change this mode to whatever you were looking for, FALLING, CHANGE etc.
previous_edge_times_us[period_index]=current;// but make sure the new time is set because this operation is atomic
}
voidnew_freq1_edge(){
new_edge(FREQ1INDEX);
}
voidnew_freq2_edge(){
new_edge(FREQ2INDEX);
}
I’ve written most of the important notes as comments in the source, but a couple more details:
The important data is stored in period_averages_ms and frequency_averages_hz. You address them using the indices defined at the top of the file. Make sure you call compute_counts() before using this data. Keep it somewhere in main().
You could easily add more frequencies, you just have to NUMSIGS, make a specific ISR, and another attachInterrupt line in setup()
It uses interrupts which might not be right for your proejct, but normally shouldn’t get in the way of too much stuff.
If the ISR hasn’t seen a new edge in 1000000us, both period_averages_ms[p_index] and frequency_averages_hz[p_index] will be set to zero! This means that slowest frequency that this code can detect is 1Hz!
If you have any questions on how to add more signals, leave a comment!
Results
Here’s the output in the serial monitor attached to my function generator from earlier:
That’s like less than 1% error! Pretty good!
I also tested the code with a real function generator. Things worked really well until around 50KHz, so I would say that this code can’t be trusted past 50KHz.
I figured that doing this would be trivial but turns out it took a little work:
I’m trying to emulate an official 7″ Raspberry Pi Touch Display in a VM, so for this post the target resolution is 800 x 480. If you want to change it to another resolution swap in yours for the rest of this guide.
First, make sure Auto-resize Guest Display is deselected in Virtualbox:
Run the following command in your terminal:
Shell
1
cvt80048060
The output should look something the the following, starting with Modeline…
Copy the text after Modeline so in this case it would be
NOTE! You may want to change the 800x480_60.00 to something without an underscore in it, it was causing problems on my system. I changed it to pidisplay. The resulting command for this example is:
Save and exit nano and reboot. Your app should open after the desktop environment loads. Yay!
If you want to be able to get access to the terminal output of your application, install screen with:
Shell
1
sudo apt-getinstall screen
And then swap:
Shell
1
@npm start--prefix/path/to/project/
For:
Shell
1
@screen-d-mnpm start--prefix/path/to/project/
In the above code snippets.
After the pi boots, you can run screen -list to see what screens are available to attach to then attach to yours with screen -r yourscreen. Here’s an example:
Press enter, and then see your terminal output. For more info on how to use screen, check out this link:
For my senior thesis project at WPI, myself and two colleagues (Rachael Putnam – RBE/ME and Mead Landis – RBE/ME) designed a tree climbing robot. I was in charge of designing and implementing the electronics and controls software. I was the most intense project I have ever worked on, both in terms of difficulty and potential impact. Here is our poster for project presentation day:
Here’s a video of the prototype climbing:
We did a blog during the project, here is the best post I wrote:
