Yet Another Way to Draw Lines

Sure, why not, right?

One of the things I was trying to accomplish with the shaky drawing from my last post was to give the idea of hand-drawn lines. By adding in a bit of random variance, lines and shapes can seem like they are drawn by a human rather than perfectly rendered by a computer.

But at some point I was actually sketching something by hand and realized another big difference between computer drawing and human sketching. Very often when a person is sketching, rather than boldly drawing what they hope is a straight line from point A to B, they will instead make a number of very light lines that roughly go between the two points. You’ve seen it, you’ve done it, you know what I mean. So let’s replicate that.

The idea is that when we want to draw a line from x0, y0 to x1, y1 for example, we’ll actually draw multiple lines – not exactly to those to points, but from somewhere around the first point, to somewhere around the second one.

In this case, I’m going to dive right into a function. This was my first attempt:

function sketchLine(context, x0, y0, x1, y1, count, rand) {
  context.beginPath();
  for (let i = 0; i < count; i++) {
    let x = x0 + Math.random() * rand - rand / 2;
    let y = y0 + Math.random() * rand - rand / 2;
    context.moveTo(x, y);

    x = x1 + Math.random() * rand - rand / 2;
    y = y1 + Math.random() * rand - rand / 2;
    context.lineTo(x, y);
  }
  context.stroke();
}

We can then call this like so:

context.lineWidth = 0.2;
sketchLine(context, 100, 100, 700, 700, 5, 20);

Note that I set the line width down to 0.2 to make very light lines. We’ll let the multiple light lines build up to create the idea of a sketch, the same way you would do it by hand. So this will draw 5 lines with a variance of up to 10 pixels in any direction from the starting and ending points. The result:

Not that great, to be honest. The lines vary a bit, but they are still too uniform. I can bump up the randomness to, say, 50, but that looks even less like what a real person would sketch.

The thing with sketching, particularly in longer lines, is that you don’t always sketch the entire length of the line on each stroke. You might start around the first point and stroke maybe half way to the other point. And then you might stroke the middle third of the distance between them, and then something closer to the ending point. You’d do a bunch of strokes with varying start and end distances. Well, we can do something like that too.

function sketchLine(context, x0, y0, x1, y1, count, rand) {
  const dx = x1 - x0;
  const dy = y1 - y0;

  context.beginPath();
  for (let i = 0; i < count; i++) {
    let t0 = Math.random() * 0.5;
    let t1 = Math.random() * 0.5 + 0.5;
    
    let x = x0 + dx * t0 + Math.random() * rand - rand / 2;
    let y = y0 + dy * t0 + Math.random() * rand - rand / 2;
    context.moveTo(x, y);

    x = x0 + dx * t1 + Math.random() * rand - rand / 2;
    y = y0 + dy * t1 + Math.random() * rand - rand / 2;
    context.lineTo(x, y);
  }
  context.stroke();
}

This should look somewhat familiar if you’ve followed along with the other two posts in this series. First I get length of the line on the x and y axes – dx and dy. In the loop, I create a t value between 0 and 1. In this case, I create two of these – t0 and t1 – one for the start of the line and one for the end. t0 will range from 0.0 to 0.5 and t1 will go from 0.5 to 1.0. This should give us a random assortment of lines – some from very near the start point to very near the end point, some more in the first part of the line, some in the middle and some more near the end. Calling this again with:

context.lineWidth = 0.2;
sketchLine(context, 100, 100, 700, 700, 5, 20);

We get this image:

That’s something I could almost start to believe was done by hand. Let’s do a rectangle function!

function sketchRect(context, x, y, w, h, count, rand) {
  sketchLine(context, x, y, x + w, y, count, rand);
  sketchLine(context, x + w, y, x + w, y + h, count, rand);
  sketchLine(context, x + w, y + h, x, y + h, count, rand);
  sketchLine(context, x, y + h, x, y, count, rand);
}

Obviously, I just copied and pasted the shakyRect from the previous post and changed the method names and parameters. So let’s call this with:

context.lineWidth = 0.2;
sketchRect(context, 100, 100, 600, 600, 5, 20);

Pretty cool, but this brings up an issue we couldn’t immediately see when drawing a single line. All too often, the lines don’t reach all the way to those starting or ending points. Too many are falling right in the middle. We can shift our random parameters a bit to make up for that.

function sketchLine(context, x0, y0, x1, y1, count, rand) {
  const dx = x1 - x0;
  const dy = y1 - y0;

  context.beginPath();
  for (let i = 0; i < count; i++) {
    let t0 = Math.random() * 0.4 - 0.1;
    let t1 = Math.random() * 0.4 + 0.7;
    
    let x = x0 + dx * t0 + Math.random() * rand - rand / 2;
    let y = y0 + dy * t0 + Math.random() * rand - rand / 2;
    context.moveTo(x, y);

    x = x0 + dx * t1 + Math.random() * rand - rand / 2;
    y = y0 + dy * t1 + Math.random() * rand - rand / 2;
    context.lineTo(x, y);
  }
  context.stroke();
}

Now, t0 will range from -0.1 to +0.3, and t1 will go from 0.7 to 1.1. This makes it more likely that some lines will make it closer to one of the points, and may even overshoot it a bit, which is just what you’d do by hand. The results of this change:

Not bad, in my opinion. You’ll still have some rectangles with open corners, but not as often as before. Here’s one with a count of 20 and a rand of 40:

Finally, just for the heck of it, I combined the shaky and sketchy techniques with this function:

function sketchLine(context, x0, y0, x1, y1, count, rand) {
  const dx = x1 - x0;
  const dy = y1 - y0;

  context.beginPath();
  for (let i = 0; i < count; i++) {
    let t0 = Math.random() * 0.4 - 0.1;
    let t1 = Math.random() * 0.4 + 0.7;
    
    let xA = x0 + dx * t0 + Math.random() * rand - rand / 2;
    let yA = y0 + dy * t0 + Math.random() * rand - rand / 2;
    let xB = x0 + dx * t1 + Math.random() * rand - rand / 2;
    let yB = y0 + dy * t1 + Math.random() * rand - rand / 2;
    shakyLine(context, xA, yA, xB, yB, 20, 2);
  }
  context.stroke();
}

Here, I calculated the start and end points of each sketched line and rather than using moveTo and lineTo I called the shakyLine function from my last post. I hard coded it with a res of 20 and a rand of 2, which makes it just a little bit less than a perfectly straight line and possibly a little bit more like something hand drawn. Called with:

context.lineWidth = 0.2;
sketchRect(context, 100, 100, 600, 600, 10, 20);

it gives us:

Even that might be too much shake, but you can easily make it more subtle.

Here’s one final example. If this doesn’t look hand sketched, I don’t know what does.

Well, that’s that. I don’t have a ready-made library to share with you on this one, but hopefully you’ll find this useful or at least inspiring.

The rest of the How to Draw a Line series:

1. How to Draw a Line

2. More Ways to Draw a Line

3. Yet Another Way to Draw Lines

More Ways to Draw Lines

In the last post, I discussed an alternate way of rendering lines, that could give a glitchy look, painterly look, or many other looks based on what parameters you used. In this post, I’ll discuss another way to draw lines.

The theme to these posts is that using moveTo / drawTo is fine but gives you clinical, antiseptic, perfect straight lines (or curves). And sometimes it’s nice to shake things up a bit. In fact, shaking things up is exactly what we’re going to do here today.

I’m going to start with something very similar to one of the earlier examples in the last post, but instead of drawing individual pixels along the length of the line, I’m going to draw short line segments.

const dx = x1 - x0;
const dy = y1 - y0;
const dist = Math.sqrt(dx * dx + dy * dy);
const res = 10;

context.beginPath();
context.moveTo(x0, y0);

for (let i = res; i < dist; i += res) {
  let t = i / dist;
  let x = x0 + dx * t;
  let y = y0 + dy * t;
  context.lineTo(x, y);
}
context.lineTo(x1, y1);
context.stroke()

Like before, I’m starting with two points defined by x0, y0 and x1, y1. And I’ve defined a res variable that dictates the length of those intermediate segments. Note that I moveTo the first point and set i equal to res to initialize the loop, and then lineTo the last point. This ensures that the start and end of the line remain constant (which will be important later). Otherwise, things should be familiar here if you read the previous post. Here’s what this gives us:

It’s exactly what you’d get if you just drew a line from the first to last point. So big deal. But now that we have these intermediate points, we can shake things up as promised.

I’ll just randomly shift each x, y point a bit before drawing to it. (Just showing the for loop here:

for (let i = res; i < dist; i += res) {
  let t = i / dist;
  let x = x0 + dx * t;
  let y = y0 + dy * t;
  x += Math.random() * 4 - 2;
  y += Math.random() * 4 - 2;
  context.lineTo(x, y);
}

This puts each x, y location somewhere from -2 to +2 on each axis. And gives us this:

I then put this all into a function to make it easily reusable:

function shakyLine(context, x0, y0, x1, y1, res, rand) {
  const dx = x1 - x0;
  const dy = y1 - y0;
  const dist = Math.sqrt(dx * dx + dy * dy);

  context.beginPath();
  context.moveTo(x0, y0);

  for (let i = res; i < dist; i += res) {
    let t = i / dist;
    let x = x0 + dx * t;
    let y = y0 + dy * t;
    x += Math.random() * rand - rand / 2;
    y += Math.random() * rand - rand / 2;
    context.lineTo(x, y);
  }
  context.lineTo(x1, y1);
  context.stroke();
}

Now I can just call it like so:

shakyLine(context, 100, 100, 700, 700, 10, 10);

And it creates a line like this:

Note that the random factor is 10 here, so it’s a lot more shaky than the earlier example. Note that the res and rand parameters both contribute in different ways to the shakiness of the line. So if I move res down to 5, but keep rand at 10, we get something more shaky, but in a tight way.

But here, I’ve bumped both res and rand up significantly:

shakyLine(context, 100, 100, 700, 700, 50, 40);

This gives us a line that varies a lot more, but because each segment is longer, it’s more of a chunky random shake.

So you can play these two parameters off each other to get various effects.

The next thing to do is create a shakyRect function. It looks like this:

function shakyRect(context, x, y, w, h, res, rand) {
  shakyLine(context, x, y, x + w, y, res, rand);
  shakyLine(context, x + w, y, x + w, y + h, res, rand);
  shakyLine(context, x + w, y + h, x, y + h, res, rand);
  shakyLine(context, x, y + h, x, y, res, rand);
}

As mentioned before, the fact that each line starts and ends on a non-shaky point means that the rectangle will be continuous. You can call it like this:

shakyRect(context, 100, 100, 600, 600, 10, 10);

And get a rectangle like this:

To show how res and rand relate, I made this for loop that draws a bunch of squares on the canvas. res increases from left to right and rand increases from top to bottom. It gives you a good idea of the different effects you can create.

for (let y = 20; y < 730; y += 70) {
  for (let x = 20; x < 730; x += 70) {
    shakyRect(context, x, y, 50, 50, x/730 * 10, y/730 * 10);
  }
}

Now, this rectangle function isn’t the best because it doesn’t support fills, but you can probably work up a better solution. You might also want to create functions for drawing other shaky shapes.

Or, maybe you just want a drop in library you can use. I created a whole JavaScript shaky library about 7 years ago. I can’t guarantee how well it has stood the test of time. There are undoubtedly some best practices in there that could be updated. But the core code should be good enough to work out things for yourself. It supports not only lines and rectangles, but circles, arcs, quadratic curves, bezier curves, and arcs. And as of this writing, it has 101 stars, which is kind of neat!

https://github.com/bit101/shaky

All the posts in the How to Draw a Line series:

1. How to Draw a Line

2. More Ways to Draw a Line

3. Yet Another Way to Draw Lines

How to Draw a Line

… in code of course.

First, I’m going to assume that you are working in some kind of system that has a drawing API. But wait, even the simplest of drawing APIs have a function to draw a line already. All right then, we’ll start with that. Assuming you are using something like HTML, JS and Canvas, you’re going to do something like this:

context.moveTo(x0, y0);
context.lineTo(x1, y1);
context.stroke();

You start with two points, defined by two x, y coordinates. You move to the first one, you line to the second one, you stroke that path. And you wind up with:

Easy. And Boring. Sure you can change the width and the color and transparency, but it’s still just a boring line. In order to do a bit more with it, let’s abandon the built-in line functionality and make our own.

The idea is to start at the first point and draw a series of small squares between it and the second point. We’ll use the Pythagorean theorem to get the distance between the two points. That will let us know how many steps we have to take. Then just linearly interpolate between the x and y positions and draw a square there.

const dx = x1 - x0;
const dy = y1 - y0;
const dist = Math.sqrt(dx * dx + dy * dy);

for (let i = 0; i < dist; i++) {
  let t = i / dist;
  let x = x0 + dx * t;
  let y = y0 + dy * t;
  context.fillRect(x - 0.5, y - 0.5, 1, 1);
}

And the result of that:

Not half bad really. Of course, if you want to have a really fast and more accurate line drawing routine, you’d probably want to go with Bresenham’s algorithm or maybe Wu’s algorithm. Honestly, I’m really just referencing them so people don’t comment to correct me and tell me I should be using them. But we’re just getting started here. This still isn’t any more interesting than the built-in line function. It’s just too straight, too even, to regular. Let’s mix it up a bit.

We can start by changing the increment from i++ to, say, i += 5, and get:

Now we have some space between each “pixel” that makes up the line. We can then increase the size of these “pixels”:

context.fillRect(x - 1.5, y - 1.5, 3, 3);

Which has the general effect of pixel art:

In the next example, I changed the increment to 8 and used the following code to draw a circle instead of a square:

context.beginPath();
context.arc(x, y, 4, 0, Math.PI * 2);
context.fill();

And got:

You can keep playing with this, making lines out of any shapes with any amount of spacing. But let’s throw in some randomness.

Instead of drawing a pixel at every single point along the line, let’s draw them at random positions along the line. We can do that just by making t a random value from 0 to 1. Altering the first version, we get:

for (let i = 0; i < dist; i++){
  let t = Math.random();
  let x = x0 + dx * t;
  let y = y0 + dy * t;
  context.fillRect(x - 0.5, y - 0.5, 1, 1);
}

And an image that will look something like this:

Now we’re getting something a bit more interesting.

Let’s play with size and transparency next. I’ll make a size variable that will let us easily change the size of each pixel. And then set the transparency down real low. This will let the pixels overlap and build up more gradually in different areas. I’ll also use dist*2 as the pixel count, which gives us a lot more pixels to play with.

const size = 4;
context.fillStyle = "rgba(0, 0, 0, 0.05)";

for (let i = 0; i < dist*2; i++){
  let t = Math.random();
  let x = x0 + dx * t;
  let y = y0 + dy * t;
  context.fillRect(x - size / 2, y - size / 2, size, size);
}

And that gives us:

Finally, lets go all out, add even more pixels, vary the size, vary the transparency, even vary the position of each individual pixel.

for (let i = 0; i < dist*4; i++){
  let t = Math.random();
  let x = x0 + dx * t;
  let y = y0 + dy * t;
  x += Math.random() * 4 - 2;
  y += Math.random() * 4 - 2;
  let size = Math.random() * 8;
  context.fillStyle = "rgba(0,0,0, " + Math.random()*0.2 + ")";
  context.fillRect(x - size / 2, y - size / 2, size, size);
}

Now we have something that looks kind of like ink on wet paper.

But we don’t have to limit it to straight lines. Here’s the built-in code for drawing a quadratic curve – defined by three points:

const x0 = 100;
const y0 = 700;
const x1 = 400;
const y1 = -100;
const x2 = 700;
const y2 = 700;

context.beginPath();
context.moveTo(x0, y0);
context.quadraticCurveTo(x1, y1, x2, y2);
context.stroke()

This gives you a bland curve:

But we can apply the same concept here. We’ll need a function that gives us a point somewhere along the curve. Fortunately, such a function is well known. We can again choose a random t value and start drawing random pixels. Here’s the code – note I chose an arbitrary value of 3000 for number of pixels. Use more or less to get a denser or lighter line.

for (let i = 0; i < 3000; i++){
  let t = Math.random();
  let p = quadraticPoint(x0, y0, x1, y1, x2, y2, t);
  x = p.x + Math.random() * 4 - 2;
  y = p.y + Math.random() * 4 - 2;
  let size = Math.random() * 8;
  context.fillStyle = "rgba(0,0,0, " + Math.random()*0.2 + ")";
  context.fillRect(x - size / 2, y - size / 2, size, size);
}

function quadraticPoint(x0, y0, x1, y1, x2, y2, t) {
  const oneMinusT = 1.0 - t;
  const m0 = oneMinusT * oneMinusT;
  const m1 = 2.0 * oneMinusT * t;
  const m2 = t * t;
  return {
    x: m0 * x0 + m1 * x1 + m2 * x2,
    y: m0 * y0 + m1 * y1 + m2 * y2,
  }
}

And the curve:

This looks even better drawn with circles:

This looks interesting all by itself, but I suggest you create some parameterized functions for drawing lines, curves and shapes like this and see if it doesn’t change the overall look of your creative pieces.

The whole series of How to Draw a Line:

1. How to Draw a Line

2. More Ways to Draw a Line

3. Yet Another Way to Draw Lines

Raspberry Pi Pico – What I’ve Learned

So the new Raspberry Pi Pico board came out a few weeks back and there’s all kinds of news about it. For the uninformed, it’s a shift from other RPi boards. Most Pis are single board computers. They run an operating system – usually some Linux-based variant. The Pico is a microprocessor. Basically an Arduino alternative. But a pretty damn powerful one. Two cores, fast, good memory, 26 gpio pins, analogue to digital converter, real time clock, temperature sensor, etc. All for $4. So I grabbed a couple. By the way, if you’re ordering on line, make sure that you buy the header pins. They are not always included by default.

There are a ton of tutorials on line already, most of which are nearly exact clones of each other: solder pins on, plug in it while holding down the bootsel button, drag the micropython uf2 file onto the mounted drive, it will reboot, install Thonny, write a script to make an LED blink. So I’m not going to go through all that. Or you could say I just went through all that in one sentence.

One thing I highly recommend is the “Get Started with MicroPython on Raspberry Pi Pico” book put out by the Pi people themselves. It’s a bit cartoony and starts off really basic. In fact, going all the way through chapter 4 will get you through what most of the tutorials on line cover. But with a lot more depth. There’s several pages on how to solder the pins on. But further chapters get into some pretty good stuff, including various sensors and controls and I2C and SPI control of an LCD panel. Good starter stuff.

Stuff I’ve learned

Ran into lots of snags going through the process of learning this board and figuring out what I even want to do with it. A lot of this has to do with the fact that it’s really in its infancy. I’m sure that things will get better as time goes on, but there are a lot of rough edges right now. And beyond the book, there’s very little searchable info out there. Unfortunately, there are other boards/technologies out there named “pico” so that clouds your results. Throw in “Rapsberry Pi” into the search and you’re mostly going to get other RPi stuff. But even when you craft a good search, mostly what you’re going to find is the multiple cloned tutorials mentioned above. There is also a scattering of C/C++ tutorials and resources for the Pico. They look dauntingly complex so I have not dived into those yet.

Another problem is that the version of MicroPython that was made for the Pico is a fork of the official version. And it’s very definitely a subset. The official MicroPython documentation is fantastic, but huge swaths of it are just inapplicable to the version that works with the Pico.

For example, MicroPython has a machine module. In the official version, machine contains the following classes:

class Pin – control I/O pins
class Signal – control and sense external I/O devices
class ADC – analog to digital conversion
class UART – duplex serial communication bus
class SPI – a Serial Peripheral Interface bus protocol (master side)
class I2C – a two-wire serial protocol
class RTC – real time clock
class Timer – control hardware timers
class WDT – watchdog timer
class SD – secure digital memory card (cc3200 port only)
class SDCard – secure digital memory card

The ones struck through are not available on the Pico right now. Since the Pico does not have an SD card, the biggest miss there is the RTC class – the Pico has a real time clock, but no way to access it directly. That doesn’t seem too bad, but it extends from there. The Pin class is missing almost half of the methods on the Pico version of MicroPython. Other classes are missing methods as well, and there are several standard Python modules that are part of MicroPython that are missing from the Pico’s version.

All that said though, there is enough there to get you started on most common projects, and as I said, I’m sure this will grow and become more expansive in the coming months.

Tips and Tricks

A few random tricks and tips for using Micropython on the Pico:

When you install the MicroPython editor, Thonny, it will ask you if you want to run in regular mode or in Raspberry Pi mode. Naturally, I chose Pi mode. And naturally I was wrong. Although it seems to work fine, you wind up missing a lot from the UI.

Here’s the Pi mode:


Thonny in RaspberryPi mode, aka “simple”

And here it is in regular mode:

Thonny in regular mode

I was running in Pi mode for a while and thought it was pretty damn lame. Then I discovered you could go into the options and change it to regular mode. (Also that Pi mode is really called “simple” mode, and there is also an “expert” mode.)

Regular mode gives you full menus, which opens up a massive wealth of features I did not realize even existed. For example, I could find no way to copy files onto the Pico other than opening them up in Thonny and then saving them to the device. But the View menu lets you open a files panel (and a lot more) where you can access files both on your device and your local file system, and “upload” and “download” between the two. Brilliant.

Another gotcha from the book. The book states that by default, gpio input pins are set to be pulldown by default. Pulldown inputs are attached to ground with a resistor and connecting them to positive voltage triggers them. Pullup inputs are connected to a positive voltage with a resistor and connecting them to ground triggers them. If you didn’t pull them one way or the other, stray charges could trigger the inputs. Anyway, the book says that input pins are pulldown and you can create a pin by typing:

pin = machine.Pin(15, machine.Pin.IN)

I did this while testing physical buttons to make an LED turn on. And I was getting all kinds of crazy behavior. If I touched a wire or even just tilted the whole breadboard in a certain direction, the LED was coming on as if I’d pressed the button. I honestly thought that I’d accidentally tapped into some internal accelerometer at one point – I was totally able to control the LED by tilting the board back and forth. After looking up the official MicroPython docs for creating a pin, I found there was a third parameter for controlling pullup/pulldown. So I changed it to:

pin = machine.Pin(15, machine.Pin.IN, machine.Pin.PULL_DOWN)

When I did that – explicitly setting the pin to be pulldown – it was rock solid and worked 100% as expected.

Another question I finally answered for myself is how to have a program run as soon as the Pico boots up. Simple enough – you just name the Python script main.py.

An Alternative Python

There is an alternative to MicroPython though, and that is CircuitPython. As I understand it, CircuitPython is another fork of MicroPython and it is supported on a multitude of microprocessor boards, including the Pico. Because it’s been around a lot longer the the Pico, it is way more expansive in what it supports and has a ton (like 280+) libraries that work with it. CircuitPython is supported by Adafruit, and they’ve also created some really useful libraries, many specifically designed to work with the hardware they sell. So all in all, this is a really great option.

You install CircuitPython the same way you flash the Pico with MicroPython – download a uf2 file, start the Pico while holding down the bootsel button and dragging the uf2 to the mounted drive. It will reboot and be a CircuitPython device now.

One thing to say about CircuitPython is that it is quite different than MicroPython. Although it may be a fork, they forked the hell out of it.

For example, a simple program to blink an LED in MicroPython:

import machine
import utime

led = machine.Pin(25, machine.Pin.OUT, machine.Pin.PULL_DOWN)

while True:
    led.value(1)
    utime.sleep(0.5)
    led.value(0)
    utime.sleep(0.5)

And the same program in CircuitPython:

import board
import digitalio
import time
 
led = digitalio.DigitalInOut(board.D13)
led.direction = digitalio.Direction.OUTPUT
 
while True:
    led.value = True
    time.sleep(0.5)
    led.value = False
    time.sleep(0.5)

Not a huge deal, but I think it might be a good idea to choose one and roll with it. You’re not going to be able to run your MicroPython projects on your Pico after you flash it to CircuitPython (and vice versa – though you can simply reflash between the two systems relatively easily).

Personally, I’m torn at the moment. CircuitPython has way more stuff in it and so many existing libraries. But it’s so far different from the standard language, and I’m hoping that the Pi folks will continue to make their implementation more expansive and people will come up with new libraries for it as well.

On the other hand, CircuitPython already has some libraries allowing the Pico to act as an HID device. In other words, you can use it to send keyboard and mouse events to the host computer it’s connected to via USB, good for making custom keyboards or other controllers. And with 26 gpio pins, this winds up being way more useful for complex projects like this than the Arduino Pro Micro clones that I’ve been using. I haven’t found anything in MicroPython to do the same thing, though there is some C++ code out there that seems to support this. I may just end up using both MicroPython and CircuitPython for a while.

More about CircuitPython – the recommended editor is not Thonny, but the Mu Editor. For the most part it seems pretty basic, more like the simple mode of Thonny. No menus or extra panels or anything like the regular mode of Thonny – at least not that I could find. But as an editor, it does have some nice autocompletion and other features that I didn’t see in Thonny.

Something that confused me about Mu: I created a new file and saved it to the Pico and hit the run button, but the other default code.py hello-world type file ran instead. Nothing I could do would make my new file run. It turns out that’s by design. As mentioned, MicroPython devices will auto-run main.py on boot. And CircuitPython will auto-run either code.py or main.py on boot. But Thonny will run whatever active script you have open when you hit run, whereas Mu will always run code.py (or presumably main.py) when you hit run. This can be good or bad. It mimics what will happen when the device boots, but makes it harder to test some other script. The solution recommended is to have code.py or main.py be a simple launcher script that in turn executes some other script. Anyway, good to know.

Another big plus about CircuitPython is that when the device is connected to your computer it also mounts as an external drive. So if you want to add additional libraries to it, you can just drag and drop them to that drive. A bit easier than Thonny’s upload feature.

Summary

So that’s some of what I’ve managed to learn so far. As there is very little data out there at the moment, I hope some of this helps people just getting into this new, fun little board.

Edits/Additions

In the first version of this posts I used the word “robust” several times, indicating that the version of MicroPython for the Pico was not as robust as it hopefully would be in the future and that CircuitPython was more robust. That was the wrong choice of word. Robust in the context of code means that it is able to handle and recover from various unexpected conditions. To say that something is not robust implies that it is unstable. That’s not what I meant to imply.

I changed the word to “expansive”. My intention was to say that the current version of MicroPython on the Pico does not support as many features as it hopefully will and to say that CircuitPython supports more features. Examples are, as I described earlier, the missing RTC class for the real time clock that is in standard MicroPython but not on the Pico. Also, stuff like HID support in the usb_hid library of CircuitPython, but nowhere to be seen in MicroPython (that I can find), as well as all kinds of libraries for working with specific sensors and peripherals that are available for CircuitPython. MicroPython has some of that, but nowhere near what I see in CircuitPython.

Probability #2

The Monty Hall Problem

You’ve surely heard of this one. It’s a really tough one to wrap your head around. I’ll describe the problem in order to solve it, but it’s worth looking into the background of the “Ask Marilyn” incident if you’re not familiar with it.

Behind Monty Hall’s Doors

The story shows that it’s not just hacks like myself that find this stuff non-intuitive. Lots of highly educated, very smart people got tricked by this one.

The problem is based on an old game show, “Let’s Make a Deal”. The host of the show was named Monty Hall. The contestant is shown three doors. Behind two doors are goats. Behind one is A NEW CAR!!!

The contestant chooses a door. If he chooses the one with THE NEW CAR, he gets to keep the car. If he chooses a goat door, he gets a goat. The unspoken assumption being that he’d rather have a car.

But… before the chosen prize is revealed, Monty Hall opens one of the two remaining doors, revealing a goat!

There are now two doors remaining. The one that the contestant chose, and one other one. Monty asks the contestant if he wants to stay with his choice, or if he’d like to switch to the other door.

What should he do?

To clarify, what should he do in order to optimize his chances of winning THE NEW CAR? Are the odds of winning the car better if he sticks with his original choice? Are they better if he switches? Does it just not matter?

Intuition tells us that it doesn’t matter. There are two doors. One has a goat, one has a car. The odds are 50/50. Doesn’t really matter if he sticks with the first choice or switches. The odds are the same.

The truth though? He should absolutely switch. It will significantly improve his odds of winning. In fact, he’s twice as likely to win if he switches than if he sticks with his original choice.

But why???

If you want a deep explanation… I mean really deep, just head over to the Monty Hall Problem page on wikipedia. But I’ll give my take on it.

When the contestant first chose a door, he had a 1 in 3 chance of winning the car. That’s pretty straightforward.

When Monty opens a door, he is not randomly opening a door. He’s opening a door with a goat behind it, always. So he is using his knowledge of what’s behind each door to change the nature of the game.

Let’s say we have doors A, B, and C. A and B have goats, C has a car.

There are three possible scenarios:

  1. Contestant chooses door A. Monty will open door B. If contestant switches doors, he wins.
  2. Contestant chooses door B. Monty opens A. If contestant switches, he wins.
  3. Contestant chooses door C. Monty opens either A or B. It doesn’t matter. If contestant switches, he loses.

So in two of the three scenarios, switching is the winning move.

Even though I’ve worked out the logic of this myself many times and done it in code more than once, it still feels vaguely magical.

Bring on the Code

Let’s start by playing the game 1000 times in a for loop. And we’ll start by setting up the three doors with goats behind them and swapping out a car on one of them.

// we'll play the game 1000 times.
for (let i = 0; i < 1000; i++) {
  // create 3 doors with goats behind them.
  let doors = ["goat", "goat", "goat"];

  // randomly choose one door and put a car behind it.
  let winner = Math.floor(Math.random() * 3);
  doors[winner] = "car";
  console.log(doors);
}

This outputs the 1000 random door configurations. Something like this:

[ 'car', 'goat', 'goat' ]
[ 'car', 'goat', 'goat' ]
[ 'goat', 'car', 'goat' ]
[ 'car', 'goat', 'goat' ]
[ 'goat', 'car', 'goat' ]
[ 'car', 'goat', 'goat' ]
[ 'goat', 'car', 'goat' ]
[ 'car', 'goat', 'goat' ]
[ 'car', 'goat', 'goat' ]
...

Now let’s have the contestant make a choice – 0, 1 or 2. Just to check ourselves, let’s see whether he wins a car and add up how many times he wins. Out of 1000 games, he should win somewhere close to 333.

let wins = 0;

// we'll play the game 1000 times.
for (let i = 0; i < 1000; i++) {
  // create 3 doors with goats behind them.
  let doors = ["goat", "goat", "goat"];
 
  // randomly choose one door and put a car behind it.
  let winner = Math.floor(Math.random() * 3);
  doors[winner] = "car";
  // console.log(doors);
 
  // choose a door and count how many wins we get
  let choice = Math.floor(Math.random() * 3);
  if (doors[choice] === "car") {
    wins++;
  }
}
// should be around 333
console.log(wins);

That checks out for me.

Next, we let Monty open a door to reveal a goat. There are probably plenty of clever ways to code this, but I’ll just loop through the three doors and choose one that is not the same as what the contestant chose, and make sure that it’s a goat door.

  ...
  // now monty chooses another door (must be a goat!)
  let montyChoice;
  for (let j = 0; j < 3; j++) {
    if (j != choice && doors[j] === "goat") {
      montyChoice = j;
      break;
    }
  }
  console.log(doors[montyChoice]);
  ...

Just to check myself, I logged what’s behind Monty’s door. Sure enough, nothing but goats.

Now we have the contestant’s choice and Monty’s choice. We can add the code back in that calculates how many times the contestant wins the car…

let wins = 0;
// we'll play the game 1000 times.
for (let i = 0; i < 1000; i++) {
  // create 3 doors with goats behind them.
  let doors = ["goat", "goat", "goat"];

  // randomly choose one door and put a car behind it.
  let winner = Math.floor(Math.random() * 3);
  doors[winner] = "car";
  // console.log(doors);

  // choose a door and count how many wins we get
  let choice = Math.floor(Math.random() * 3);

  // now monty chooses another door (must be a goat!)
  let montyChoice;
  for (let j = 0; j < 3; j++) {
    if (j != choice && doors[j] === "goat") {
      montyChoice = j;
      break;
    }
  }
  // console.log(doors[montyChoice]);
 
  // contestant does not switch
  if (doors[choice] === "car") {
    wins++;
  }
}
console.log(wins);

This isn’t any different than the first time we counted the wins. I consistently get numbers in the low 300s. Of course, because the fact that Monty opens a door doesn’t change the fact that there was a 1 in 3 chance of the contestant winning.

But let’s see what happens if the contestant switches doors. Again, you can get fancy here, but I’ll go brute force, looping through the doors till I find the one that is neither the contestant’s choice nor Monty’s choice. And I’ll count how many times he wins with that choice.

let wins = 0;
// we'll play the game 1000 times.
for (let i = 0; i < 1000; i++) {
  // create 3 doors with goats behind them.
  let doors = ["goat", "goat", "goat"];

  // randomly choose one door and put a car behind it.
  let winner = Math.floor(Math.random() * 3);
  doors[winner] = "car";
  // console.log(doors);

  // choose a door and count how many wins we get
  let choice = Math.floor(Math.random() * 3);

  // now monty chooses another door (must be a goat!)
  let montyChoice;
  for (let j = 0; j < 3; j++) {
    if (j != choice && doors[j] === "goat") {
      montyChoice = j;
      break;
    }
  }
  // console.log(doors[montyChoice]);
 
  // contestant does not switch
  // if (doors[choice] === "car") {
  //   wins++;
  // }
 
  // contestant switches
  let newChoice;
  for (let j = 0; j < 3; j++) {
    if (j != choice && j != montyChoice) {
      newChoice = j;
      break;
    }
  }
  if (doors[newChoice] === "car") {
    wins++;
  }
}
console.log(wins);

Running this I get numbers in the upper 600s! Winning roughly 2 out of 3 times, exactly as predicted.

Summary

So there you go. The code proves things out. But it still feels a bit magical.

I did think of another though experiment that helps to make it make sense. Let’s say there were four doors – three goats, one NEW CAR. Contestant chooses one, and Monty Hall opens two of them to reveal two goats. Or better yet, there are ten doors. Contestant chooses one and Monty opens eight. The only way he’d lose by switching is if he had chosen the car to begin with. And there was only a 1 in 10 chance he did that. So if he switches now, he’s got a 9 out of 10 chance of winning! That helps my brain a bit. But still seems a bit magical.

Probability #1

I started re-reading The Drunkard’s Walk again today.

I read it a few years ago and remember really liking it. There are lots of examples in there of situations that seem to defy logic, or at least defy our sense of what is logical. But these are provable mathematically using the basic probability.

Whenever I come across a problem like this, even once I get my head around it as much as I can, I like to write some code to prove it out.

One of my favorite such problems is the two children problem. I haven’t gotten to it in the book yet, but I know it’s coming up. It’s a classic. Here it is, my paraphrase:

A woman says she has two children. She says one of them is a boy. What are the odds that the other one is a boy?

The obvious answer is 50%. There’s a child you don’t know about. It’s either a girl or a boy. Everything else is irrelevant, right?

Nope. Actually the odds are 1 in 3 that the other child a boy, 2 in 3 that it’s a girl.

And the really odd part of it is you can change the wording in a way that seems to make no difference, but totally changes it:

A woman says she has two children. She says the first born one is a boy. What are the odds that the other one is a boy?

In this case, yes, the odds are 50% that the other child is a boy.

To understand the odds of a particular situation occurring, such as the genders of two children, you have to consider all possible arrangements and then how many of those arrangements satisfy the criteria. Then divide.

In this case, we have a family who had one child, then another child. They may have had a boy first, then a girl. Or maybe a boy, then another boy. Or a girl and then a boy. Or a girl and another girl. That’s four possibilities:

  • boy boy
  • boy girl
  • girl boy
  • girl girl

So in the first problem, the mother says that one of her children is a boy. This narrows us down to just three possibilities.

  • boy boy
  • boy girl
  • girl boy

In all three of those, one of the children is a boy. In two of them the other child is a girl and in only one, the other child is also a boy. So, 1 in 3 for boy, 2 in 3 for girl.

But let’s look at the second problem. Mom says that her first child is a boy. That gives us only two possibilities:

  • boy boy
  • boy girl

In one of those, the other child is a girl and in one, it’s a boy. 50/50.

Prove it with Code

I’m doing this with JavaScript using node.js. But use whatever you want.

For probability situations, it helps to have a large number of samples. So let’s make 1000 families. Each family will have two children. We can represent these by strings: “bb” means they had a boy, then another boy. “gb” means they had a girl, then a boy, in that order. Likewise, “bg” means the opposite order and “gg” means they had two girls. We’ll store all the families in an array, and we’ll actually go through one by one, randomly choosing the gender of the first child, and then the second child.

let families = [];

// create 1000 families with two randomly gendered children.
for (let i = 0; i < 1000; i++) {
  // no kids yet.
  let family = "";

  // first child
  if (Math.random() < 0.5) {
    family += "b";
  } else {
    family += "g";
  }

  // second child
  if (Math.random() < 0.5) {
    family += "b";
  } else {
    family += "g";
  }

  families.push(family);
}
console.log(families);

This should give you something like the following output, with 1000 total strings.

['gb', 'gb', 'gg', 'gb', 'gb', 'gg', 'bb', 'bg', 'bb', 'gb',
'bg', 'gg', 'gg', 'bg', 'bg', 'gg', 'gg', 'bg', 'gb', 'gg',
'gg', 'gb', 'bb', 'bg', 'gg', 'gb', 'gg', 'gb', 'bb', 'gb',
'bg', 'bb', 'gg', 'gb', 'gb', 'bb', 'bg', 'bg', 'gb', 'bb',
'gb', 'gb', 'gg', 'gg', 'bb', 'bb', 'bb', 'gb', 'gg', 'gb',
'gb', 'bg', 'gg', 'bg', 'bg', 'gb', 'bg', 'gg', 'gg', 'bg',
'bb', 'gb', 'bg', 'gb', 'gg', 'gg', 'bg', 'bg', 'gg', 'bb',
'gb', 'bg', 'gg', 'gb', 'bg', 'bg', 'bg', 'gg', 'bb', 'gb',
'gg', 'bb', 'bb', 'gg', 'gg', 'bb', 'gg', 'gg', 'bg', 'bb',
'bb', 'gg', 'gg', 'gg', 'gg', 'bg', 'gg', 'gg', 'bg', 'gg',
...
]

We have multiples of every type of family in there: “bb”, “bg”, “gg”, “gb”. Now, the mother said that one of the children was a boy. So let’s filter this down to only the families that have a “b” in them.

// now lets get all the families who have at least one boy
let oneBoyFamilies = families.filter(family => family.indexOf("b") > -1);
console.log(oneBoyFamilies);

My var name isn’t the greatest. Consider oneBoyFamilies to mean “at least one boy”. This should give you something like the following output.

['gb', 'gb', 'gb', 'gb', 'bb', 'bg', 'bb', 'gb', 'bg', 'bg',
'bg', 'bg', 'gb', 'gb', 'bb', 'bg', 'gb', 'gb', 'bb', 'gb',
'bg', 'bb', 'gb', 'gb', 'bb', 'bg', 'bg', 'gb', 'bb', 'gb',
'gb', 'bb', 'bb', 'bb', 'gb', 'gb', 'gb', 'bg', 'bg', 'bg',
'gb', 'bg', 'bg', 'bb', 'gb', 'bg', 'gb', 'bg', 'bg', 'bb',
'gb', 'bg', 'gb', 'bg', 'bg', 'bg', 'bb', 'gb', 'bb', 'bb',
'bb', 'bg', 'bb', 'bb', 'bg', 'bg', 'bg', 'bb', 'bb', 'bb',
'gb', 'bb', 'bb', 'bb', 'bg', 'bg', 'bg', 'gb', 'bb', 'bb',
'gb', 'bb', 'bg', 'bg', 'bb',
...
]

I don’t see any “gg”s in there, but just to be sure, we can say:

// validate that there are no families with two girls here
console.log(oneBoyFamilies.indexOf("gg"));

If we’ve done that right, we should get a -1 here. I do, so I’m satisfied.

Now, to find families where the other child is a boy, we need to look for families consisting of “bb”. But for families where the other child is a girl, we need to look for either “gb” or “bg”. Already you can see why it’s 2 to 1 in favor of the other child being a girl. But here’s the code:

let otherChildIsABoy = oneBoyFamilies.filter(family => family === "bb");
let otherChildIsAGirl = oneBoyFamilies.filter(family => family === "bg" || family == "gb");
console.log("boy: " + otherChildIsABoy.length);
console.log("girl: " + otherChildIsAGirl.length);

Alternately, for the girls, you could do something like:

let otherChildIsAGirl = oneBoyFamilies.filter(family => family.indexOf("g") > -1);

Shouldn’t make any difference. With either method, I consistently get in the mid-200s for boys and around 500 for girls. Total would be around 750-ish, which makes sense since we filtered out one of the four arrangements (“gg”). So, 1 in 3 for boys, 2 in 3 for girls. Spot on.

Finally, let’s do the second version. Here, the mom says her first child is a boy. So for that we have to search for families where the first character is a “b”. Then we just get the count of the resulting families where the second character is a “b” and the count where it’s “g”.

// this time, lets get all the families where the FIRST child is a boy
let firstBoyFamilies = families.filter(family => family.charAt(0) === "b");
console.log(firstBoyFamilies);

let secondChildIsABoy = firstBoyFamilies.filter(family => family.charAt(1) === "b");
let secondChildIsAGirl = firstBoyFamilies.filter(family => family.charAt(1) === "g");

console.log("boy: " + secondChildIsABoy.length);
console.log("girl: " + secondChildIsAGirl.length);

The results you see here will vary. Sometimes you’ll get more boys than girls as the second child. Sometimes the opposite. Occasionally they’ll be dead even. We’ll call that 50%.

Why am I doing this?

This is firmly in the realm of recreational mathematics. I just find it really interesting to prove these things out even when my brain doesn’t agree 100% all the time. I enjoy doing it and I’ll probably do some more.

PS:

Here’s all the code in one spot:

let families = [];
// create 1000 families with two randomly gendered children.
for (let i = 0; i < 1000; i++) {
  // no kids yet.
  let family = "";

  // first child
  if (Math.random() < 0.5) {
    family += "b";
  } else {
    family += "g";
  }

  // second child
  if (Math.random() < 0.5) {
    family += "b";
  } else {
    family += "g";
  }

  families.push(family);
}
console.log(families);

// now lets get all the families who have at least one boy
let oneBoyFamilies = families.filter(family => family.indexOf("b") > -1);
console.log(oneBoyFamilies);

// validate that there are no families with two girls here
console.log(oneBoyFamilies.indexOf("gg"));

let otherChildIsABoy = oneBoyFamilies.filter(family => family === "bb");
let otherChildIsAGirl = oneBoyFamilies.filter(family => family === "bg" || family == "gb");
// alternately…
// let otherChildIsAGirl = oneBoyFamilies.filter(family => family.indexOf("g") > -1);

console.log("boy: " + otherChildIsABoy.length);
console.log("girl: " + otherChildIsAGirl.length);

// this time, lets get all the families where the FIRST child is a boy
let firstBoyFamilies = families.filter(family => family.charAt(0) === "b");
console.log(firstBoyFamilies);

let secondChildIsABoy = firstBoyFamilies.filter(family => family.charAt(1) === "b");
let secondChildIsAGirl = firstBoyFamilies.filter(family => family.charAt(1) === "g");

console.log("boy: " + secondChildIsABoy.length);
console.log("girl: " + secondChildIsAGirl.length);

Learning CNC and Making a MediaBox

Earlier this year I talked about my “Bit-Box” – a custom keyboard, program launcher, Stream Deck clone, device. https://www.bit-101.com/blog/2020/07/bit-box/

The box was handmade, but I had purchased a 3d printed plate to hold the switches. A little later I had the idea of making my own plate with wood. Initial tests, chiseling out a square hole for a single switch worked pretty well, but as soon as I tried to cut out several adjacent holes, the wood between the holes kept chipping out.

I started thinking about using a CNC to do this, and eventually picked up a Sainsmart 3018 Prover.

It took a couple of hours to assemble. Pretty easy actually. And it only came with some relatively useless v-shaped engraving bits, so I ordered a set of flat endmills in different sizes. Since then I’ve picked up a bunch of different bits.

In terms of software, I’ve tried a few different options.

One is Inventables Easel. This is a web app made for the Inventables X-Carve cnc machine. But it can export gcode that can be used with the 3018. Easel has some decent features for free, but you have to pay for full functionality.

The other one I’ve used is Carbide Create. This is made for Carbide3D’s Shapeoko machines. It’s desktop software and is totally free. It also exports gcode. I like Carbide Create a lot better.

The basic flow is to create a set of simple 2d vector shapes – rectangles, circles, paths – then apply tool paths to each shape. For example, you’d specify that you want to use this rectangle as an outline shape that is cut 1/4″ deep. Or you want to use this circle as a pocket, 1/8″ deep. A pocket cut cuts the entire inner area of a shape to a certain depth. You can also do boolean operations to combine or subtract different shapes. It’s super basic, but really does most of what you’d need.

If you want to really go crazy, you can get into 3d modeling with something like FreeCAD or Fusion360, and then create tool paths from those models. A much bigger learning curve and probably overkill until you get into some really complex stuff.

I use Candle to send the gcode to the machine itself.

MediaBox

My goal was to create a “MediaBox”. This is just what I call a custom mini keyboard with media keys – play/pause, next, previous tracks, volume up/down/mute. Six keys in all. Here’s an overview of all my attempts from one of the original hand-cut versions, some test cuts, a couple of failed attempts, through the final working build:

The initial holes for the keys worked perfectly. A 0.555 inch square hole is all you need. Spacing is something like 0.205 inches between keys.

The main design issue beyond that was where to fit the Arduino board and how to route the usb cable. I was initially using 1/2″ black walnut. On the top were the holes for the keys. I then flipped it over and created a recess on the bottom. But the half inch depth was really too shallow. And my original design was just too small once I attached the cable.

So I switched over to 3/4″ walnut and made the whole thing just a bit larger.

Wired it up much the same as I did for the BitBox. Did some finish sanding and applied some tung oil, glued on a leather bottom.

The software presented a bit of a problem. The Arduino keyboard library does not provide a way to send media key codes. Luckily there is another 3rd party library, HID-Project.

You can add this library to your project by going to sketch / manage libaries and searching for “hid project”.

Here’s the code I came up with:


#include <hid-settings.h>
#include <hid-project.h>

// Define Arduino pin numbers for buttons and LEDs
#define VOL_DOWN 2
#define VOL_MUTE 4
#define VOL_UP 3
#define PLAY_PREV 5
#define PLAY_PAUSE 6
#define PLAY_NEXT 7

const long debounceTime = 30;
unsigned long lastPressed = 0;
boolean A, a, B, b, C, c, D, d, E, e, F, f;

void setup() {
  pinMode(VOL_DOWN, INPUT_PULLUP);
  pinMode(VOL_MUTE, INPUT_PULLUP);
  pinMode(VOL_UP, INPUT_PULLUP);
  pinMode(PLAY_PREV, INPUT_PULLUP);
  pinMode(PLAY_PAUSE, INPUT_PULLUP);
  pinMode(PLAY_NEXT, INPUT_PULLUP);

  a = b = c = d = e = f = false;
  Consumer.begin();
  BootKeyboard.begin();
}

void loop() {
  if (millis() - lastPressed  <= debounceTime) {
    return;
  }

  lastPressed = millis();

  A = digitalRead(VOL_DOWN) == LOW;
  B = digitalRead(VOL_MUTE) == LOW;
  C = digitalRead(VOL_UP) == LOW;
  D = digitalRead(PLAY_PREV) == LOW;
  E = digitalRead(PLAY_PAUSE) == LOW;
  F = digitalRead(PLAY_NEXT) == LOW;
  if (A && !a) {
    Consumer.write(MEDIA_VOL_DOWN);
  }
  if (B && !b) {
    Consumer.write(MEDIA_VOL_UP);
  }
  if (C && !c) {
    Consumer.write(MEDIA_VOL_MUTE);
  }
  if (D && !d) {
    Consumer.write(MEDIA_PREV); // alternately MEDIA_REWIND
  }
  if (E && !e) {
    Consumer.write(MEDIA_PLAY_PAUSE);
  }
  if (F && !f) {
    Consumer.write(MEDIA_NEXT); // alternately MEDIA_FAST_FORWARD
  }
  a = A;
  b = B;
  c = C;
  d = D;
  e = E;
  f = F;
}

This was adapted from a few other sample projects I found, as well as the code I had for the BitBox. It works great.

Want one?

I made this for myself, but I’d love to make some more. The materials aren’t cheap though. Well over $30 for the wood, leather, Arduino, keys and key caps. Then the time for cutting, finishing, soldering. I’ve got to work out pricing and different options, and the best way to sell them, but contact me if you’re interested.

I’d also be open to selling just the wooden box, either finished or straight off the mill and you can buy the other parts and put it together yourself. It’s a fun project.

Or… if you have a cnc already, I’m going to post the Carbide Create file I used, with instructions, for free. Check back soon for that.

I Think Bluetooth is Finally OK

Bluetooth was introduced on May 7, 1989. I think I first heard of it in the mid-2000’s. People would use it to try to send contact info or other files between feature phones. As I recall, it had about a 50% chance of actually working. All of my attempts fell squarely in the failing 50%. So I ignored it for a few more years.

Then there were smart phones with Bluetooth and laptops had Bluetooth. There were Bluetooth mice and eventually Bluetooth fitness devices and smart(ish) watches. And they all SUCKED.

Bluetooth and Me: A History

Mice

Every Bluetooth mouse I had was slammed down on the desk in frustration at least once. And only very narrowly avoided being hurled across the room. When you’re using something all day every day, 99% uptime is unacceptable. I’d be in the middle of something and the mouse would just stop responding and I’d have to spend a minute or so reconnecting it. Then it might be fine for several more hours. I tried several and finally quit. I’m firmly in the wireless USB dongle camp now as far as mice go. Logitech’s MX Master 3 is glorious. It actually supports Bluetooth AND wireless. I think I tried an earlier version of the MX Master on Bluetooth and quit the first time it disconnected. The wireless dongle has never once failed me, and I’ve used many.

Headphones

Specifically, I’m talking about “earbuds” or what the kids call “IEMs” (in-ear monitors) these days. I’ve had multiple sets of these. Historically, they suffer from four issues:

  1. Poor audio quality.
  2. Discomfort due to weight.
  3. Poor battery life.
  4. Connectivity issues.
  5. Cost.

You could probably come up with something where you could say you get to choose three out of those 5 points. Maybe. The point is, they play off each other. Better battery life means more weight and cost. Anyway, I never had a pair that I was happy with. In the end, the hassle of a cord (and these days a USB-C adapter) has always been less than the hassle of battery, discomfort, poor sound, and connection problems.

Speakers

I’ve also had multiple Bluetooth speakers. And I’ll even throw my car stereo system into this category. These have been so-so. Connectivity has often been an issue. Some good, some not so good. My car in particular is really bad. It always takes a minute or so and at least two tries to actually connect my phone.

The other thing that has killed me with Bluetooth speakers is that they’ve always had horrible performance on listening to voice audio sources. Music is ok, but just about every one I’ve had cuts out in the silence between words. It will pick up again when it hears the next set of words, but routinely a few words will be lost on almost every sentence. I listen to a lot of podcasts and audiobooks, and this was always impossible with every Bluetooth speaker I had.

Fitness Devices / Smartwatches

I’ve had multiple running watches that had Bluetooth, as well as several Fitbits and an Android Wear watch. Generally, the Bluetooth has worked great. Until it stopped working great. When they decided to stop connecting via Bluetooth, it seemed like there was nothing I could do to get them to reconnect. Even rebooting the device and whatever device it was trying to connect to. But then at some point it would just start working again for however many days.

All this is to say that I’m not just someone who hates something they’ve never tried. I’ve had dozens of Bluetooth devices and every single one of them has caused me some level of frustration. And yet, I keep buying them, holding out hope. (Except mice. I’ve eternally given up on Bluetooth mice.)

But wait!

In the last couple of months, I’ve purchased three Bluetooth devices that I’m actually quite happy with!

Galaxy Buds Plus

For some reason, I decided to take another leap of faith and got another set of Bluetooth ear buds. I checked out a ton of reviews on these things and these seemed like a solid buy. The cost was $139 on Amazon, which isn’t cheap, but not exorbitant. I’ve been amazed at how happy I am with these things. There’s nothing I can say about these that is negative.

Battery life is great. They have the charging case, which itself has wireless charging. I already have wireless charges scattered around the house, so it’s super easy to just toss it on one of them.

Connectivity has been flawless. They connect instantly, never lose the connection.

They are comfortable. I use them with foam tips, which I always get for any earbuds. Never get uncomfortable. I’ve used them while running and they stay put and feel fine.

Sound is quite good. Most of the time I’m listening to podcasts and audiobooks on my phone. They sound great for that. To be honest, for music, I stick with my Sony Walkman NW-A55 and wired Ikko OH-1 IEMs. That’s been a life changing combination. But if I’m running with my phone and want to listen to music, I’ll use the Buds for that, as the music is just background at that point.

I’ve had these for two and a half months now and I can’t say enough good about them. These are the items that have finally sold me on the idea that Bluetooth has made it.

JBL Flip 5

Speaking of sound, I recently picked up a Bluetooth speaker. To be completely transparent, I got this for free. A while back I switched to Verizon Fios and out of the blue they sent me this $100 coupon for the Verizon store as thanks for switching. Lots of phones and phone cases, chargers and headphones, none of which I really needed. I didn’t really need a Bluetooth speaker, but this had pretty good reviews and came to $95 with tax, so why not?

It sounds good, connectivity even on multiple devices has been great, and it works flawlessly with audiobooks and podcasts. Huge battery with lots of listening time. Also, you can turn off the power on/off and Bluetooth connect/disconnect sounds, which has been a big annoyance on every other speaker I’ve had.

Garmin Forerunner 235

In the last month I started running again. I pulled out my old Garmin running watch, which I hadn’t used in … sadly, years. After a full day of charging and trying to get it running, with no success, I ordered a new Garmin watch, the Forerunner 235.

It’s very nice. It’s a full on smartwatch (not Android), which you can add apps and watch faces to. I did set up a better watch face, but not really interested in other apps. It does all day heart rate and sleep tracking. Battery lasts a week if you’re not running. GPS while running will suck it down faster, but will still let you run for many hours without a problem.

It connects to the Garmin Express phone app via Bluetooth and that’s been nearly perfect. When I finish a run, if I have my phone on me, it nearly instantly syncs to the cloud via Bluetooth and phone. If I don’t have my phone on me, it often syncs as soon as I walk into my driveway, with my phone inside the house. Downright impressive.

Summary

Bluetooth may have won me over. I look forward to seeing other quality implementations, though I’m not holding my breath on the mouse situation.

My Wireguard Setup

Disclaimer

Someone has been submitting my recent posts to online tech news aggregators, where they are criticized for not being cutting edge or paradigm shifting enough. If you’ve been led to believe that this post awe and amaze you, complain to the person who submitted it, not me. This is just my personal blog where I write about stuff that I’m doing, mostly technology based. It will not change your life. That said…

Background

I’ve had a “home server” for close to ten years now. It’s a Linux-based desktop pc. It acts as a file server, media server, backup server and a place to try out different things. I guess it’s what is now popularly called a “home lab”. All that’s great when I’m at home on my home network. I can stream movies and music, get files, ssh into the server and do whatever I need to do.

But when I’m out and about, traveling, working (when we used to go out and do stuff like that), I’d also like to have that same access. That’s all simple enough. You go into your router settings, do some port forwarding to that box and then you can stream, ssh, ftp, vnc, whatever. I’ve certainly done just that often enough. But as I became more security conscious, this started to worry me more and more. Having all those ports open into my main machine made me nervous. Yeah, they are behind passwords, or hopefully keys. I locked down ssh pretty tightly, but still worried about it, and all those other services. When I was on Xfinity for home internet, their management app provided a security section which listed all the various attempts to access different ports on the network with their IPs and locations. It was shocking. It became something that was not just theoretical. People were (and are) actually trying to hack into my network. That’s when I shut everything down.

Enter Wireguard

I’d heard quite a bit about Wireguard and it sounded like what I needed. I came upon this tutorial which described exactly what I wanted to do and in pretty clear terms:

https://zach.bloomqu.ist/blog/2019/11/site-to-site-wireguard-vpn.html

This all went together really well. It took a bit of learning and messing things up and fixing them, but I eventually got it all working really nicely and doing exactly what I need. Here’s my current setup:

  • Main wireguard server hosted on an inexpensive VPS in the cloud.
    • ufw set up to block all traffic other than specific ports from specific wireguard clients.
    • rinetd to forward any needed ports to my home server. Currently, that’s just the port that my airsonic server is running on.
  • Main wireguard server hosted on an inexpensive VPS in the cloud.
    • ufw set up to block all traffic other than specific ports from specific wireguard clients.
    • rinetd to forward any needed ports to my home server. Currently, that’s just the port that my airsonic server is running on.
  • wireguard client running on my home server.
    • airsonic music streaming server running there.
  • wireguard clients running on a couple of laptops, my Android phone and tablet. Each client has it’s own private key and the public key of the server. The server has its own private key and the public keys of each client.

With this setup I can ssh into the VPS from anywhere in the world, provided I’m doing it from one of the configured clients. Once I’m into the VPS, I can then ssh into any one of the other clients that has an ssh server running. I could use rinetd to forward ssh on specific ports to specific clients. But for now, that use case is not that common. When the world gets back to normal and I’m out of the house more, that will be useful.

I’ve got my airsonic server running on a specific port of my home server, let’s say it’s 1234. rinetd is set up to forward port 1234 on the VPS to port 1234 on the home server. So I can access my music in the browser from any wireguard client, or I can use any one of many subsonic-compatible Android apps and have my music streaming to my phone or tablet no matter where I am.

This setup is pretty flexible, and I will be able to add other services to it just by opening up a port in ufw and forwarding it as needed using rinetd. Important thing to remember is that when I say “opening up a port in ufw” I mean a wireguard client accessible port. Nothing is open on the VPS except via wireguard. Nothing is open on my home server except via the VPS or local LAN.

Monitoring and Recovery

One downside to this setup is that to access my music for example, I’m relying on a chain of multiple links: wireguard on VPS, ufw, rinetd, wireguard on home server, airsonic. If any one of those doesn’t function just right, I’m listening to silence. This has happened a couple of times, especially when I first set things up and had some things not quite right. Actually, if ufw goes down, I’ll still be able to listen to my music, but my VPS will be open. So I wanted to get some monitoring in place. When things were down early on, I’d be making assumptions on which piece was broke and spending time trying to fix it, only to find out it was one of the other links. With correct monitoring, I can now tell exactly what is up and down.

Monitoring with Healthchecks

I’ve been a big fan of Healthchecks.io. You set up “checks” which provide you with a url to ping. If a check doesn’t get a ping within a specified time period, it notifies you via email, sms, or through more than twenty other integrated services. I’ve been using it to monitor my daily backups. If a backup doesn’t happen at a specified time, I know about it.

So I set up a cron job that runs a script every 10 minutes on my VPS, and a similar one on my home server. This script first checks the status of wireguard. If it’s up, it pings Healthchecks. It does the same for rinetd and ufw. My home server checks wireguard and airsonic. Each of these five services is set up as a separate check in Healthchecks so I can see the status of each of them separately. The cron job runs every 10 minutes, so I give it one extra minute leeway – if Healthchecks doesn’t get a new ping after 11 minutes, that service is marked as down.

Recovery

Eventually I realized that if a particular service was down, once I became aware of it, I’d just go to whatever machine and restart it, so why not just do that automatically. So I built that into each of my checks.

If, say, wireguard is down on the VPS, it will NOT send the ping to Healthchecks. So a minute or so later it will be flagged as being down. But in this case, the script will also automatically try to restart wireguard. The next time it runs (10 minutes later), hopefully it sees that wireguard is up and sends the ping.

Healthchecks also has a “grace period” configuration. Once it notices something is down, it will not alert you until that grace period is done. I set this to 10 minutes. This results in the following sequence if something goes down:

  1. Service X is up and Healthchecks gets pinged at 10:00 pm.
  2. Service X goes down at 10:05 pm.
  3. At 10:10 pm, the script sees that Service X is down and fails to ping Healthchecks.
  4. The script also attempts to restart Service X.
  5. At 10:11 pm Healthchecks has not had a ping in 11 minutes and marks Service X as down.
  6. At 10:20 pm, the script runs again. Service X is up so it pings Healthchecks, which marks Service X as up again.
  7. Alternately, the restart didn’t work and at 10:20 pm no ping is sent.
  8. In this alternate case, at 10:21 pm, Healthchecks emails and texts me about the fact that Service X is down.

A potential improvement to this is that after step 4, when Service X is restarted, I could verify that it’s now working and ping Healthchecks. immediately. This way, if the restart works, nothing is marked as down. But I’m going to run it as is for a while and see how this works out. So far, so good.

I’ve gone through and tested each on of these checks, turning the service off and leaving it off. Within 11 minutes it was marked as down and restarted. And shortly thereafter marked as back up. All automatically.

If this were some kind of public service or mission critical workflow, I could easily set up the pings for every minute or so. But the 10 minutes seems perfectly adequate for my purposes.

More Details?

This post is pretty high level. Most of what went into the wireguard setup is covered in the above link. If you want to set up something similar, I’d be happy to go into more detail on any specific points. Just let me know.

version 1.3

Me again, talking about this silly version program still.

Actually, there are some pretty cool updates over the past few point releases. They came fast and on the heels of each other. The idea was posed to use the Linux package manager – apt or pacman or whatever – to get data on a program instead of relying on a hard-coded list.

Background

After some back and forth I warmed up to the idea, but as a backup to the known program list, not as a replacement. My reasoning is that you might have multiple versions of foo installed. Maybe one was through the default package manager, one through some download-and-run-an-install-script method. They might get installed to different locations in your PATH. But when you call foo on the command line, you’ll only get one of them.

If you query the package manager, it’s going to tell you about the one that it knows, which may or may not be the default. But when you run foo -v on the command line, you will get the one that’s going to be actually run in most cases. So that should be the first place we look. If version doesn’t know about foo then it can turn to the package manager.

Details

I decided to tackle two of the major Linux package managers first – apt (used on Ubuntu and most other Debian derivatives) and pacman (used on Manjaro and other Arch derivatives).

On apt, you can find info about a package, say neovim, you’d type:

apt list neovim --installed

This will give you something like:

neovim/focal,now 0.4.3-3 amd64 [installed]

That 0.4.3-3 is the version number that we’re looking for. It took a bit of regex trickery, but I was able to parse that bit out of it.

On pacman you’d type pacman -Qi neovim and the result would look something like:

Name : neovim
Version : 0.4.4-1
Description : Fork of Vim aiming to improve user experience, plugins, and GUIs
Architecture : x86_64
URL : https://neovim.io
Licenses : custom:neovim
Groups : None
Provides : vim-plugin-runtime
Depends On : libtermkey libuv msgpack-c unibilium libvterm luajit libluv
Optional Deps : python-neovim: for Python 3 plugin support (see :help python)
xclip: for clipboard support on X11 (or xsel) (see :help clipboard) [installed]
xsel: for clipboard support on X11 (or xclip) (see :help clipboard) [installed]
wl-clipboard: for clipboard support on wayland (see :help clipboard)
Required By : None
Optional For : None
Conflicts With : None
Replaces : None
Installed Size : 20.45 MiB
Packager : Sven-Hendrik Haase svenstaro@gmail.com
Build Date : Wed 05 Aug 2020 04:16:43 AM EDT
Install Date : Fri 21 Aug 2020 07:37:52 AM EDT
Install Reason : Explicitly installed
Install Script : No
Validated By : Signature

So we can use grep and/or sed to find the one line of that which starts with Version: and grab the 0.4.4-1 part of it.

I then did basically the same thing for dnf which is the package manager on Redhat, Fedora, and derivatives.

So the process is:

  1. Check to see if version already knows about the program. If so, just do what it already does.
  2. If now, check apt, pacman and dnf. First we can just check to see if each one of those exist and only run the one that does exist. It’s unlikely that many people will have more than one of those. If we find one of those, we do the parsing and spit out the version it tells us about.
  3. If those all fail, then we can just tell the user we couldn’t find any information on that command.

Can we do more?

There are all kinds of other package managers on both Linux and Mac. I started making a list of the different ways you can find and install software and came up with

  • snaps
  • flatpaks
  • pip
  • npm
  • homebrew / linuxbrew

There are others, but those all cover a huge amount of ground. And it turns out that most of them were able to be solved with the same general strategy:

  • Does this package manager exist?
  • Does it know about this program and what info does it have?
  • Parse out the version number from the info it returns.

So, now version supports all of those. It just looks at each one of them in turn until it finds one that give an answer.

This also has the added functionality of being able to return the version of more than just executable programs. Package managers know about various libraries and other assets that aren’t directly executable or don’t have any way of querying them directly for their version. But version can tell you about them. Want to know what version of libusb you have installed? Typing version libusb will tell you.

A personal perk of doing this project is that I was forced to really learn grep and sed. Two programs that ranged from confusing to very mysterious in my mind. Now I get them and really like them. I wrote something up about them too: https://www.bit-101.com/blog/2020/09/grep-and-sed-demystified/