pibuilder (Paraphraser/PiBuilder) | MoltPulse

Back to Directory

pibuilder

Ideas for building a Raspberry Pi from "bare metal" to ready-to-run IOTstack

Paraphraser/PiBuilder00

MoltPulse

Based on repository activity, growth velocity and community engagement.

26

Growth1/30

Activity6/25

Popularity8/25

Trust11/20

116

Stars

High

Sentiment

Votes

116

README.md

PiBuilder

<a name="introduction"></a>

Introduction

This project documents my approach to building Raspberry Pi operating systems to support SensorsIot/IOTstack.

Design goals:

To have a reliable, repeatable and auditable approach to building Raspberry Pi OS, primarily as a support platform for SensorsIot/IOTstack.
As far as possible, to favour speed over any need for interaction during the build process.
All work done "headless" via ssh.
To have all IOTstack prerequisites satisfied so there is no need for the menu or other IOTstack scripts to be installing docker, docker-compose, and so on.

PiBuilder can't possibly be a "one size fits all" for all possible Raspberry Pi configurations. Of necessity, the scripts and supporting files implement my decisions and assumptions. You will almost certainly need to make adjustments for your own situation. I have tried to make allowances for that by providing a patching system that is flexible and extensible.

I have tested PiBuilder on:

Raspberry Pi 3B+, 4B and Zero W2 hardware
32-bit versions of Raspberry Pi OS (aka Raspbian) Buster and Bullseye
64-bit version of Raspberry Pi OS Bullseye
64-bit version of Debian Bullseye running in an AMD64 guest system under Parallels
64-bit version of Debian Bookworm running in an AMD64 guest system under Proxmox VE (see tutorial)
64-bit version of Ubuntu Bookworm running in an AMD64 guest system under Proxmox VE

The scripts are written in bash and there is a reasonable chance that they will work on any Debian- or Ubuntu-based system. Any steps which have a specific dependency on the Raspberry Pi are skipped. Please see Running on other platforms for more information.

without other forms of surrounding quote mark. Example:
```
$ ssh «user»@«hostname»
```
This syntax means you should replace the entire placeholder, including the guillemets, with the actual value. Example:
```
$ ssh pi@raspberrypi.local
```
with other forms of surrounding quote mark. Example:
```
$ git add "«filename»" 
```
This means you should replace the entire placeholder, including the guillemets, with the actual value, while retaining the surrounding quote marks:
```
$ git add "resolvconf.patch@iot-hub" 
```

Connect your media (SD or SSD) to your support host (eg Mac/PC).
Launch Raspberry Pi Imager.
Click <kbd>Device</kbd>, then
- Select your hardware platform (eg Raspberry Pi 5).
- Click <kbd>NEXT</kbd>.
Under "Choose operating system"
- Scroll until you see "Raspberry Pi OS (other)", and click that.
- <a name="liteOS"></a>From the list that appears, choose "Raspberry Pi OS Lite (64-bit)".
- Click <kbd>NEXT</kbd>.
Under "Select your storage device"
- Select the media connected in step 1. Be careful with this step!
- Click <kbd>NEXT</kbd>.
Under "Customisation: Choose hostname".
- <a name="firstBootHostName"></a>Enter a name. I recommend "raspberrypi". These instructions assume you accept that recommendation. Don't worry, you can (and should) change the name later.
- Click <kbd>NEXT</kbd>.
Under "Customisation: Localisation".
- Set the three fields to whatever is appropriate.
- Click <kbd>NEXT</kbd>.
Under "Customisation: Choose username".
- Enter a username. I recommend "pi". Once again, these instructions assume you accept this recommendation. If you choose a different username, you will have to substitute accordingly as you follow these instructions.
- <a name="firstBootUserPassword"></a>Enter a password of your choice, twice.
- Click <kbd>NEXT</kbd>.
Under "Customisation: Choose Wi-Fi".
- Enter the SSID of your WiFi network.
- Enter the network's pre-shared key, twice.
- Click <kbd>NEXT</kbd>.
Under "Customisation: SSH authentication".
- Turn on "Enable SSH".
- Leave the "Authentication mechanism" at its default setting of "Use password authentication"
- Click <kbd>NEXT</kbd>.
Under "Customisation: Raspberry Pi Connect".
- Disable "Enable Raspberry Pi Connect".
- Click <kbd>NEXT</kbd>.
At "Write image".
- Click <kbd>WRITE</kbd>.
- Follow the prompts.
At this point, and particularly if you're on a slow Internet connection, you get to curse whoever thought it was a Good Idea to prevent everyone from burning images via the old "download once, burn many" methodology. Now you get to download Every Single Time.
One the write completes:
- Click <kbd>FINISH</kbd>.

$ ping -c 1 raspberrypi.local

On your support host:

When your Raspberry Pi responds to pings, connect to it like this:
```
$ ssh-keygen -R raspberrypi.local
$ ssh «username»@raspberrypi.local
```
Note:
- The ssh-keygen command is protective and removes any obsolete information from your "known hosts" file. Ignore any errors.
Normally, SSH will issue a challenge like this:
```
The authenticity of host '«description»' can't be established.
ED25519 key fingerprint is SHA256:gobbledegook/gobbledegook.
Are you sure you want to continue connecting (yes/no)? 
```
This is sometimes referred to as <a name="tofudef"></a>the TOFU (Trust On First Use) pattern. Respond with:
```
yes
```
Your Raspberry Pi will ask for the password for the user «username». Respond with the password you set in Raspberry Pi Imager.

If you started from the Raspberry Pi OS Lite image, you will need to install git:
```
$ sudo apt update ; sudo apt install -y git
```
git is pre-installed in the non-Lite images so you can skip this step if you started from the Raspberry Pi OS with Desktop image.

Clone the PiBuilder repository:

$ git clone https://github.com/Paraphraser/PiBuilder.git ~/PiBuilder

On your Raspberry Pi:

If you accepted the advice at set hostname, your Raspberry Pi's name will be raspberrypi. Now you should select a unique name for your Pi.

Because it is adopted by multicast DNS, any name you choose must follow the DNS rules: lower-case letters, digits and hyphens (no underscores). The script enforces these rules by sanitising any non-compliant name.

You change your host's name by passing the new «hostname» as an argument to this script. The curly braces indicate that the {«hostname»} argument is optional. If you omit it, your host will retain its current name.

Omitting the argument would be appropriate if you already chose a unique name using Raspberry Pi Imager, or if you are using PiBuilder in a non-Raspberry Pi environment such as a Debian or Proxmox VE installation where the hostname may have been set via other means.

Run the first script:
```
$ ~/PiBuilder/boot/scripts/01_setup.sh {«hostname»}
```
Examples:
- change the host name:
```
$ ~/PiBuilder/boot/scripts/01_setup.sh iot-hub
```
- retain the existing host name:
```
$ ~/PiBuilder/boot/scripts/01_setup.sh
```
The 01 script runs to completion and reboots your Raspberry Pi. Rebooting disconnects your SSH session, returning you to your support host.

Note:
- The 01 script forces your Raspberry Pi's boot mode to "console". You may notice this change if you have an external screen connected your Pi. It is the expected behaviour. See VNC + console + PiBuilder if you want to understand the reason for this.
On your support host:

If the last part of the 01 script prompts you to do so, run the command:
```
$ ssh-keygen -R raspberrypi.local 
```
A normal reboot takes about 30-40 seconds. Sometimes, Raspberry Pis hang during this reboot. The most common symptoms are:
- Your Raspberry Pi responds to pings; but
- You are unable to connect via SSH ("Connection refused").
If this happens, you should remove power from your Raspberry Pi, turn it on again, and wait for it to start responding to pings.

On your support host:

You will know your Raspberry Pi is ready when it starts responding to pings:
```
$ ping -c 1 «hostname».local
```
Connect and login:
```
$ ssh -4 «username»@«hostname».local
```
Notes:
- If you see the TOFU pattern again, respond with "yes". * The -4 parameter on the ssh command instructs SSH to stick to IPv4. The 02 script disables IPv6 as part of its function so forcing IPv4 avoids locking-up the terminal session.
On your Raspberry Pi:

Run:
```
$ ~/PiBuilder/boot/scripts/02_setup.sh
```
The 02 script runs to completion and reboots your Raspberry Pi.

On your support host:

Connect and login:
```
$ ssh «username»@«hostname».local
```
On your Raspberry Pi:

Run:
```
$ ~/PiBuilder/boot/scripts/03_setup.sh
```
A common problem with this script is the error "Unable to connect to raspbian.raspberrypi.org". This seems to be transient but it also happens far more frequently than you would like or expect. The script attempts to work around this problem by processing each package individually, while keeping track of packages that could not be installed. Then, if there were any packages that could not be installed, the script:
- displays a list of the failed packages;
- invites you to try running the failed installations by hand; and
- asks you to re-run 03_setup.sh (which will skip over any packages that are already installed).
Please don't ignore these errors. Prerequisites will be missing if the 03 script does not complete normally. Missing prerequisites will almost certainly prevent PiBuilder from achieving its goal of creating a stable platform for IOTstack. Please keep iterating until the 03 script completes normally.

Normal completion of the 03 script is signalled with a logout (not a reboot).

On your support host:

Connect and login:
```
$ ssh «username»@«hostname».local
```
On your Raspberry Pi:

Run:
```
$ ~/PiBuilder/boot/scripts/04_setup.sh
```
The 04 script runs to completion and reboots your Raspberry Pi. This script is where docker and docker-compose are installed.

On your support host:

Once your Raspberry Pi comes back, login using:
```
$ ssh «username»@«hostname».local
```
On your Raspberry Pi:

Run:
```
$ ~/PiBuilder/boot/scripts/05_setup.sh
```
The 05 script ends with a logout (not a reboot) so you can login again immediately.

On your support host:
```
$ ssh «username»@«hostname».local
```
On your Raspberry Pi:

At this point, your Raspberry Pi is ready to run IOTstack. You can either restore a backup or go into the IOTstack menu and start choosing your containers:
```
$ cd ~/IOTstack
$ ./menu.sh
```
Once you have restored a backup or completed the menu run, you can bring up your IOTstack with:
```
$ cd ~/IOTstack
$ docker-compose up -d
```