Two years ago, on a Friday morning at work, I bought my first Raspberry Pi.
I didn't really knew what I was gonna use it for but I really loved the idea of a pocket computer so I got one and for the better part of the year that followed I used it as a media server attached to a TV. All this was awesome but I wanted it to do so much more.
About the same time, we started experimenting with Docker at work and as any good developer that has a linux server attached to his TV, I wondered can i run Docker on this?, the answer is yes and it's a lot easier than I thought, after a quick Google search I found the guys at Hypriot who specialised in building Docker ready system images for ARM devices.
As the time went on and I had the chance to explore the Docker ecosystem, the release of Raspberry Pi 3 and of a reminiscing course on distributed systems made me decided to finally build a Raspberry Pi stack.
Building the stack is not that hard, I could've wired everything on a table and that would've been everything but I strive to be tidy and organised so I opted to get a stackable case.
Two weeks later when all the parts arrived, I could finally start building so I unpacked everything, striped the protective foil from the layers on the case and planned the building process which took an hour or so. The only exciting part was when my switch would not fit beside the power hub in the bottom layer of the case so... I cut the switch out of the case.
I don't have any photos from the build process as I'm writing this half a year after the events described, but here's the final product.
The guys at Hypriot have an awesome getting started guide, I'll present here what I did the Mac OS X as that's what i needed.
First things first, download the latest image from the download page.
Go to your downloads folder.
cd ~/DownloadsExtract the zip, this should have a new **.img** file.
unzip hypriot-rpi-???.img.zipInsert the SD card in your computer and find the disk alocated to it by running:
diskutil listFor this example I'll presume your SD card is mounted at `/dev/disk4`.
Let's unmount the disk and prepare for flashing.
diskutil unmountdisk /dev/disk4Now we are ready to flash the SD card. We are going to use the dd command for this.
Before you execute the command below, make sure to:
sudo dd if=hypriot-rpi-???.img of=/dev/rdisk4 bs=1m
Now we only have to do the same thing for the other 3.
I did a quick search on their repo and found flash, it's a tool they developed to make this whole process simpler and you can set the WiFi credentials during flash time which skips the whole monitor and keyboard part making the process more lean.
For the master node, after inserting the SD card in the Raspberry Pi, connect it to a monitor, attach a keyboard to it and wait for it to boot you can set the WiFi connection by updating your /boot/occidentalis.txt to something like:
# hostname for your Hypriot Raspberry Pi:
hostname=rpi-0
# basic wireless networking options:
wifi_ssid=SSID
wifi_password=12345And with this done we can access the rpi-0 through ssh.
ssh pirate@rpi-0.local
Having the general idea in mind represented by a master node that acts as ethernet access point for the other 3 nodes connected to the switch and knowing that from now on everything happens on the master node, let's connect to it by ssh pirate@192.168.0.100 assuming this is the IP assigned to the node from the router.
Like any good linux setup we start with updating the system.
sudo apt-get update && sudo apt-get upgrade
Install the dhcp server.
sudo apt-get install isc-dhcp-server
In the configuration file `/etc/dhcp/dhcpd.conf` comment the next 2 lines
#option domain-name "example.org";
#option domain-name-servers ns1.example.org, ns2.example.org;And uncomment the next one
authoritative;After that, at the bottom of the file define the new subnet that will serve the other nodes.
subnet 192.168.2.0
netmask 255.255.255.0 {
range 192.168.2.100 192.168.2.200;
option broadcast-address 192.168.2.255;
option router 192.168.2.1;
max-lease-time 7200;
option domain-name "rpi";
option domain-name-server 8.8.8.8, 8.8.4.4;
}
In /etc/default/isc-dhcp-server set which interface should the DHCP server manage.
INTERFACES="eth0"After that we need to set the ip for the master node, /etc/network/interfaces.d/eth0 should look something like this after.
autho eth0
iface eth0 inet static
address 192.168.2.1
netmask 255.255.255.0Reboot the master node for the dhcp configuration to load and connect back to it.
Inside the /etc/sysctl.conf we need to set the ip forwarding to true, for this we need to uncomment the following line.
net.ipv4.ip_forward=1We activate the ipv4.ip_forwarding by running:
sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"Run the following commands to create the network translation between the wifi port wlan0 and the ethernet port eth0.
sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
sudo iptables -A FORWARD -i wlan0 -o eth0 -m state --state RELATED
sudo iptables -A FORWARD -i eth0 -o wlan0 -j ACCEPTCheck if everything is set as expected, should look something like this.
sudo iptables -t nat -S \
-P PREROUTING ACCEPT \
-P INPUT ACCEPT \
-P OUTPUT ACCEPT \
-P POSTROUTING ACCEPT \
-P POSTROUTING -o wlan0 -j MASQUERADE
sudo iptables -S \
-P INPUT ACCEPT \
-P FORWARD ACCEPT \
-P OUTPUT ACCEPT \
-A FORWARD -i wlan -o eth0 -m state --state RELATED \
-A FORWARD -i eth0 -o wlan0 -j ACCEPT To make this happen on reboot (so you don't have to type it every time) run
sudo sh -c "iptables-save > /etc/iptables.ipv4.nat"And now we just have to tell the ethernet interface to pick the settings at reboot, in /etc/network/interfaces.d/eth0 update the ethernet interdace to something like this.
iface eth0 inet manual
address 192.168.2.1
netmask 255.255.255.0
post-up iptables-restore < /etc/iptables.ipv4.natConnect the first node to the switch and check inside the /var/lib/dhcp/dhcpd.leases to see which ip was assigned to the new node. You may see multiple entries in this that look something like this.
ease 192.168.2.102 {
......
hardware ethernet XX:XX:XX:XX:XX:XX;
......
}Get the mac address and assign it a static ip for easier reference by adding in the dhcp config file /etc/dhcp/dhcpd.conf at the end, an entry that would look something like this.
host rpi-1 {
hardware ethernet XX:XX:XX:XX:XX:XX;
fixed-address 192.168.2.101;
}Reboot the node and you could access it from the master with ssh pirate@192.168.2.101
Set the hostname in /boot/occidentalis.txt
# hostname for your Hypriot Raspberry Pi:
hostname=rpi-1Now we can add the rest of the nodes by repeating the same process.
On the master node let's initialize the swarm by running
docker swarm init --advertise-addr 192.168.2.1
#Swarm initialized: current node (e216jshn25ckzbvmwlnh5jr3g) is now a manager.
#To add a worker to this swarm, run the following command:
docker swarm join \
--token some-very-secret-code-generated-by-docker \
192.168.2.1:2377
#To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.Now that the swarm is created we need to add the nodes to it by ssh-ing into each one and joining them to the swarm.
docker swarm join \
--token some-very-secret-code-generated-by-docker \
192.168.2.1:2377Checking if everything went ok.
$> docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
1bcef6utixb0l0ca7gxuivsj0 rpi-1 Ready Active
38ciaotwjuritcdtn9npbnkuz rpi-2 Ready Active
sfasdsgxzfsdfsafa6as76655 rpi-3 Ready Active
e216jshn25ckzbvmwlnh5jr3g * rpi-0 Ready Active LeaderNow we're going to start a proxy service, I'm gonna use traefik.
docker service create --name proxy \
--constraint=node.role==manager \
--publish 80:80 --publish 8080:8080 \
--mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock \
--network proxy hypriot/rpi-traefik \
--docker --docker.swarmmode --docker.domain=192.168.0.100 \
--docker.watch --web --logLevel=DEBUGAnd a simple httpd container for demo purposes.
docker service create --name httpd \
--label traefik.port="80" \
--label traefik.backend="httpd" \
--label traefik.frontend.rule="PathPrefixStrip:/httpd" \
--label traefik.docker.network="proxy" \
--network ingress --network proxy \
--replicas 1 hypriot/rpi-busybox-httpdScale it up with docker service scale httpd=3 and check if everything is ok.
$> docker service ps httpd
NAME IMAGE NODE DESIRED STATE CURRENT STATE
httpd.1 rpi-busybox-httpd rpi-2 Running Running 6 minutes ago
httpd.2 rpi-busybox-httpd rpi-3 Running Running 4 minutes ago
httpd.3 rpi-busybox-httpd rpi-1 Running Running 4 minutes agoGoing to http://192.168.0.100:8080 we should see the Traefik dashboard and it should look something like this.
Going to http://192.168.99.100/httpd we should see:
And it is done. We have a raspberry pi stack running docker swarm, a proxy that routes and load balances requests to nodes in the cluster and a service that serves an html page.
Thank you!
Come check out our other fun projects on our blog.
----