Introduction: Shutdown/Reboot Non Responsive Raspberry PIs Remotely
This was a project I did to shutdown/reset a pair of PIs remotely. I run 2 raspberry PIs in a remote location on battery/solar power. The project was ultimately about running IP cameras but I added a PI to monitor the performance of the solar and batteries. I wanted a way to make sure everything was functioning correctly. I added 2 PIs for redundancy but I found there were cases when a PI would stop responding and a simple reset would fix it. But that required a site visit and a 3 hour drive. So I wanted a way to shutdown and reset a PI remotely. I could have simply gone with a reset but thought it was a good idea to at least attempt a graceful shutdown. This is a good idea in the case where a PI is still running but has lost WiFi. In my case I used PI Zeros but this will work with all models that have GPIO pins. Initially I tested this with a first generation PI connected to a PI3.
This is actually a very simple project. To sum up, we need to
- Solder the reset pin
- Change some config to configure the pins
- Add the jumper wires
- Create some basic scripts
Supplies
Materials
- Pair of raspberry PIs, any model with GPIO pins
- 4 jumper cables, can be purchased from eBay or can possibly salvage from scrapped PC
- 1 pin header soldered to the reset pin
- Soldering iron
Assumed knowledge
- Basic PI knowledge
- Know how to get to the command prompt to run commands
- Can use vi or nano to edit files
- Basic soldering skills
Note that all reference to pin numbers refer to the GPIO pin number, not the pin number itself.
Step 1: Solder the Reset Pin
The first step is to solder the reset pin. This may be different for other PIs. Finding the reset pin wasn't super simple as most information on the net simply showed a pair of pins and suggested shorting the 2 pins caused a reset. But we need to do it by pulling the reset pin low. On the PI Zero the pin is shown in the attached pic. I found it by using a multi-meter to determine which pin was ground. If you put the multi-meter on the 2 pins then the reset pin will have a higher voltage (3.3V from memory). Once the pin is found then solder a single header pin in place. To get the pin nice and straight use a header socket to hold it while soldering.
Step 2: Setting the Config
Next step is to configure the pins. This is fairly simple, just add this config to end of /boot/firmware/config.txt. If it already contains an [all] line then don't duplicate that line. On the older PI operating systems the file will be /boot/config.txt
[all]
#shutdown pin
dtoverlay=gpio-shutdown,gpio_pin=22
#triggers shutdown
gpio=23=np,ip
#triggers reset
gpio=24=np,ip
We may also need to update/un-comment this entry in /etc/systemd/logind.conf. From my understanding this may be the default behaviour.
HandlePowerKey=poweroff
After changing these settings it is necessary to do a full reboot as the config.txt file is only read during boot.
What do these settings do?
The first setting sets GPIO22 as the shutdown pin. If that pin is pulled to ground then a shutdown will be initiated. The next 2 lines set GPIO23 and 24 as input (ip) with no pull ups (np). This means these pins will float until we do something with them. Why inputs? It's just a way to set them as unused until we want to use them.
Are there any issues during boot?
I was concerned these pins would not continue to float during a boot, if they go low for even the shortest time then it would trigger a reset or shutdown of the other PI. In my testing this didn't happen, it was possible to reboot 1 PI without impacting the other.
Testing
At this point you can test by momentarily shorting GPIO22 to ground and confirming a shutdown is initiated. Once the PI has fully shutdown then momentarily short the reset pin to ground and it should boot back up. This step isn't essential but is a very good idea. If it fails to work at the end then it's a more complex situation to debug.
Step 3: Adding the Jumpers
This step is fairly simple. Connect GPIO22 and GPIO23 between the PIs but reversed, ie GPIO22 to GPIO23. Then connect GPIO24 to the reset pin on the other PI
Step 4: Scripts to Trigger Shutdown/Reset
Simply add these 2 scripts:
shutdown.sh
pinctrl set 23 op dl
sleep 1
pinctrl set 23 op dh
sleep 1
pinctrl set 23 ip pn
reset.sh
pinctrl set 24 op dl
sleep 1
pinctrl set 24 op dh
sleep 1
pinctrl set 24 ip pn
Then mark them as executable
chmod +x shutdown.sh
chmod +x reset.sh
Note this uses the pinctrl library which comes included with the latest PI OS. It may be necessary to install this with older PI OSes.
Testing
Simply run ./shutdown.sh and confirm the other PI shuts down gracefully
Then, after a period of time (1 or 2 mins), run ./reset.sh to boot the PI back up.
Note that this is fairly safe in that you can't accidentally shutdown both PIs.
What if it doesn't work?
There is zero feedback when you run the shutdown or reset scripts, it can't tell what the other PI has done but here are some pointers
- confirm you have the right GPIO pinout for your PI. I wasted several hours because I had the wrong pinout (try googling "Pi Zero Pinout" the very first hit didn't work for me)
- Check the pins go low with a voltmeter
- Try pulling GPIO22 or reset pin low manually (connect them to a ground pin for about 1 second)
- Make sure you rebooted after setting the config.txt file
- Use a multi-meter to test the GPIO23 and GPIO24 do go low for about 1 second when the script is run
- You can test with a single PI by connecting GPIO22 to GPIO23 and GPIO24 to reset
- Use the pinctrl command to check the status of pins and set them low manually. (use "pinctrl help" to see commands)
- Confirm other scripts/utilities aren't using the same pins
Step 5: Further Steps
In order to make the PIs as robust as possible there are some other things worth doing. I won't detail them here but guides are easy to find on google:
- configure the hardware watchdog timer
- write a script to reboot on lost connectivity. The script can try disabling/enabling with wifi/lan connection before a reboot
- google PI power saving if running from batteries. For some PIs the savings can be very significant. For PI3 I found it's possible to get it down to 40mA from a 12V battery (1Ah per day) which is the same as a PI Zero.
- For power savings I found WiFi to be the lowest. Google appears to say the opposite.
- Configure additional scripts to run the shutdown/reset scripts automatically on loss of connectivity.
If you have your own tips for making a PI more robust, please add them in the comments section.