Automated Google Drive File Backup [Python - Google API - Service Account - Linux]

In this Instructable I will be showing you how to use the Python Google Drive API with a service account. In particular we will use the service account to perform automated backups of our file-system to Google Drive. Below I will list a few Pros and Cons of this project so as not to waste any of your time:

Pro's of this project:

- Shows you how to automate backup of local filesystem folders to Google Drive storage

- Once you have the basis, you can follow the Google API documentation to perform more tasks alongside other Google services

- Service accounts do not require permission every time an operation must be performed, therefore they are perfect for calling the Google API without a user being present for said interactions (i.e: scripted robots)

Con's of this project:

- Still in its infancy (may be phased out as Google creates new versions of its client API)

- Assumes user is operating within a Linux environment

- Can only modify service account data through Google API

- May be difficult to adapt to a Windows System without some modification

- Assumes you are using Python 2.7 (Python 3.0 versions of the script will be provided at some future time)

IMPORTANT: Service accounts have their own Google accounts, which means they have their own Google Drive storage space that holds space in the parent account's Drive storage, but is inaccessible to the parent account and through any Google graphical interface. This means the only way to modify the data on the service account is through the Google API, or with a service account Drive folder that gets shared via email to the parent account.

The following site will become your go-to reference if you would like official guidance: https://developers.google.com/drive/v3/web/quickstart/python

Step 1: Program Flow and Prerequisites

Program flow:

- Program connects to Google services and authenticates a credentials object using a credentials file

- Program passes the credentials object to form a service object

- Service object is used to make Google API calls, specifically to the Google Drive Service

- Using system calls, the program creates a ZIP folder of files to upload

- Program uploads ZIP folder to Google Drive through service object

- Program deletes the ZIP folder from the local system then terminates

Prerequisites:

Before starting I would like to list the programs that I assume you have installed on your computer (running a Linux OS). I have provided commands to install them below.

- Python 2.7 [Install instructions here]

- Python Pip Installer [sudo wget https://bootstrap.pypa.io/get-pip.py] then [sudo python get-pip.py]

- Zip [sudo apt-get install zip]

If you don't already have these installed please install them on your system. Once acquired, you will need to install the Google Drive Client API libraries, you do so via the following command:

sudo pip install --upgrade google-api-python-client

Once you have all this, you should be set to go! Now on to setting up the service account!

Step 2: Service Account Setup

This process is detailed in the attached images.

To start setting up the service account you must visit https://console.cloud.google.com and create a new project. Once complete, go to the API Manager and find the credentials tab on the navigation bar. You should then proceed to create a new Service Account Key which will prompt you to select a service account, click New Service Account and name it to your liking, then assign it the role of Owner***. After assigning the role, download the JSON key file and store it securely. After modifying all these settings, hit the Create button and your JSON key file with download.

With the service account created, you can now make calls to it through the Google Drive Python API, but we need to lastly enable the Drive API. Go to the Library tab on the navigation bar then search for and select the API (Google Drive API) you would like to enable. Once enabled, you are good to go!

***Note: As Owner, the service account can create, read, update and delete any information stored on its Google Drive, be very careful and select another role for the account if you would not prefer this functionality.

Step 3: Python Scripts

Once our Service account is setup and ready to go, the next step will be writing a program to actually use with the API. Luckily I have already written a bare-bones script to get you started with some very basic functions and API calls. Please refer to the attached images on how to run the scripts and where to place them. I suggest you visit https://developers.google.com/drive/v3/web/quickstart/python if you would like more information and examples on how to perform various other actions such as searching for a file or deleting items.

There are three versions of the script that I have attached, the first one being for simply interacting with the API and the second one being for performing some additional actions and the third being for backing up our file system. Please download the script of your choosing and give it a test.

File 1: [gdrive_api_bare.py]

This script simply grabs the services object for your Google Drive service account. You will need to look at further documentation online to use the services

File 2: [gdrive_api_basics.py]

This script has a few basic operations that create a gdrive folder, populates it with the provided file, then shares the link to the folder to the provided email. It does not check for errors and should be modified to a users liking.

File 3: [gdrive_auto_backup.py]

This program is in no way perfect, but it gets the job done. The user should modify it to their liking, but the essential features are present.

All the scripts have a PREDEFINES section close to their tops. As long as all the information entered in that section is valid, then the program will function without error. I have tested each file to be sure they work, but in the event of an issue, please post it in the comments section of this Instructable.

REMINDER:

The JSON key file must be in the same directory as the python script you will run. Also, run each script with sudo or they may not function as expected.

From here you're done! :D Go on to the next step if you would like to see how to schedule the python script with Crontab so that it runs without requiring a human to maintain it.

Step 4: Cronjob Automation

After downloading and testing the python scripts, you may next want to schedule it to run every so often. Specifically with backing up files, you may not want to have to perform the backups yourself; with Linux Crontab we can automate this process and backup our file system without human intervention. What we will do is create a Crontab entry to run our script in the following manner:

1. Open up the terminal and run:

sudo crontab -e

This will open an editor (preferably nano) to then edit the crontab entries to tell it to run our script

2. Once in the editor you need to properly format the event schedule. Below is an example:

* * * * * cd /home/scriptFolder && /usr/bin/python /home/scriptFolder/gdrive_auto_backup.py > /tmp/output.log

The following event tells the scheduler to change the directory to that of the script folder, then get python in the /usr/bin/ directory to run the script in the /home/scriptFolder/ directory. The last bit tells Crontab to store all the printed output from the script to output.log located in the /tmp/ directory.

You can see from the attached image these two steps.

3. Finally, I suggest visiting https://crontab.guru/ to figure out what you want to replace the * * * * * (five stars) with, which tells the program at what time it is scheduled to run. Once you do this, you are good to go and should have a fully automated Google Drive system backup for free!

Step 5: You're Done!

Now that you have completed this Instructable, feel free to comment with any problems you encountered so that I may update this tutorial in the coming weeks. Since you're done, enjoy!