In a previous post I explained how to set up a preconfigured Google Cloud Deep Learning VM from command line.

In this post, I will extend those configurations by adding a startup script that will contain all startup instructions needed for the 2nd homework of UC Berkeley's Deep Reinforcement Learning course.

The code from this post can also be found on my github account.

Shell script

Here is the shell script pgpu-instance.sh, which will be executable (make sure to chmod it correspondingly):

#!/usr/bin/env bash
export IMAGE_FAMILY='tf-latest-cu92'
export ZONE='us-central1-c'
export INSTANCE_NAME='hw2'
export INSTANCE_TYPE='n1-highmem-2'

# https://stackoverflow.com/a/12694189/4804137
DIR="${BASH_SOURCE%/*}"
if [[ ! -d "$DIR" ]]; then DIR="$PWD"; fi

gcloud compute instances create $INSTANCE_NAME \
        --zone=$ZONE \
        --image-family=$IMAGE_FAMILY \
        --image-project=deeplearning-platform-release \
        --maintenance-policy=TERMINATE \
        --accelerator='type=nvidia-tesla-k80,count=1' \
        --machine-type=$INSTANCE_TYPE \
        --boot-disk-size=200GB \
        --metadata='install-nvidia-driver=True' \
        --preemptible \
        --metadata-from-file startup-script=$DIR/startup-gpu.sh

What has changed:

  • an extra line that saves the current directory in a variable called DIR
  • --preemptible creates a preemptible VM
  • --metadata-from-file here we specify the startup script that will be run in the VM

Startup script

And here the startup script startup-gpu.sh which is being referenced by the --metadata-from-file flag:

#!/bin/bash

# set umask for all users such that created directories under a specific user
# can be written by users of the same group (group will be set in the end).
# This will be needed when different machines are going to ssh into the VM
echo "umask 002" >> /etc/profile
umask 002

# get cmake which is needed for gym's atari extension
sudo apt-get --assume-yes install cmake

# create a new directory within the default deeplearning workspace that comes
# with the preconfigured VM
cd /opt/deeplearning/workspace
mkdir berkeleydeeprl
cd berkeleydeeprl

# git clone my personal berkeleydeeprl fork
git clone https://github.com/reinka/homework.git
# openai's gym
git clone https://github.com/openai/gym.git

# replace lunar_lander.py as required in hw2 (for more info see the hw
# instructions)
cp homework/hw2/lunar_lander.py gym/gym/envs/box2d/

# install gym with atari extension; install seaborn for plotting
pip3 install -e gym[atari]
pip3 install seaborn

# install box2d for the lunar lander homework part
sudo apt-get install swig3.0 && sudo ln -s /usr/bin/swig3.0 /usr/bin/swig
pip3 install Box2D==2.3.2
pip3 install box2d box2d-kengz

#  interactive system-monitor process-viewer and process-manager
sudo apt-get install htop

# change folder group to `adm`
sudo chgrp -R adm /opt/deeplearning/workspace
# change access rights of the group so any ssh user can read / write
sudo chmod -R g+sw /opt/deeplearning/workspace

This will set up the VM with all needed dependencies for homework 2. The Atari extension is also installed, which will be used in hw3.

Now we only need to make pgpu-instance.sh executable (e.g. chmod 755) and run it to set up a new VM.

Default Access Rights

By default, the /opt/deeplearning/workspace directory is only writeable by the root user. Changing the direcotry's group to adm and making it writable for all users of that group will fix this. However, newly created directories by specific users will be writeable by its creator only: I sometimes log into my VM from a second laptop. In that a case, a directory created from my main laptop was not writable. Changing the umask to 002 fixed this issue.

Getagged mit:
gcloud Tutorial English
blog comments powered by Disqus