Independent game developers - making Avoyd

Implementing a static website in Google App Engine

Juliette Foucaut - 23 Aug 2013 - edited 10 Feb 2014


A robust website that successfully weathers spikes in traffic is a must when trying to sell and support a game over the internet. Last July Picroma suffered temporarily when they released their game, Cube World, for purchase. They then had to deal with a DDoS attack. More recently, Oculus Rift's site stalled when they tweeted about John Carmack's involvement in their technology.

Whilst we can only dream of enjoying the same level of interest, we'd like to spare ourselves the worry. When Doug researched a web hosting solution, he spotted that Wolfire used Google App Engine. They list their reasons clearly on their blog, and given that they have years of hands-on experience on the matter, we decided to follow their lead. As an added bonus, this solution is free for low levels of traffic.

We plan to eventually automate our site to support a blog, comments, a forum and purchases, but we currently only need a static website. This means limited cleverness on the client side and none on the server, just some basic html and a minimum of scripting for content and a css for the looks. In this post I'll explain how to host a "quick and dirty" static site on Google App Engine. It involves a few tricks but nothing too complicated. I've added a Links and tools section at the end of this post where you'll find all the resource I used (including to train myself). I hope you find the information useful and you enjoy crafting your site as much as I did.

Google Sites didn't work out

In September last year, Doug started rewriting Avoyd from scratch. At the time we only needed a presence on the internet. Our old website, which I'd written 12 years earlier, was still online but it badly needed a redesign (using tables for layout, difficult to maintain, etc...). We were already using Google Sites for our personal intranet. It's free, fast to configure and easy to use, great for our "we don't care about the looks" private space. So we decided to use Google Sites for our website as well.

Whilst it is powerful, Google Sites turned out unsuitable for our public website. I won't start a detailed rant about it here, but I'll say that if at least the documentation had been up to date - or in some cases had existed - it would have saved me a lot of time and frustration. We ended up with a decent-looking 2-page site including a blog that served its purpose for several months. However, it wasn't the right solution for us in the long term. That's when we decided to build the site ourselves from scratch, and to use Google App Engine to host it. Personally, I find that writing the website manually is easier, and I'm neither an expert in this domain nor am I a coder. I'm learning on the fly as I implement the site with Doug's help.

Moving to Google App Engine

Let's get on with the practical tasks involved in setting up a static website in Google App Engine.

Styles: Bootstrap and Font Awesome

Twitter Bootstrap is a simple (and free) way to make a website look sleek by using their predefined css. I wanted a slightly different look to start from so I used a css file from Bootswatch. They provide a few themes based on the bootstrap toolkit that can easily be tweaked.
I first downloaded the bootstrap.css file from one of Bootswatch's themes and added it to my css directory. Next I added the html <link href="https://www.enkisoftware.com/css/bootstrap.css" rel="stylesheet"> to all my html files <head> sections. Finally I tweaked the css to get the looks I wanted.
A side note on all the *.min.css and *.less files you'll encounter: I've not included (nor referenced them in my html) because a. it's easier; and b. I want to see the effects of editing my css file straight away.

With Bootstrap to address the layout, we also wanted to add some cool Font Awesome icons as visual cues, for instance for a link to Twitter or to illustrate a (as a side note, Doug also uses Font Awesome with libRocket for Avoyd's in-game GUI elements).
I followed the same principle as with Bootstap: downloaded Font Awesome, added the file font-awesome.css to my css directory, then the html <link href="https://www.enkisoftware.com/css/font-awesome.css" rel="stylesheet"> to all my html files <head> sections. I also needed to add the fonts themselves: I simply dumped the "font" directory and all of its contents at the same level as my css folder.
Once this is done, all you have to do to add an icon is to include <i class="fa fa-thumbs-o-up"></i> in your html, et voilà !

At this point my file structure looks like this:

 my-gae-website
   static_website
     css
       bootstrap.css
       font-awesome.css
     font		<= Font Awesome font files
       *.eot, *.otf, *.svg, *.ttf, *.woff
     *.html

As an example of how handy Bootstrap and Font Awesome are, the box above was created with a Bootstrap <pre> tag and Font Awesome icons.

Google App Engine

Download and setup

Although we have no server-side code (for now), we use the Python Google App Engine SDK. The steps needed to integrate your static website with GAE are as follows:

  • Follow the Google App Engine site's download and installation instructions.

    Note: if you've never used the Google App Engine Python SDK, it's a good idea to do the Hello, World! example. You won't need to know more for the purpose of a static website and you won't need to understand the programming involved.

  • In the root directory where you keep your static website (in this example, my static website is in directory static_website, under root directory my-gae-website, see previous section), add an empty text file and rename it "app.yaml".

 my-gae-website
   static_website
   app.yaml
  • Open the Google App Engine Launcher (I'll refer to it as the GAE launcher).

    • Select menu item File > Add Existing Application...

    • Set the application path to directory my-gae-website and select Add.

  • An application named "my-gae-website" is added to the list and is displayed in red. To make the application work, we need to add some code in app.yaml. To start with we'll use a default configuration:

  • Paste the default text below into your app.yaml and save it. You'll notice that the app "my-gae-website" in the GAE launcher immediately turns from red text to black.

application: my-gae-website
version: 1
runtime: python27
api_version: 1
threadsafe: yes

handlers:
- url: /
  static_files: static_website/index.html
  upload: static_website/index.html

libraries:
- name: webapp2
  version: "2.5.2"

If you want, you can already run the "my-gae-website" application from the GAE launcher and view your website locally in your browser. However it may not show anything yet: you need to configure app.yaml to serve your own static pages.

In the next section I'll explain how to configure app.yaml to serve a static website using our site as an example.

Configure app.yaml

This step sets the rules for displaying the contents of the website. In other words, app.yaml describes what will be returned (web pages, images...) when specific urls are entered. We've found the syntax of app.yaml not completely straightforward so I'm going to describe in detail how I've configured it in our specific example.
For a general understanding of the principles of app.yaml, see the links section about regex and app.yaml at the end of this post.
If you want to skip this section go straight to the deployment part.

app.yaml overview

Our app.yaml reads as follows (This is an overview. I'll explain the handler section contents in the next section):

application: my-gae-website
version: 1
runtime: python27
api_version: 1
threadsafe: yes

handlers:
#root
- url: /
  static_files: static_website/devlog.html
  upload: static_website/devlog.html

#serve our home page in case index.html is requested
- url: /index.html
  static_files: static_website/devlog.html
  upload: static_website/devlog.html

#specific html pages:
- url: /about.html
  static_files: static_website/about.html
  upload: static_website/about.html

- url: /devlog.html
  static_files: static_website/devlog.html
  upload: static_website/devlog.html

#specified zip file for download
- url: /downloads/AvoydV1_7_1.zip
  static_files: static_website/downloads/AvoydV1_7_1.zip
  upload: static_website/downloads/AvoydV1_7_1.zip

#the devlog post pages: since we're going to add more pages with the format 
#devlogpost-<yyyymmdd-dailyIncrement>.html and I don't want to update the  
#app.yaml each time, I've used a rough regex to limit the cases where an 
#invalid url would return the default 404 not found page.
- url: /(devlogpost-201[3-9][0-1][0-9][0-3][0-9]-[1-4]\.html)
  static_files: static_website/
  upload: static_website/(devlogpost.*\.html)

#all images and support file (css, fonts...): return file if found, 
#otherwise the default 404 page so it can be handled by sites that link
#directly to images.
- url: /(.*\.(gif|png|jpg|ico|bmp|css|otf|eot|svg|ttf|woff))
  static_files: static_website/
  upload: static_website/(.*\.(gif|png|jpg|ico|bmp|css|otf|eot|svg|ttf|woff))  

#all other urls: return the enkisoftware 404 not found 
- url: /.*
  static_files: static_website/notfound.html 
  upload: static_website/notfound.html

libraries:
- name: webapp2
  version: "2.5.2"

In the handlers section we see a repeating pattern of 3 lines headed url, static_files and upload (note: you'll find more info on Google's site). Here's what each one of them means:

- url: <the regex of the anticipated url>

  static_files: <the regex of the directory or file path which will be be
	served for the url above>. 
	The "" is particularly useful as it matches the 1st marked 
	subexpression which is the interior of the outermost url regex 
	brackets. For further information, see the definition of "
" in 
	the wiki on POSIX)
  
  upload: <the regex of the actual file path and name the url is referring
	to, on our local machine, before deployment>
url handling

We have to decide how we'll handle each file / file type and add the behaviour to app.yaml (see the comments headed with "#" in our implementation of app.yaml). For reference, our file names and folders are as follows:

 my-gae-website
   static_website
     css		<= cascading style sheets (Bootstrap, Font Awesome)
       *.css
     downloads	<= all our downloadable files
       AvoydV1_7_1.zip
     font		<= Font Awesome font files
       *.eot, *.otf, *.svg, *.ttf, *.woff
     images		<= all our images
       *.gif, *.ico, *.jpg, *.png
	 about.html	<= about page
     devlog.html	<= home page, also the development blog posts list
     devlogpost-20130823-1.html	<= an individual blog post
     devlogpost-20130411-1.html	<= an individual blog post
     devlogpost-20130427-1.html	<= an individual blog post
     devlogpost-20130509-1.html	<= an individual blog post
     devlogpost-20130529-1.html	<= an individual blog post
     notfound.html	<= our custom file not found
   app.yaml
url handling: custom page not found

Whenever a url is entered that doesn't match any of our files, we want to display our custom-defined file not found page so that people stay on our website. Note: there is an exception to this rule in the case of support files such as images, css... where we want to serve the default file not found error (see section below).

url handling: custom page not found

The url pattern is defined last in the handlers section:

#all other urls: return the enkisoftware 404 not found 
- url: /.*
  static_files: static_website/notfound.html 
  upload: static_website/notfound.html

Our custom file not found will be served for any url that matches none of the regexes described in the handlers section, except for the last: /.*.

url handling: html pages loaded with regex

With the file not found case taken care of, we can concentrate on defining the pages to serve for each url. The desired behaviour is as follows:

url handling: html pages loaded with regex

First of all we address the specific cases where we have unambiguously defined pages and files.

Root: If no file is requested in the url, for instance navigating to https://www.enkisoftware.com, we serve our home page which is (currently) devlog.html.

#root
- url: /
  static_files: static_website/devlog.html
  upload: static_website/devlog.html

Home page: most sites use index.html as their de facto home page. Since our home page has a different name and it's likely someone will request https://www.enkisoftware.com/index.html and it would be a shame if they ended up on our custom file not found, I'm adding an extra handler to redirect them to our home page devlog.html:

#serve our home page in case index.html is requested
- url: /index.html
  static_files: static_website/devlog.html
  upload: static_website/devlog.html

Specific files: we have two predefined html pages and a zip file that can be exactly matched:

#specific html pages:
- url: /about.html
  static_files: static_website/about.html
  upload: static_website/about.html
  
- url: /devlog.html
  static_files: static_website/devlog.html
  upload: static_website/devlog.html
  
#specified zip file for download
- url: /downloads/AvoydV1_7_1.zip
  static_files: static_website/downloads/AvoydV1_7_1.zip
  upload: static_website/downloads/AvoydV1_7_1.zip

Less specific files: I had to create individual pages for each blog post to work around a Disqus limitation. I chose a simple pattern for naming those posts since I can't predict how many nor how often we'll add them: devlogpost-<yyyymmdd-dailyIncrement>.html. To save having to update app.yaml every time we add a new post, I'm using this regex devlogpost-201[3-9][0-1][0-9][0-3][0-9]-[1-4]\.html.

I'm sure you've noticed this regex is not perfect. Why? Because if someone enters a url that matches the regex, i.e. a valid date (though I'm sure you've spotted that e.g. the 32nd March will incorrectly be considered a valid date) which doesn't correspond to an existing devlogpost file, they'll get the default file not found instead of our custom file not found (because the custom file not found will only be served if the url doesn't match the regex). Now I would prefer the custom file not found to be served (any suggestions welcome), but I'm going to automate the site at some point so this solution will do for now.

#the devlog post pages: since we're going to add more pages with the format 
#devlogpost-<yyyymmdd-dailyIncrement>.html and I don't want to update the  
#app.yaml each time, I've used a rough regex to limit the cases where an
#invalid url would return the default 404 not found page.
- url: /(devlogpost-201[3-9][0-1][0-9][0-3][0-9]-[1-4]\.html)
  static_files: static_website/
  upload: static_website/(devlogpost.*\.html)  
url handling: support files and images

We match images and support files (css, fonts etc.) based on their filename extension. If the file doesn't exist, we want to serve the default file not found so the error can be handled automatically by third party sites that e.g. link directly to images. In other words, in this case, we don't want to serve our custom file not found, as it wouldn't be processed by third parties.

url handling: images, css, fonts, etc.

#all images and support file (css, fonts...): return file if found, 
#otherwise the default 404 page so it can be handled by sites that link
#directly to images.
- url: /(.*\.(gif|png|jpg|ico|bmp|css|otf|eot|svg|ttf|woff))
  static_files: static_website/
  upload: static_website/(.*\.(gif|png|jpg|ico|bmp|css|otf|eot|svg|ttf|woff)) 
Test locally

From the GAE launcher, run my-gae-website and view your website locally in your browser.

Deploy

To speed up our deployment I created a batch file *.bat containing appcfg.py --email=<email address used for google app engine> update <my-gae-website>\ pause
This is a workaround for Google App Engine Launcher requiring that I enter my email and password each and every time I deploy. Whereas when I use the command prompt interface I only have to enter the details once per session.

Comments: Disqus integration

Disqus was relatively straightforward to integrate. It mostly consisted in adding a block of javascript at the bottom of each html page where we wanted comments. I first registered with Disqus as "enkisoftware", then followed the instructions. When asked to choose my platform, I picked Universal Code then integrated the javascript as instructed.

Unfortunately Disqus doesn't support more than one comments stream per page. To keep the number of pages to a minimum whilst the site is static, I wanted to have all the blog entries on one page devlog.html. Since that wasn't possible, as a workaround I created an individual page devlogpost-*.html for each blog post with its own comments area. I then duplicated each post on devlog.html, each with its Disqus comment counter which doubles as a link to the corresponding devlogpost-*.html. The links simply have #disqus_thread appended. As an example, in the devlog.html source, you'll find the link <a href="devlogpost-20130411-1.html#disqus_thread">. Note: to implement this I didn't need to use Disqus Identifiers.

Stats: Google Analytics

To add Google Analytics coverage, I followed the instructions on their support page, Set up(web), namely adding a piece of javascript to each html page.

Links and tools

Code references and self-training

html and css

html knowledge is necessary if you write your web pages manually, and css is useful to understand if you want to customise the bootstrap css file or make your own.

  • Code Academy provides a fast and succinct course on html and css

  • CSS positioning: try rainbodesign's tutorial if you find the Code Academy explanations confusing. (I found that using bootstrap, in particular their grid system, meant that I didn't need in-depth knowledge of css positioning).

app.yaml

To help understand how the app.yaml file works with regards to static files, see the static file handlers section in the app.yaml google documentation

Regular expressions

You'll need a basic understanding of regular expressions (a.k.a. regex) to edit the app.yaml file.

Tools

Google App Engine

Python google app engine SDK

Editor and versioning
Third party functionality

All the optional third party material I've used/integrated in the website:

[Edit 10 Feb 2014: thanks weaver for the comments, I've reworked the following sections of the post:
- Rewrote the GAE Download and Setup section to add information about the integration with GAE and the GAE launcher;
- Removed mentions of index.yaml since for a purely static website, index.yaml isn't required;
- Added a Test Locally section;
- Reworded the "Less specific files" part under url handling: html pages loaded with regex;
- Replaced all instances of "static/" with "static_website/";
- Replaced all instances of "mymain" with "my-gae-website".
Added link to presskit() for GAE.]


comments powered by Disqus
 › 2017
 › Avoyd Editor Prototype
 › 2016
 › Black triangles and Peter Highspot
 › Colour palettes and lighting
 › Concept art by Rebecca Michalak
 › 2015
 › Internals of a lightweight task scheduler
 › Implementing a lightweight task scheduler
 › Feral Vector
 › Normal generation in the pixel shader
 › 2014
 › Python Google App Engine debugging with PyCharm CE
 › Lighting voxel octrees and procedural texturing
 › Patterns and spheres
 › Python Google App Engine debugging with PyTools
 › Interview
 › Domain masking using Google App Engine
 › Octree streaming - part 4
 › Black triangles and nervous_testpilot
 › Presskit for Google App Engine
 › Octree streaming - part 3
 › Octree streaming - part 2
 › Octree streaming
 › 2013
 › LAN discovery with multiple adapters
 › Playing with material worlds
 › Developer Diary archive
 › Website redesign
 › First Person Editor
 › First Avoyd tech update video
 ›› Implementing a static website in Google App Engine 
 › Multiplayer editing
 › First screenshots
 › Thoughts on gameplay modes
 › Back in 1999
 › 2002
 › ECTS 2002
 › Avoyd Version 1.6.1 out
 › Avoyd Version 1.6 out
 › 2001
 › Biting the bullet
 › Avoyd version 1.5 out
 › Monday Mayhem
 › Avoyd version 1.5 alpha 1 out
 › Avoyd version 1.4 out
 › ECTS 2001
 › Fun with Greek letters
 › Closer just a little closer
 › Back already
 › Artificial Humanity
 › Products and promises
 › Ecommerce
 › Explosions galore
 › Spring fixes
 › Open source and ports to other operating systems
 › Avoyd LAN Demo Version 1.1 is out
 › Thanks for the support
 › Avoyd LAN Demo Ready
 › Game Tech
 › Internals of a lightweight task scheduler
 › Implementing a lightweight task scheduler
 › Normal generation in the pixel shader
 › Lighting voxel octrees and procedural texturing
 › Octree streaming - part 4
 › Octree streaming - part 3
 › Octree streaming - part 2
 › Octree streaming
 › LAN discovery with multiple adapters
 › enkiTS
 › Internals of a lightweight task scheduler
 › Implementing a lightweight task scheduler
 › Web Tech
 › Python Google App Engine debugging with PyCharm CE
 › Python Google App Engine debugging with PyTools
 › Domain masking using Google App Engine
 › Presskit for Google App Engine
 ›› Implementing a static website in Google App Engine 
 › Avoyd
 › Avoyd Editor Prototype
 › Black triangles and Peter Highspot
 › Colour palettes and lighting
 › Concept art by Rebecca Michalak
 › Feral Vector
 › Patterns and spheres
 › Interview
 › Black triangles and nervous_testpilot
 › Playing with material worlds
 › Website redesign
 › First Person Editor
 › First Avoyd tech update video
 › Multiplayer editing
 › First screenshots
 › Thoughts on gameplay modes
 › Back in 1999
 › Avoyd 1999
 › Developer Diary archive
 › Back in 1999
 › ECTS 2002
 › Avoyd Version 1.6.1 out
 › Avoyd Version 1.6 out
 › Biting the bullet
 › Avoyd version 1.5 out
 › Monday Mayhem
 › Avoyd version 1.5 alpha 1 out
 › Avoyd version 1.4 out
 › ECTS 2001
 › Fun with Greek letters
 › Closer just a little closer
 › Back already
 › Artificial Humanity
 › Products and promises
 › Ecommerce
 › Explosions galore
 › Spring fixes
 › Open source and ports to other operating systems
 › Avoyd LAN Demo Version 1.1 is out
 › Thanks for the support
 › Avoyd LAN Demo Ready