Written by Rudy D'hauwe on June 2, 2013.
This article explains how I added Expires headers to the Whizu website which is running on a free Google App Engine (GAE) instance. The intent was to optimize the usage of server resources in order to respect the daily usage quota and to avoid extra billing for additional resources. Of course this is also beneficial for the overall performance of the website.
Google App Engine (GAE) lets you run web applications on Google's infrastructure. Google App Engine applications are easy to deploy and easy to scale as your traffic and data storage needs grow. With Google App Engine, there are no servers to maintain. You just upload your application, and it's ready to serve your users.
A Google App Engine application can however consume resources up to certain safety quotas. With these quotas, Google App Engine ensures that your application won't exceed your budget, and that other applications running on Google App Engine won't impact your application's performance. Additional resources will result in additional billing so there's an interest to avoid unnecessary usage of resources.
The point of using browser caching and Expires headers is to reduce the number of HTTP requests, which improves the performance for your returning visitors. It also avoids unnecessary usage of GAE resources as the outgoing bandwidth is limited by a daily quota.
The first time someone visits your site, their browser will fetch all your images, CSS files, JavaScript files, etc. Normally that happens every time the same visitor comes back to your site. With Expires headers you tell your website visitor's browser that the files you specify are not changing until after a certain time, for example a month. This means that the browser doesn't have to to re-fetch images, CSS stylesheets, JavaScript files, etc every time your visitor comes back to your site.
Many web applications have files that are served directly to the user's browser, such as images, CSS stylesheets, or browser JavaScript code. These are known as static files because they do not change, and can benefit from web servers dedicated just to static content. Google App Engine serves static files from dedicated servers and caches that are separate from the application servers. Files that are accessible by the application code using the file system are called resource files. These files are stored on the application servers with the app.
By default, all files in the WAR are treated as static files, except for JSP files, which are compiled into servlet classes and mapped to URL paths, and files in the WEB-INF/ directory, which are never served as static files and always available to the app as resource files.
Unless told otherwise, web proxies and browsers retain files they load from a website for a limited period of time. You can configure a cache duration for specific static file handlers. The value is a string of numbers and units, separated by spaces, where units can be d for days, h for hours, m for minutes, and s for seconds. For example, “4d 5h” sets cache expiration to 4 days and 5 hours after the file is first requested. If the expiration time is omitted, the GAE production server defaults to 10 minutes.
Before wanting to set the Expires header on certain file types, I had not overruled the default behavior that defines which files are to be considered as static files. I was just using the default behavior as explained before.
So I added the following configuration to WEB-INF/appengine-web.xml.
<static-files>
<include path="/**.png" expiration="7d" />
<include path="/**.jpg" expiration="7d" />
<include path="/**.ico" expiration="7d" />
<include path="/**.css" />
</static-files>
This actually does two things:
Simple, right? Well, it didn't work. Also it has some rather unexpected side effects.
At first this additional configuration did absolutely nothing and didn't change any caching behavior. Apparently a definition of url-mappings in WEB-INF/web.xml, e.g. for a Java servlet or filter mapping overrules this configuration section.
I had the following filter-mapping in WEB-INF/xml.
<filter>
<filter-name>myFilter</filter-name>
<filter-class>org.whizu.www.RequestFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>myFilter</filter-name>
<url-pattern>*</url-pattern>
</filter-mapping>
This filter-mapping basically says that any file is to be considered as a resource file, to be served and handled by the application server. In other words, there are no static files. And if there are no static files, there is no Expires header to set.
In my case the solution was to change the url-pattern of the filter mapping to avoid the configuration conflict, and to avoid my static files being treated as resource files.
As a side effect of the static-files configuration, I noticed that my JavaScript files (.js) were no longer being served to the browser. The GAE Console logs were showing “No handlers matched this URL” warning messages for any URL ending with .js.
Actually my static-files configuration didn't contain a mapping for .js files. So implicitly this means that .js files are not static files but resource files to be treated by the application server. The application server however can't serve these files to the browser but warns that no handler is available to serve this file type.
The solution was to add all my static files and all my static file types explicitly to the static-files configuration in appengine-web.xml.
<static-files>
<include path="/**.gif" expiration="7d" />
<include path="/**.png" expiration="7d" />
<include path="/**.jpg" expiration="7d" />
<include path="/**.ico" expiration="7d" />
<include path="/**.js" expiration="7d" />
<include path="/**.css" />
<include path="/robots.txt" />
</static-files>
Don't forget robots.txt ;-)