Automatic file minification on nginx

For our HiFi content management system we wanted to automatically combine and compress CSS and Javascript files. On complex sites, this minification can have a significant impact on page load times. You can read more details about the benefits and our template-side implementation in a previous post. Here I am going to discuss the technical details of how we set things up on our nginx server.

Getting Started

There is a very useful and stable Minify project on Google Code. Both CSS and Javascript minification libraries are included and they can easily be switched out if you prefer another library. The normal way to use Minify is to install it in your root directory and call its main file with a query string to specify the specific files that should be included: /min/?f=js/jquery.js,js/jquery.cycle.js,js/setup.js. However with this setup, even after caching the minified files server-side, the PHP interpreter is involved in delivering the file, which is relatively inefficient. Ideally, after a file is cached the first time nginx would serve the compressed file directly.

There is some sample code included in the Minify download that uses Apache url rewriting to take a more "normal"-looking url and point it to the Minified version. This was almost what we were looking for, except for two things: it requires a very specific url structure—with all JS and CSS files stored inside the Minify folder—and we are not using Apache. For HiFi, we decided on Nginx, as the configuration for running many sites off of the same code-base is much simpler. Nginx has a HttpRewriteModule, which is similar to Apache's mod_rewrite, so I figured there must be a way to replicate the Minify sample code in the new environment.

Aside: Template Tags

A quick preview of how HiFi handles stylesheets and scripts to give you a bit of context: We added two new tags to our templating language: {% css %} and {% js %}. These are designed to be used every time a script or stylesheet is included in a template, and they do everything needed to hook into the minification system. Each tag takes a comma-separated list of file names that will be included. For example, a standard js tag might be {% js 'jquery,jquery.cycle,setup' %}.

By default, these tags will output straight or tags for each file in the list. However, adding min as the last parameter of either tag will cause it to instead output a single tag containing a special url that will get the Minified version. This way, it is just a matter of adding and removing that single parameter to switch between development and production modes. You could even set up a global variable to change it for all the tags in a layout at once.

The url format for serving the minified file is scripts/jquery,jquery.cycle,setup_123456.mjs. There are three things to note here: The first is that we use the .mjs extension (or .mcss). That change is what tells Nginx to serve the Minified version. The second thing is the timestamp we are appending to the filename. Since we will be setting far-future expires headers (so visitors' browsers know to cache the files locally), we need a way to manually expire the files when we have changed them. Our template tag automatically resets the timestamp whenever the source files change, but you could also increment it manually. Finally, this script assumes that JS files will be in a folder called /scripts and CSS files will be in /styles. Eventually, it would be nice to be able to place files in any arbitrary location, but that makes things considerably more complicated.

How It’s Going to Work

For maximum speed, we are going to enable two different types of cacheing: of the minified files on the server and in the visitor's web browser. Whenever a particular combination of files is requested, nginx will check to see if there is already a cached copy. If there is—and there will be except for the first time after a source file is modified—the server will return that file directly, without involving the PHP interpreter at all. This operation is very fast, as it required almost no processing.

If nginx cannot find a cached copy of the file, it will pass the request off to our minification script. This code uses the Minify library to combine and compress the source files, returns the end result to the visitor's browser, and caches it for future use. It will actually cache two versions: one as plain-text and one that has been further compressed with gzip. Nginx uses content negotiation to determine which of the two should be returned to the browser.

What happens when you change a source file? Since we will be telling the visitor's browser to cache our asset files indefinitely, you will need to change the timestamp/version number on the file when you make updates. By doing that, you will invalidate both the server-side and the client-side caches and cause the files to be re-minified.

The Code

Nginx Configuration

Open up the nginx.conf file on your server. In the http block, add the following code:

# Logic for serving minified CSS & JS
location ~* \.(mjs|mcss)$ {
  set $domain      www.domain.com;        # Change this to your site's domain name
  set $root_fcgi   /var/sites/site_name;  # Change this to the public root folder of your site
  set $root_cache  $root_fcgi/cache;      # Change this to a folder in which to cache the minified files
  set $min_dir     /var/sites/min;        # Change this folder to wherever you put the Minify files
  
  include fastcgi_params;
  fastcgi_param SITE_ROOT $root_fcgi;
  fastcgi_param SCRIPT_FILENAME $min_dir/nginx-mininification.php;
  fastcgi_param PATH_INFO nginx-minification.php;
  fastcgi_param SERVER_NAME $domain;
  fastcgi_param CACHE_DIR $root_cache;
  
  root $root_cache;
  
  expires max;
	
  gzip_static on;   # You will need to have installed Nginx using the --with-http_gzip_static_module flag for this to work
  gzip_http_version 1.1;
  gzip_proxied expired no-cache no-store private auth;
  gzip_disable "MSIE [1-6]\.";
  gzip_vary on;
  
  # If there is not already a cached copy, create one
  if (!-f $request_filename) {
    root $root_fcgi;
    fastcgi_pass 127.0.0.1:9000;
  }
}

Make sure to set the four variables at the beginning to reflect the way your server is actually configured.

Minify and Cache

First, copy the Minify files to anywhere you like on your server. They do not need to be in a publicly-accessible location, since Nginx will be proxying requests to Minify behind the scenes. The Minify config.php file contains sensible defaults, but check it to be sure.

Create a new file inside the Minify directory, called nginx-minification.php. Paste the following code into it:

 $sources,
  'quiet' => true,
  'encodeMethod' => '',
  'lastModifiedTime' => 0
));
if (! $output['success']) {
  send404();
}

// Clear old cached files
if ($handle = @opendir($cachedir)) {
  while (false !== ($file = @readdir($handle))) {
    if ($file != '.' && $file != '..' && stristr($file, $filename)) {
      @unlink($cachedir . '/' . $file);
    }
  }
  @closedir($handle);
}

// Cache the output
error_reporting(0);
if (false === file_put_contents($cache, $output['content']) ||
  false === file_put_contents($cache.'.gz', gzencode($output['content'],9))) {
  echo "/* File writing failed. Your cache directory, {$cachedir}, must be writable by PHP. */\n";
  exit();
}

// And return it to the client
unset($output['headers']['Last-Modified'], $output['headers']['ETag']);
foreach ($output['headers'] as $name => $value) {
  header("{$name}: {$value}");
}
echo $output['content'];

Testing It Out

You should be ready to go. Try converting your script and style tags to use the new url format ("/scripts/jquery,setup_123.mjs" or "/styles/layout,typography,homepage_123.mcss"), making sure that the source files are in either the scripts or the styles folders. If it doesn't work, make sure all the paths in the Nginx configuration file are set correctly and that the cache folder exists. If you have suggestions or problems, please post them in the comments! I'm no expert on the ins and outs of nginx, so I am sure there are better ways to do certain things here.