Search:

Nginx Upload Progress Module

Note: this module is not distributed with the Nginx source. Installation instructions are below.

nginx_uploadprogress_module is an implementation of an upload progress system, that monitors RFC1867 POST upload as they are transmitted to upstream servers.

It works by tracking the uploads proxied by Nginx to upstream servers without analysing the uploaded content and offers a web API to report upload progress in JSON format. It works because Nginx acts as an accelerator of an upstream server, storing uploaded POST content on disk, before transmitting it to the upstream server. Each individual POST upload request should contain a progress unique identifier.

The JSON idea and the mechanism idea are based on Lighttpd mod_uploadprogress: http://blog.lighttpd.net/articles/2006/08/01/mod_uploadprogress-is-back

WARNINGS:

• this software has never been tested under load. It only passed a few lab tests, including various stress tests of many parallels uploads and myriads of progress probe.

• it is not intended to be deployed on production systems.

• when compiled with --with-debug, this module will produce high number of log messages.

Directives

• upload_progress

• track_uploads

• report_uploads

upload_progress

syntax: upload_progress zone_name zone_size

default: n/a

context: server

This directive enables the upload progress module and reserve zone_size bytes to the zone_name which will be used to store the per-connection tracking information.

track_uploads

syntax: track_uploads zone_name timeout

default: n/a

context: location

This directive enables tracking uploads for the current location. Each POST landing in this location will register the request in the zone_name upload progress tracker.
Since Nginx doesn't support yet RFC 1867 upload, the location must be a proxy_pass or fastcgi location.
The POST _must_ have a query parameter called X-Progress-ID (or an HTTP header of the same name) whose value is the unique identifier used to get progress information. If the POST has no such information, the upload will not be tracked.
The timeout parameter controls the time upload information are kept after the upload request has finished. This allows the upload progress probe to be able to get the "upload done" information or when an error as occurred. Usual value is 30 seconds.
Warning: since version 0.4, the track_uploads must be the last directive of the location (i.e. it should be placed after any proxy_pass or fasctgi_pass).

report_uploads

syntax: report_uploads zone_name

default: n/a

context: location

This directive allows a location to report the upload progress that is tracked by track_uploads for zone_name. The returned document is a JSON text with the possible 4 results:

• the upload request hasn't been registered yet or is unknown:

new Object({ 'state' : 'starting' })

• the upload request has ended:

new Object({ 'state' : 'done' })

• the upload request generated an HTTP error:

new Object({ 'state' : 'error', 'status' : <error code> })

One error code that is interesting to track for clients is HTTP error 413 (Request entity too large)

• the upload request is in progress:

new Object({ 'state' : 'uploading', 'received' : <size_received>, 'size' : <total_size>})

The HTTP request to this location must have either an X-Progress-ID parameter or X-Progress-ID HTTP header containing the unique identifier as specified in your upload/POST request to the relevant tracked zone. If you are using the X-Progress-ID as a query-string parameter, ensure it is the LAST argument in the URL.

Configuration Example

http {
        
  # reserve 1MB under the name 'proxied' to track uploads
  upload_progress proxied 1m;

  server {
    listen       127.0.0.1 default;
    server_name  _ *;
                
    root /path/to/root;

    location / {
      # proxy to upstream server
      proxy_pass http://127.0.0.1;
      proxy_redirect default;
        
      # track uploads in the 'proxied' zone
      track_uploads proxied 30s;
  }
                
  location ^~ /progress {
    # report uploads tracked in the 'proxied' zone
    report_uploads proxied;
  }
}

Usage

This usage example is taken from Lighttd mod_uploadprogress module example:

First we need a upload form:

  <form id="upload" enctype="multipart/form-data" 
    action="/upload.php" method="post" 
    onsubmit="openProgressBar(); return true;">
  <input type="hidden" name="MAX_FILE_SIZE" value="30000000"  />
  <input name="userfile" type="file" label="fileupload" />
  <input type="submit" value="Send File" />
  </form>

And a progress bar to visualize the progress:

  <div>
   <div id="progress" style="width: 400px; border: 1px solid black">
    <div id="progressbar" 
       style="width: 1px; background-color: black; border: 1px solid white">
     &nbsp;
    </div>
   </div>
   <div id="tp">(progress)</div>
  </div>

Then we need to generate the Unique Identifier and launch the upload on submit action. This also will start the ajax progress report mechanism.

 interval = null;

function openProgressBar() {
 /* generate random progress-id */
 uuid = "";
 for (i = 0; i < 32; i++) {
  uuid += Math.floor(Math.random() * 16).toString(16);
 }
 /* patch the form-action tag to include the progress-id */
 document.getElementById("upload").action="/upload.php?X-Progress-ID=" + uuid;

 /* call the progress-updater every 1000ms */
 interval = window.setInterval(
   function () {
     fetch(uuid);
   },
   1000
 );
}

function fetch(uuid) {
 req = new XMLHttpRequest();
 req.open("GET", "/progress", 1);
 req.setRequestHeader("X-Progress-ID", uuid);
 req.onreadystatechange = function () {
  if (req.readyState == 4) {
   if (req.status == 200) {
    /* poor-man JSON parser */
    var upload = eval(req.responseText);

    document.getElementById('tp').innerHTML = upload.state;

    /* change the width if the inner progress-bar */
    if (upload.state == 'done' || upload.state == 'uploading') {
     bar = document.getElementById('progressbar');
     w = 400 * upload.received / upload.size;
     bar.style.width = w + 'px';
    }
    /* we are done, stop the interval */
    if (upload.state == 'done') {
     window.clearTimeout(interval);
    }
   }
  }
 }
 req.send(null);
}

Installation

This module is not distributed with the Nginx source.
You can download the ningx_uploadprogress_module here:
nginx_uploadprogress_module-0.5.tar.gz

After extracting, add the following option to your Nginx ./configure command:

    --add-module=path/to/nginx_uploadprogress_module

Then "make" and "make install" as usual.

Safari-enabling patch (uploadprogress-module < 0.3)

There is an incompatibility between this version of nginx_uploadprogress_module and the Safari web browser. The problem is caused by the fact that the rules Safari uses to "normalize" case of HTTP request headers varies from the rules followed by Firefox and IE, and by the fact that nginx_uploadprogress_module looks for the X-Progress-ID HTTP request header case-sensitively. See this article for more information, or If you don't care to actually read about it just download the patch to make the lookup case-insensitive.

This particular problem has been fixed since v0.3 which now uses case-insensitive lookup to accomodate Safari and Internet Explorer (Brice Figureau).