Gzip

This week we enabled gzip for our mobile libraries. Gzip is a compression format widely used in HTTP networking. With gzip we saw over 10x reduction in the POST request body to upload our batched event data.

Our Tracking API uses (mostly) vanilla Go. Enabling gzip decompression was a breeze using the compress/gzip package (thanks Amir).

func (s *Server) handle(w http.ResponseWriter, r *http.Request)
  encoding := r.Header.Get("Content-Encoding")
  if encoding == "gzip" {
  	z, err := gzip.NewReader(r.Body)
  	if err != nil {
  		http.Error(w, "malformed gzip content", 400)
  		return
  	}

  	defer z.Close()
  	r.Body = z
  }
  ...
}

On Android, we can take advantage of GZIPOutputStream from the Java standard library.

void post(byte[] data) throws IOException {
  URL url = new URL("https://api.segment.io/v1/import");
  HttpURLConnection conn = (HttpURLConnection) url.openConnection();
  conn.setRequestProperty("Content-Encoding", "gzip");
  conn.setRequestProperty("Content-Type", "application/json");
  OutputStream os = conn.getOutputStream();
  OutputStream gzipped = new GZIPOutputStream(os);
  gzipped.write(data);
  ...
}

Adding iOS support was the most challenging of the three. There are no standard library APIs for gzipping data, so we pulled in the relevant code from Nick Lockwood's implementation on Github. The final snippet is tiny and fits perfectly as an NSData extension.

#import "NSData+GZIP.h"

- (void)sendData:(NSData *)data
{
    NSMutableURLRequest *urlRequest = [NSMutableURLRequest requestWithURL:@"https://api.segment.io/v1/import"];
    [urlRequest setValue:@"gzip" forHTTPHeaderField:@"Content-Encoding"];
    [urlRequest setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
    [urlRequest setHTTPMethod:@"POST"];
    [urlRequest setHTTPBody:[data gzippedData]];
    ...
}

Implementing it across our different code bases was surprisingly easy. We were up and running from discussion to implementation within a day's worth of work across our server and mobile libraries. And the savings definitely made it worth our time!