Make mod_deflate work reliably with mod_jrun and ColdFusion MX

My earlier post yesterday asked if anyone had it working and the real answer is “no”. Some people have it working by saying “compress everything except these file extensions” using a configuration like this:

<Location />
  SetOutputFilter DEFLATE
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
  SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
  SetEnvIfNoCase Request_URI \.(?:pdf|wav|swf|doc|xls|rtf|ppt|mp3|avi|mov)$ no-gzip dont-vary
</Location>

This approach falls down if you generate any content like PDF, SWF, or JPG from a .cfm page which can’t be safely compressed. Plus, you have to constantly figure out which file types are not OK to compress instead of saying which are known to be OK. It’s the wrong way of specifying something and in my application, was not an option. This approach details the proper way to make the following safe configuration options work:

<Location />
  AddOutputFilterByType DEFLATE text/html text/plain text/css text/javascript text/xml application/x-javascript
</Location>

The Bug in mod_jrun

With the help of Rici on irc.freenode.net:#apache, I found that when someone requested “/” rather than “/index.cfm”, the results were compressed. Because the behavior was different when mod_autodir was involved versus a direct request, I filed bug #43830 at Apache.org.

My man Rüdiger Pluem wrote back a few hours later and reported it was a bug in mod_jrun20.c (for Apache 2.0 and it still exists in mod_jrun22.c for Apache 2.2, recently released). On line 815, Adobe directly sets the content-type of the request like so:

r->content_type = pstrdup_lower(r, value);

Rather than use the proper Apache API call of:

ap_set_content_type(r, pstrdup_lower(r, value));

When ap_set_content_type() is called, it notifies Apache that the content-type has been set and then all of the corresponding output filters (of which mod_deflate is the one we care about currently) are executed properly.

Fixing and Recompiling

Simply replace that line and then recompile. I am on CentOS 4.x with CFMX 7 and Apache 2.0.x and recompiled this way first, but ran into this problem when restarting Apache:

Cannot load /opt/coldfusionmx7/runtime/servers/lib/wsconfig/1/mod_jrun.so into
server: /opt/coldfusionmx7/runtime/servers/lib/wsconfig/1/mod_jrun.so: undefined symbol: hexstrtol

Using these directions instead, everything went swimmingly and I had Apache running right away.

I am including the recompile directions here for posterity as the target website has been up/down/changed URLs a handful of times:

#!/bin/bash
export CFMX=/opt/coldfusionmx7
export APACHE_PATH=/usr/local/apache
export APACHE_BIN=$APACHE_PATH/bin
#CFMX connector path eg $CFMX/runtime/lib/wsconfig/1
export CFMX_CONNECTOR=$CFMX/lib/wsconfig/1
#stop apache
$APACHE_BIN/apachectl stop
${APACHE_BIN}/apxs -c -Wc,-w -n jrun -S LIBEXECDIR=${CFMX_CONNECTOR} mod_jrun.c
jrun_maptable_impl.c jrun_property.c jrun_session.c platform.c
jrun_utils.c jrun_mutex.c jrun_proxy.c jrun_ssl.c
${APACHE_BIN}/apxs -i -n jrun -S LIBEXECDIR=${CFMX_CONNECTOR} mod_jrun.la
strip $CFMX_CONNECTOR/mod_jrun.so

Update 2/2010: This same approach works on CFMX 7 and 8 and with Apache 2.0 and 2.2 (although some filenames may change – just look in the directory and put them all on the compile line).

I have done rudimentary testing to verify this works with HTML, PDF, XML, SWF and other formats; everything seems to be good. The only catch so far is one page with some jQuery where I load the page and there seems to be a 5-second delay between loading the HTML (130k, compresses to about 38k) and the rest of the page loading. Everything else seems to be blazing fast though and my RSS feeds are popping on the screen like crazy.

Update 1/2011: Just learned that this has not been fixed in ColdFusion 9. I recommend you go and vote for bug #79532 so it might be fixed in CF10. Also, in response to the comment exchange between Dan and I regarding content-encoding, it appears that Apache themselves in mod_negotiation.c use the same technique to directly set the content-encoding in the request rather than an API call like ap_set_content_encoding (which, AFAICT, doesn’t exist). Also, here are some compilation commands from Edge Web Hosting for Apache 2.2 on RHEL/CentOS:

/usr/sbin/apxs -c  -Wc,-w -n jrun22 /opt/jrun4/lib/wsconfig/4/mod_jrun22.c \
/opt/jrun4/lib/wsconfig/4/jrun_maptable_impl.c \
/opt/jrun4/lib/wsconfig/4/jrun_mutex.c \
/opt/jrun4/lib/wsconfig/4/jrun_property.c \
/opt/jrun4/lib/wsconfig/4/jrun_proxy.c \
/opt/jrun4/lib/wsconfig/4/jrun_session.c \
/opt/jrun4/lib/wsconfig/4/jrun_utils.c \
/opt/jrun4/lib/wsconfig/4/platform.c
/usr/sbin/apxs -i -S LIBEXECDIR=/opt/jrun4/lib/wsconfig/4/ -n jrun22 /opt/jrun4/lib/wsconfig/4/mod_jrun22.la
strip /opt/jrun4/lib/wsconfig/4/mod_jrun22.so