On Transfer and performance

When I started the process of rebuilding my application in Model-Glue, Transfer and Coldspring nearly 9 months ago, the DIY programmer in me moaned at giving up direct control over the persistence layer in using Transfer. But it was clear that if I was going to commit to going OO with my ColdFusion application, that Transfer gave me a lot of power and would save me from writing a lot of code by hand at the expense of a black box in my system, for both better and worse.

I must give great credit to Mark Mandel who has listened to my frequently crazy and incomplete bug reports, found and resolved many bugs ahead of the 1.0 release last week. When I first relaunched MotorsportReg.com six weeks ago, we were getting a lot of strange errors, particularly during application startup and under load. Having just checked my email this morning, it’s refreshing how empty the inbox is totally devoid of our automated bug report emails.

Transfer’s Yin and Yang

There is, however, one area in which Transfer is a clear and direct tradeoff: performance. In some ways, Transfer can boost your performance with its caching model that will return fully composed objects in just a few milliseconds that would otherwise require many database queries and createObject() calls letting you take the hit once for creating the objects and then speed along on subsequent use.

Sean Corfield has mentioned many times that using Coldspring as a transient beanfactory would be extreme overkill and not very performant due to all of the other things Coldspring does beyond createObject(). I am now coming to grips with Transfer performance for large batch operations or where the beans you are persisting will not be read back (frequently, if ever). These are cases where I am finding Transfer may also be overkill.

My Example

My application has an email blaster that organizers can use to send email to their attendees. I record each campaign with the subject, body and sender in the database and then separately record each delivery so that I can use a unique ID in my return-path header for automatic bounce handling and reporting for the original sender. Typically an organizer will send emails out to less than 100 people at a time but there are very valid scenarios where they will send 1000-2000 messages.

Before the conversion to MG/CS/Transfer, even these very large sends would complete in a matter of seconds. It was effectively an insert for the campaign and then one insert per delivery and spool the message to disk. Figure at 20ms per delivery + spool, you can send 1000 messages in approximately 20 seconds.

Since the conversion, we had received an increasing number of timeouts during blasts. At this point, I have our timeout set up to 1200 seconds to try and let these blasts go out and some are still not making it. I have done a bit of instrumentation using CFLOG and debugging to see how long each step is taking averaged out over 3 deliveries in a single campaign:

The absolute numbers aren’t that important; it’s the relative size of them that matters. The actual hard time to save the delivery data and send the message is right around 15% of the total time. Now, this is my dev/test machine, CF7/Linux running older hardware (dual P3 800mhz processors) after a recent restart. Here are some numbers from a production server which is dual 2.8ghz Xeons after it’s been running for a few days and is using ~600mb of RAM (presumably a significant part is the Transfer cache, set to Application scope):

You can see the faster hardware in the create() and the CFMAIL tag. I listed the three SQL inserts as they were all over the place; probably a result of other load on the box whereas the test server was only used by me. Clearly the 6ms INSERT time experienced on the test server (which points to the same database server) is feasible. Without being able to instrument Transfer internally, my only guess is that these much longer save times are related to the large cache size. Anecdotally, we do notice the site seems to run more quickly after either bouncing CF or restarting the MG application which includes a rehash of Coldspring/Transfer.

Conclusion

I have come to love Transfer over the past 9 months. The things it can do, especially if you are building smart decorators that create intelligent business objects, is quite powerful. It has especially helped to reduce duplication of validation checks and centralization of key business rules (like no duplicate email addresses in our system, for example).

But you don’t get everything for free. The overhead incurred for this functionality hurts high-performance operations. As I am scaling the learning curve with these frameworks, the saying “if you have a hammer, everything looks like a nail” comes to mind. With only a few months of experience with Transfer and now just six weeks of production history, the judgment to know when and where to use that hammer is still lagging.

With this research under my belt, I am going to revisit some key places in my application:

• Anywhere I run Transfer.save() in a loop to evaluate the potential total number of objects
• Anywhere I run Transfer.save() without first running .validate() on my bean (I use decorators for almost every object in my app with a .validate() routine)
• Anywhere I use Transfer to persist data but don’t interact with it in single-object form (e.g., I have an audit log system that I use transfer objects to persist but I pretty much exclusively display it in aggregate query output)
• Our JVM is currently using less than half the RAM allocated to it; if the size/age of the cache is a factor in the performance, switching the cache to be session scope may potentially resolve some of these issues (but will probably create others…)

These are probably “duh” moments for experienced users of these frameworks but when migrating an existing application to a new framework, my instinct was to convert everything consistently to simplify the process and take advantage of the new tools. The good news is that I still have my SQL-based code in subversion so going backwards should be easy.