Thursday, February 14, 2008

Searching for ColdFusion Immortality

I was working on a real estate rental site and needed to have a calendar that would display the days the rental was available.

Now, there are lots of ways to do this. I wanted something that was simple and fast and, probably because I'm not particularly good with tricky algorithms, I was hoping for something kind of cool so that I could just sort of mention this awesome new technique in casual conversation with some of the Masters of Computer Science at the upcoming ColdFusion conferences, waiting for them to recognize me as a member of The Fraternity, worthy of being taught the Secret Handshake.

I played around with some stuff and came up with something I thought might fit the bill! Now, it so happens that I work with a fellow programmer who has a Masters degree in Computer Science, so I tried my technique on him.

"Sure," he said, but without the note of awe I was hoping for. "That's a good idea," but it was obvious that my idea was no great revelation to him.

Great minds, it's often been said, must deal with such petty jealousies. So, heedless of his lackluster response, I present it to you...

For any given month, the unavailable dates (either blacked out or already booked) are stored as a single number. That's possible if we assign all dates of the month with a unique number from the powers of 2.

For example...
the 1stof the month = 1 (or 2^0)
the 2d of the month = 2 (or 2^1)
the 3d of the month = 4 (or 2^2)
the 4th of the month = 8 (or 2^3)
...and so forth.

To come up with a single integer describing all unavailable dates, I just add up the "powers of 2" values for all dates on which the property can't be rented. So, if the first through the third are unavailable, I add 1 + 2 + 4, giving me 7. I can now store this number somewhere handy (a database, for instance).

In my calendar, I can use the first of the CF functions mentioned in the title of this post: BitAnd. This function takes two numbers and checks to see if both bits in each location are "on".

In my example, the number for unavailable dates for this month is 7. Now, let's see if the 2d of the month (corresponding with 2^1) is unavailable: BitAnd(7, 2). If so, I will get back a non-negative, non-zero number. (In fact, I'll get back 2, corresponding to the bit the two numbers share in common.)

Further, if I want to see if a range of dates is available, I can add the desired numbers up and BitAnd them against the unavailable dates integer. If all the dates are available, I'll get back a 0. That will provide me with a quick way of testing for availability searches.

That works pretty well! But, I need to keep track of the next 12 months worth of dates. Maybe an array would work well here, a one-dimensional array with 12 elements in it, each element having a struct that holds both the actual month number (e.g. 2 for February, 3 for March, etc.) and that number representing unavailable dates.

I could try something like this:


But that runs me into trouble. If I start with, say, month 7, September, and add 11 to it, I get month 18 and, while enlightened people call this month "Haltober", alas, the wider world has not caught on: I need to do something different.

A little code that says "If the lastMonth is greater than 12, subtract 12 from it" will work fine.
I await immortality...

The Pre-Post Mortem

Years ago, I worked for a company doing Smalltalk development. These were some of the brightest people I'd ever worked with; they were very committed to the projects they worked on -- yet the results were so often dismally the same: failure.

We developed a routine: after a failure, we'd repair to the local Mexican restaurant (this was L.A. where they know Mexican food!), drink Margaritas, and commiserate. Very often during these post mortems (lit. 'after deaths'), it would happen that someone had anticipated the failure before any coding was done.

Years later, when developing for myself the process that would become FLiP, I thought about those post mortems and wondered, "What would happen if we held the post mortem before the project began?" Could we actually discover the project risks before they made themselves so ingloriously known?

I began experimenting with clients. "Suppose," I began, "that a year from now, this project has turned out to be a failure. I know that we don't want to think of that, but humore me for there's method to my madness. The question is this: why did it fail?"

Well, it turned out that in many cases, someone was aware of the Big Risk, but in the normal flow of events, one doesn't raise unpleasant issues. The idea of the pre-post mortem is that it gives everyone permission to explore the negative side of projects: risk.

Once we started doing this, we knew where to focus our "risk mitigation strategies" -- where, in simpler words, Murphy was most likely to appear. Then, we normally devised a plan to work around the risk. And, amazing to me, the failures stopped.

Most of the time, they stopped because we saw what was coming and were able to head it off. A small percent of the time, we realized that the risk was so great that the project wasn't worth undertaking. The difference -- to our bottom line, to customer satisfaction, to developer morale -- was immense.

You might want to just try this on your next project. I hold pre-post mortem meetings with the client and I find they appreciate it -- once they get over the idea that I've completely lost my mind.

A Cautionary Tale

There's a great bit of irony that pertains to us developers: six months in the lab can save ten minutes in the library. I read it recently in a new O'Reilly book, Beautiful Code. It made me think about an unqualified disaster -- and the reason for that fiasco.

In 2000, having finished a large successful ecommerce project, I was called by the CIO of Toys R Us asking if I'd be interested in heading the ongoing development of their ecommerce website. I spent about 45 minutes talking with him. During that call, I asked what their philosophy of web development was. He told me, "We believe in hiring the best developers and getting out of their way."

Hmmm. At first, I thought, "What an enlightened outlook!" but that thought gave way to another: what if the FAA took that view: "We hire the best pilots and get out of their way." Well, with the chaos that would surely ensue, I'd get out of their way too!

The job would also require that I move to New York City during the development. I declined. Lucky for me: the site melted down during the Christmas rush, causing Toys R Us to partner with Amazon.

The failure of the site was a very public black eye for ColdFusion. When they relaunched their own site, the site was written using JSP. As I understand it, the main reason for the failure came from the fact that the developers decided to load the many thousands of products into a structure, hoping to get better performance than they would from a traditional relational database. Things didn't quite work out that way.

Now, the failure of the developers wasn't that they thought of this non-traditional solution. Rather, it was that they didn't verify their idea by trying the idea out during development by loading a structure with 20,000 sub-structures, accessed by many simultaneous users and testing the stress on the server. Had they done so, they would have encountered what the public did during that infamous meltdown.

I love the fact that they thought "outside the box". But a little bit of "R & D programming" as I like to call it would have taught them what they needed to know. And the results of that testing might have led them to another outside-the-box solution: an in-memory relational database, giving them the benefits of speed they sought while preserving the integrity of the server.

The line between a brilliant and a bonkers idea is often quite fine. We want brilliance. To get it, we just need to test our ideas to see which side of that line they fall on.

The Faster I Go, The Farther Behind I Get

Recently, I was developing a CMS. Normally, I wouldn't write my own, but in this case I had to: the requirements of the client prohibited me from using one of the excellent ones available.

I should further explain that my client was not the end client; we had been sub-contracted to create part of the web application for their client. The end client -- let's call them ABC Co. -- didn't understand software development and, in what I can only suppose was an attempt to keep the project on course, began issuing a series of impossible deadlines.

They were not impossible for us, per se, but rather for our client, who worked heroically to finish the project. The problem was that the client laid down the timelines before they determined exactly what they wanted!

Perhaps you've been in that unhappy situation. The result to our client was endless frustration. The result to ABC Co was greatly increased cost and, ironically, a guaranteed late delivery. Decisions were made and work was done prematurely, all of which had to be redone and then redone again when ABC Co finally arrived at what they wanted.

I observe in my own life that my own foolish choices are responsible for much of my misery and it appears that others share this same failing. Rushing a project increases rather than decreases the risk of failure. Frederick Brooks of IBM observed this almost thirty years ago in his famous "No Silver Bullet" paper. Some things like great wine, great cigars, and great software only suffer when the natural creation process is over-hurried.