I was in a bit of a funk today, wondering when I will get to start working on some truly challenging projects any time soon. For the last year or so, most of the programming projects I have been involved with boiled down to a lot of Excel macros or C/R/U/D operations in VB.Net. Mind you, there is no shame in this; it pays the bills. Some programmers are delighted with these kinds of projects, and there is nothing wrong with that. But I like a little more meat with my potatoes. So I started reflecting on some of my favorite programming projects from the past, to find out what they all had in common.

One great project I worked on (in terms of my enjoyment factor; personally I think the product was a dog and the marketplace did not want it) was writing filters for screen scraping software. The software was selective, it was made to extract information from various online databases. The job was a true challenge. In a nutshell, I would spend hours or even days analyzing the HTML code from search results, then crafting a king sized regex to parse the page. The debugging process involved changing maybe a few dozen characters in the regex per day and constant testing. What I really liked about this project was that it was highly creative; there was typically no one “right” regex to write, and the work was mostly searching for patterns. It was like being a treasure hunter. To this day, I still know my regex’s inside and out, thanks to that job, and I have a tendency to use regex’s for just about everything I can.

Another wonderful project I was on was a personal project for myself. I was writing my own shopping cart system. Like any good hacker (in the purest sense of the word), I was not satisfied to create an online shopping cart using existing components. I devised the most devilish project specs possible: no libraries at all (even standard ones), no database server, upload-and-go installation, and so on. At the end of the day, I ended up writing my own session handlers, logging methods, a flat file database system, a template engine, and emulating a significant portion of .htaccess’s functionality (including emulating the status code 403 system), all in under 2,000 lines of code. This project taught me a lot (especially about RFC 2616) about good coding techniques, and making the most out of what I had on hand.

In college, I had the opportunity to use StarLogo to put a scientist’s theories on ant behavior to the test. I closely modeled her theory in code, and then set it running. I also prepared a full post-mortem of the project. This project was a ton of fun for me on many levels. It was a great experience to model theoretical behavior in code. Even better, I was using code in an abstract level to create visual patterns and data sets to be analyzed. At the end of the day, I determine that the theory I was testing had a serious flaw, and by adjusting the program to work around it, I was able to make a much needed correction to the theory.

What all of these projects have in common is my love for writing code that goes beyond the C/R/U/D process, and involves deep logic. They also all required a substantial amount of planning and thought before the first line of code was written. None of them could have been created by the “drag-n-drop” or “shake and bake” techniques that are filling the days of more and more programmers. Don’t get me wrong, I am grateful that I no longer need to keep writing the same database access code into every project. But my passion for programming goes far beyond seeing a project reach an end user and getting a paycheck. I truly love problem solving and attention to details. I have written more “get data from the user, validate it, stick it in the database, and display it to another user” projects than I can count, but I barely remember any of them. Projects that involve less mainstream code techniques like heavy use of recursion, functional programming, pattern matching, those are the projects that I remember. If I did not love programming, I would not be doing it. And those are the projects that I love the best.

What are you favorite programming projects from the past?

J.Ja