bITs and blogs

When you add in the declining enrollment rates for Computer Science,
Engineering and Technical degrees at our US universities, it does look
bleak. However, if you throw in the fact that many of the baby boomer
era are nearing retirement age, and that many in Federal or government
positions are looking to retire in the next 5-10 years, there will be
great opportunity for IT people. You just have to be patient, and
develop up those skills. Try learning more about security, compliancy
issues (SOX, HIPAA, etc.) that all companies will eventually have to
face, those areas where you are weak and need to beef up. Look for
areas of opportunity within your company where you can improve upon
business processes, or simplify things for your users.

Most importantly, start thinking like a businessman. Treat the company
like it was your own! Learn more about operations and how things work.
Make friends with the accountants down the hall, or talk to the Sales
guys about how to improve their lives. IT is no longer about
implementing the latest and greatest technology, but making technology
work for the end users and the company. Take a class on making
presentations, or join some professional organizations to help build
your presentation/speaking skills.

Just my $0.02 worth as well! 

Dear Apotheon,

You say:

"the number of IT workers ... (is) in constant decline".

But the U.S. Bureau of Labor Statistics in their Occupational Outlook Handbook, 2004-05 Edition (at URL: http://www.bls.gov/oco/home.htm ) says that:

"Employment of programmers is expected to grow about as fast as the average for all occupations through 2012."

and

"Computer software engineers are projected to be one of the fastest
growing occupations from 2002 to 2012. ... Despite the recent downturn
in information
technology, employment of computer software engineers is expected to increase much faster than the average for all occupations..."

and

"Employment of computer support specialist is expected to increase faster than the average for all occupations through 2012..."

and

"Computer systems analysts, database administrators, and computer
scientists are expected to be among the fastest growing occupations
through 2012. Employment of these computer specialists is expected to grow much faster than the average for all occupations..."

According to their chart
(see chart below or at URL: http://www.bls.gov/oco/oco20016.htm) that
means that the BLS is predicting that employment of computer related
professionals will increase by 10 to 36 percent or more (depending on
specific occupation) during the interval of 2002 to 2012. In no way can
that be considered a "constant decline".

While I sympathize with your point of view and know that many IT
professionals have suffered hard times recently (myself included),
perhaps we should not descend into a state of complete gloom and
doom.

The limited, anecdotal evidence of IT unemployment that any of
us have as individuals is not a sound basis for decision making.
And the BLS data seems to refute your assertion.

Do you think that their data are wrong or that they are lying to us?
The BLS is in the business of collecting employment data and making
employment trend predictions. I trust their data. Don't you?

Regards,
lfeldman

Changing employment between 2002 and 2012

I have a couple of notes for the previous commenter:

First, notice that all of those quoted statistics say "expected" and
the like. They're prognostications, not hard and fast facts. They may
or may not come to pass as predicted and, even if they do, they may not
particularly apply to the domestic industry in the manner presented.
Lies, damned lies, and statistics: aside from the potential for
misapplying statistical analyses (projected need for programmers
doesn't necessarily mean increased domestic employment of them),
there's always the likelihood of a whole lot of spin. Government
industry prophets work to get people elected to second terms, and to
get extra funding for their bureaus based on a perception of increased
relevance, while private industry prophets work to get people to pay
them for their prognostications. Even when they're telling the truth,
they may not be telling all of it. Completely aside from all that,
though, there's my second point.

Second, notice that I was talking about the present and the recent
past when I said "decline", using words like "is", while the statistics
cited refer to the future, using words like "will". The two don't even
conflict. I don't see how that chart in any way contradicts my
reference to employment in the IT industry being in decline.

I'll throw in a bonus point for you: Third, you might take note of
the fact that those projected statistics refer to specific narrow job
types, not to the IT industry as a whole (not all of us are
DBAs, for instance).

I'm confused by something.  Whenever I hear complaints about the IT industry I hear the decline of the programmer.  Since when is the IT industry only about the developer?  In the 17 years that I have been directly involved with computer related technology I have seen three different areas.  Developer, Break/Fix technician and Network Administrator. 

When I graduated High School I was on my way to become a programmer.  The Navy got ahold of me and my greed set in.  That is another story and I departed the technology industry.  A few years after my discharge I got back in as sales and progressed from there.  To make a long story short I now have my own business as a Independent Information Technology Consultant. What I mean by that is that I suggest what to buy, install what was bought and repair what needs repairing.  I do this for small businesses and residential customers.  Just because I do not create or manipulate software does not mean that I am no longer in the IT industry.

The part of IT I?m most disillusioned with is the
management.  I used to manage an IT
department of 15 people.  Small, by a lot
of accounts, but we struggled to support a fairly large infrastructure, mixture
of platforms, etc. that was not only giving a lot of work-related satisfaction,
but kind of fun at the same time.  It was
a great team effort.

As the quantity of PCs, Macs and the infrastructure grew,
corporate began downsizing IT.  They
added IT Directors to head departments that had little if any hands-on IT
experience.  Their main focus was cutting
costs.  I figured this would come through
advancements in the industry, but it just turned into a hatchet-wielding
nightmare, beginning with basic support jobs and ending at my original manager
position being turned in to a supervisor/analyst position, to no position at
all.  The IT department went from a place
with great teamwork, support and customer service to one of sheer drudgery,
worry and ignorance.  It was also pared
down from the original 15 to five?maybe two of which have any talent at all?or
at least ?seasoned? enough to continue aiding in the work.

That?s what you?ve got left to work with.  Employers looking for immediate expense cuts
running the risk of a long and more expensive future with antiquated talent and
equipment that will drive costs even higher. 
Prospective employees just out of college with no work experience that
expect $50K ? $60K per year to start or ones that have been around the block
and weeded out of organizations because they didn?t fit in or just couldn?t do
the work.

If I sound bitter, it?s probably because I am.  I?m one of those IT ?professionals? that
struggled to stay with their company?the same company mind you?for about 17
years.  They cut my resources so much
that I had to leave or go insane?but come to think of it, I would have fit in
perfectly once the insanity hit!

I agree with the general thread here. However, it is not just IT in this position. There is a trend in all business that the business itself creates these magical positions that have no real added benefit to the company. I work for a major (some say 'the") manufacturer of microprocessors and we have probably a 50% overload in management and engineering people. The "boots on the ground" numbers seem to be the real shrinking factor.

As far as statistics goes, remember, these are the same people who say inflation is at 3%, while I have calculated this years rate (mainly energy, interest rates and costs passed on to me by others because of these two) at closer to 25%. So their accuraccy is as good as a dart board.

I do a lot of freelance IT work for people in a one on one way, and make a little at it, but will rather take that than work to support the bloated management structure American Companies seem to think they deserve and need. Look at a lot of your daily interactions and you will see almost all of the symptoms illustrated in many areas, from automobile repair to tech support for devices you have purchased. The quality and effort to take care of the customer is the first casualty of the "herd" management hired today, and the result is such brilliant thinking that results in the offshoring of those pesky "support" people. I invite you to go look at Dell's wonderful reputation as seen on the hundreds, if not thousands of posts on the various web sites devoted to just "Dell Hell". A vast majority of that comes from their offshoring. Dell is the poster child for poor management and bad decisions, and is illustrative of the comment of how cutting costs is akin to cutting their throats. IBM bailed out specifically because they could not fathome quality and customer support as a part of any product they make. Dell just hasn't imploded enough yet to get there.

Good overall points about a subject no "herd manager" wants to discuss, or can even envision.

LFeldman,
Do u have any idea where the BLS gets those figures? They get the data
they base those numbers on from businesses. As the person who reported
those numbers to the BLS for the company I worked for, I can tell you
that there is a lot of room for error. First of all, not all companies
that are required to report, do so. Secondly, the company I worked for
did not spend a whole lot of time making sure that the report I used to
generate the numbers, worked as it should. I pointed out many times
where errors were occurring. The general consensus was, oh well, who
cares. When I mentioned this to my contact at the BLS, I was told, just
give me what you can.

Reporting to the BLS is required, BUT there are NO ramifications if
businesses DO NOT report. So many companies do not waste their time.

As you can see, the numbers being given are a "best guess" and probably have no basis in reality.
Sincerely,
An unemployed IT professional

Yeah MS SQL Server will run high performance apps, but only if you are willing to scale out dramatically.

What it is is a kludge, and I am speaking from real-world experience as well.

I must agree that you are badly misinformed about SQL Server's
capabilities.  It is most certainly capable of enterprise-level
applications and I also speak from experience.  What information
do you base your opinion on?  SQL Server 2005 is even more robust
and I think three years down the road we will see that it has increased
MS share of the enterprise db market.

And I'm not anti-open source by any means.  But open source
products, in general, are NOT easy to use compared to commercial
rivals.  I think open source is by its nature decentralized to the
point that ease-of-use is one of the first casualties.  It can be
a huge pain to do even simple things as was also described above. 
There are, of course, exceptions.  I have found MySQL to be very
easy to use and as was pointed out, it is used in some very heavy use
environments.  Open source will survive and prosper and help keep
company's like MS on their toes, but they will never be dominant. All
the various Linux installs (which have morphed into essentially
different OSs based on a common core), the driver issues, etc. are too
much of a pain and kill you on TCO.  Plus the revenue models of
open source companies don't allow for the real R&D expenditures
necessary to compete with commercial products.  Just my opinion...

A lot of the discussion about the state of the IT industry today seems to ignore the past and how the IT industry in general conducted itself over the past 10 years.  In large measure, the industry itself is responsible for the broader business community pushing IT down the list of priorities and not giving it much respect (which is the general root cause of outsourcing and cost-cutting).  It is responsible because of the greed that pervaded the industry during the Y2K scare and the dot.com booms.  Businesses were repeatedly told that product X was the going to save them billiions of dollars per year.  OR that if they don't do Y, they were going to be obsolete and the business would die.  OR you have to buy Y.  Thanks for the cheque.  Oh btw, that also means that you have to buy A,B & C (which are more expensive than Y is).

My point is that after an era of broken promises, little ROI and being generally misled about the costs, benefits and implications of IT technology, how many people are still going to trust this industry?  Sure the industry has changed, and is addressing the issues.  But it will take time for the mistrust to dissipate.  When it does and when the business world has a better perception of IT, the industry will become a valued part of any large business, almost all medium business and most small businesses.  Until then suck it up, people and work to make things better.

Respect is not given, it's earned.

At the risk of sounding Pollyanna-ish, I'd like to point out that if
all else fails, everyone who contributed to this discussion thus far
could certainly find employment as writers!  How lucid can you
get?  I'm impressed.

You may be right about open source offering entry level opportunities where other such opportunities have been cut off. Conceptually, I don't entirely agree with your analysis. It's been said many times that companies get open source because it's "free beer". They like it because it costs almost nothing to get it legally. Just taking it to its "logical" conclusion it could encourage offshore outsourcing, since the company got the OSS for free/low-cost, why not continue the idea by getting low-cost labor to work on it? I put "logical" in quotes because my own gut estimation of the kind of analysis companies do before they decide to offshore I think is pretty shallow in most cases. They offshore/outsource the wrong stuff for the wrong reasons. Typically outsourcing is best when you're looking to increase quality of service, or just offload something that's not your core competency (not central to your business). I talked with the guy I'm working with now, who has a degree in economics, about this subject a year ago. He said that outsourcing just to cut costs is the last thing a company really should do, because in the long run, outsourcing actually will end up costing more than to do the same thing in-house. The question is, is it worth it? In the cases where outsourcing is used properly, the increased cost is worth it, because quality of service goes up as well.

brief note: I direct the attention of those who read this blog post and immediately think "But I use MS SQL Server in a 'serious, high-load enterprise implementation!'" to some follow-up commentary in the second half of my next blog post, entitled "failing to understand how it works". The second half of that post bears on this one, and the misunderstandings inherent in the sort of contradictions I've received. Ultimately, I think people who make a career of supporting MS products (a challenging career that demands a certain amount of professionalism to perform well, to be sure) end up with a very limited perspective in terms of what constitutes certain economies of scale. I have never, ever, in my entire life, been told "I work with MS SQL Server in this situation," where "this situation" compared to the loads I've seen handled well by some of the other DMBSes I've mentioned. MS SQL Server really seems to push its limits at about the level where Oracle (for instance) begins to make economic sense. Something like PostreSQL significantly overlaps both scales. Go read the other blog post before posting about how deluded I am. Thanks.

I just don't agree.  While it's true that entry-level jobs are being outsourced; have you not heard what's going on in the support industry.  The offshore support is so poor and there is so much repetitive training involved, not to mention supervision, even short-term savings are not being realized by companies. 

I predict a huge return to entry-level jobs in first-world countries.  We have always had some sort of open-source software!  This is not new.  My advice:  Don't put all your eggs in one basket.

Not Just IT

I agree with the comments on outsourcing, but I also want to point out that it is a much wider ranging problem than IT and IT was not the first industry affected.

Look at the PC in front of you.  How much of it is US manufactured?  The monitor, the hard drives, ICs, and motherboards are all manufactured off shore.  The US no longer manufactures TVs.  The automobile industry is in decline.  Steel is imported.  In a recent article in the Washington Post (11/6/05), companies are starting to export health care.  In some border locations, companies are requiring US employees to go over the border to Mexican doctors' offices.  That some of the employees actually prefer Mexican doctors to US assembly line medice is another, related issue.

Meanwhile, we are told that this is good for us.  It is a good thing to send these jobs overseas.  We each should retrain for an ever-shrinking set of well paying jobs.  You don't have bread, why not eat cake?  I am proud to say, though, that our Texas based politicians do draw a line.  We can offshore everything except oil!  Here is where we take a stand!

There has been an ongoing dehumanization of business in America.  Loyalty, trust, reliability have been replaced with clever manipulation on streadsheets and accounting ledgers.  This is the uderlying problem, and though I have respect for open source software, it is not going to resolve the problem.  Businesses need to rediscover service to society and abandon PR spin and accounting tricks.

I remember the naive excitement of all things Web and Dynamic, back in the early days of the dot com boom. I remember that Internet Explorer was the new undisputed champion that rose from the ashes of a brief, vicious, and very one-sided browser war. I remember DHTML, which some people were calling HTML 5.0. In those days, DHTML meant HTML and Javascript on Internet Explorer, since nobody who used DHTML could get everything to work on other browsers or, for that matter, much cared if it worked on other browsers. Oh, how times have changed.

A number of problems began to develop. People became increasingly disenchanted with the way Internet Explorer's handling of HTML 4.0 and Javascript changed from one minor IE revision to the next. People started to notice that Javascript was behind all those horrible, annoying advertising popups that were creating unremovable porn-site ad windows covering up everything else they were trying to do. People started to get hit with arbitrary remote code execution exploits made possible by unscrupulous use of client-side scripting with Javascript and ActiveX. People started noticing that, by coding only to the standards of a fully activated, unsecured, plugin-heavy, latest version Internet Explorer, they were losing increasing amounts of business. Something had to be done.

Ever-larger segments of the web development community were rethinking their approaches to putting together a website. Server-side scripting was becoming the thing of the day. A lot of people were going back to the temporarily neglected Perl/CGI, PHP started getting linked in people's minds with MySQL in a way that made them almost indivisible, and ASP became the Microsoft loyalist's web development rallying cry. Soon, other players started entering the scene: we had Python, JSP, and even a smattering of Scheme and Ruby server-side scripting.

Meanwhile, web browsers were changing, both in configuration and, eventually, in basic design. Opera gained a small, but extremely loyal (some might even say "rabid") following. People started simply turning their browsers' Javascript interpreters off. A lot of people simply didn't bother to install plugins like JVM and Flash. Konqueror and Mozilla started nickel-and-diming their way into a tiny little slice of browser market share   particularly Mozilla, which was based on Netscape code as well as being cross-platform compatible. The much ignored (by IE coders) World Wide Web Consortium's web standards started to become relevant, particularly to web surfers with disabilities and the developers who wanted their business. Netscape was revitalized, just a little bit, by the influence of Mozilla's rapid improvement on its original codebase. The Phoenix, then Firebird, then Firefox browser project spun off Mozilla and became the rousing mindshare success the Mozilla suite never was. Now, some browsers even allow you to turn specific parts of Javascript functionality on and off, and ActiveX is widely recognized as the unnecessary security risk it is.

People design pages using XHTML now (like HTML, but built from XML, and making more sense than HTML ever did) with CSS for client-side dynamic behavior. It became very unpopular and frowned-upon to use Javascript unless you were in the business of taking advantage of the credulous public to earn your ill-gotten gains as a spammer. LAMP   Linux, Apache, MySQL, and PHP (or Perl or Python)   has become a force to be reckoned with. Secure web coding that doesn't invade privacy and doesn't use plugins or Turing-complete client-side scripting languages has become the order of the day. XML, other than XHTML, is in a state of flux somewhere between the Next Big Thing and Unnecessary Fluff in the minds of most web developers, though for some reason it's seeing impressive growth rates as the format language of choice for a whole bunch of things for which it should never be used (like network protocols).

It's not over yet, though. Some things are happening. Agile web devlopment with Ruby on Rails is becoming very popular in certain circles, and seems to be the best server-side web development framework since the original webserver, at least according to the hype, and I'm inclined to agree. Google has returned the oomph to Javascript by proving that, even with all but the most basic functionality of Javascript turned off in your browser, you can design wonderfully dynamic browser-based web applications that run quickly and smoothly, all without stepping on anyone's toes. Suddenly, Rails and AJAX are the tools of the day. This is what everyone wants to learn and use: either AJAX or Ruby on Rails. At least, everyone with an eye toward the cutting edge of web development wants to use one or the other.

Do we really need this stuff?

Well, we do need a client-side scripting language, and Javascript is what we've got. Asynchronous data transport opens the activity of web design to whole new levels of dynamic eye candy functionality. AJAX is Asynchronous Javascript And XML, and even XML is sort of a necessity at least insofar as it plays a part in XHTML. I really don't think we need all of XML in there, though. This is an infusion of the "new toy revivified", where XML is having trouble making inroads because all it seems to do is duplicate the functionality of HTML and some CSS, with a bit of extensibility. It tends to make markup hard to read, and webpages hard to write. It has all the problems of a dynamic syntax with few, if any, of the benefits of dynamic execution. Maybe this isn't the way to go (aside from XHTML, of course — thank whatever you hold to be sacred for markup that makes sense).

I've been a pretty dedicated opponent of Javascript in web design for a while, but I'm rethinking that. My reasoning was that we didn't need a fully Turing-complete language interpreter in our browsers that had hooks into all kinds of crazy local system capabilities, the way Javascript did. With Firefox (for instance), though, we now have an approximation of what I said we actually did need: by making sure Javascript is turned on, but everything (or nearly everything) in the Advanced configuration for Javascript is turned off, you basically get the functionality of the language I felt we needed, with the limitations I felt it should have. It's not perfect, but nothing ever is. Combine neutered Javascript with CSS, and you've got a heck of a lot of functionality with very little risk at all. There's hope for Javascript yet.

I'm still not keen on full XML functionality. I think we'd probably be better off able to deactivate, or not even install, full XML support in our browsers, and only support XHTML. Maybe I'll rethink that some day, but for now that seems to be the case. We might at least think about designing without any XML aside from XHTML until such time as people start figuring out that unfettered XML use shouldn't be applied, willy-nilly, to absolutely everything under the sun, but considering the way people still haven't learned that Java isn't the all-singing all-dancing panacea Sun claims it is, I don't hold out high hopes that'll happen. Still, having the ability to turn off non-XHTML support for XML in a browser might have the same positive effect on the XML-heads that turning off Javascript support did on the guys in the late '90s who tried to use Javascript for absolutely everything, just so they could create ever-uglier webpages that took exponentially increasing times to load, introduced geometrically increasing avenues for security exploits, and made websites ever-more difficult to navigate. In the meantime, maybe we should focus on the dynamic functionality of our client-side design, and stop screwing around so much with a markup language with dynamic syntax when all we're doing with it really is producing unreasonable facsimiles of functionality other tools provide with far greater efficiency, elegance, and ease.

Couple this with server-side back end scripting that dramatically reduces the code you have to write without gutting its flexibility and functionality, or slowing it down to the speed of a Visual Basic crawl, like you can get with Rails. Maybe call our front-end, client-side tool AJACSS: Asynchronous Javascript And CSS. XHTML is assumed these days, anyway, and really you could as easily make do with old-school HTML 4.0 if you wanted to. The point of markup, after all, is just to point at remote files. CSS provides (almost) all the formatting you need, and Javascript makes it dynamic.

Yeah, let's see if we can do a lot with very little. Minimal cruft for maximum functionality. Speed and elegance over buzzwords and byzantine code.

For the love of all that's holy, don't use something just because it's on the cover of a magazine. That's how we ended up with server-side Java and arbitrary remote code execution with Javascript and ActiveX in the first place.

We'll call this Perrin's First Law of Web Development: Don't use tools you don't need.

It'll only end in tears, otherwise. Well, either tears or ugly, slow, spaghetti-code websites that are a significant security threat.

How in the world do you go from posting a humorous programming parody to being accused of converting people to the dark side? I must laugh. Also, what is with everyone adding a "yeah, but..." statement to everything they read? How about adding some useful comments like I am about to do?

Useful comment #1 - I have invented ROPE WAY BEFORE YOU! I was reinventing the ROPE in fact when you were the same age as me and writing complex algorithms in a REAL progamming language while I played around with SED/AWK.

Useful comment #2 - SED/AWK are precursors to Perl.  If you know perl, you pretty much know SED/AWK minus a few syntax differences.

Useful comment #3 - Why would you use SED/AWK when you can use Perl? Maybe there is a good reason, so please create your own blog with it so I can find it and read it.

Useful comment #4 - Avoid eating large quantity of prunes the night before you are supposed to get married.

There, I hope I have added value to this very good blog. 

I was checking my laptop's snort logs this morning, as I typically
do, and decided that one of the tables of events would be enlightening
for some of you out there who have incorrect assumptions about security
and technical worth of various technologies. Keep in mind, as you read
this, that the more poorly patched implementations of something there
are out there on the Internet, the more attack variations there are
going to be -- and the more vulnerable the technology is,
fundamentally, the more attack types per attack method variation.

Keep in mind, also, that this is just the stuff that showed up from
within the two different networks to which I connected this laptop
yesterday: one at home and one at work, both protected by firewalls.
Imagine the hilarity that would ensue in my snort logs if connected
directly to an ISP without any intermediary firewall.

%      # of  method
=====================================================
97.93 42783  MISC UPnP malformed advertisement
 1.50   655  POP3 TOP overflow attempt
 0.25   108  SCAN UPnP service discover attempt
 0.09    40  WEB-IIS %2E-asp access
 0.03    12  WEB-PHP viewtopic.php access
 0.03    11  MS-SQL Worm propagation attempt
 0.03    11  MS-SQL Worm propagation attempt OUTBOUND
 0.03    11  MS-SQL version overflow attempt
 0.02     8  WEB-PHP viewtopic.php access
 0.02     7  ICMP PING CyberKit 2.2 Windows
 0.01     6  WEB-PHP viewtopic.php access
 0.01     5  WEB-PHP viewtopic.php access
 0.01     5  WEB-PHP viewtopic.php access
 0.01     5  WEB-PHP viewtopic.php access
 0.00     2  WEB-IIS %2E-asp access
 0.00     2  WEB-PHP viewtopic.php access
 0.00     2  WEB-PHP viewtopic.php access
 0.00     2  WEB-PHP PHPBB viewforum.php access
 0.00     2  (http_inspect) OVERSIZE CHUNK ENCODING

Here's what some of the more difficult-to-parse stuff on that list means:

• MISC UPnP malformed advertisement \u2014 This is an attack on a
  Microsoft Windows Universal Plug and Play buffer overflow
  vulnerability. This type of attack is attempted by way of the Windows
  XP Internet Connection Sharing client. When you set up WinXP to share
  its Internet connection with other computers on the network, you are
  prompted to install this client on other Windows machines (Windows 98,
  98SE, and ME) or to use the already-present client on WinXP machines
  that will be sharing the connection. Thus, machines with WinXP ICS
  client installed and UPnP running are vulnerable to this buffer
  overflow attack. There is a patch for this, but as far as I'm aware it
  doesn't usually get installed due to the fact that with non-XP systems
  patching is sort of a lost art.
• http://www.snort.org/pub-bin/sigs.cgi \u2014 This is a generic overflow
  attack aimed at older, vulnerable implementations of POP3 mail servers.
  If you have a reasonably new version of a reasonably well maintained
  mail server, this should not affect you. Of course, a lot of computers
  (regardless of OS) have old, clunky Mail Transfer Agents running a POP3
  service on them. It's worth checking what you're running.
• SCAN UPnP service discover attempt \u2014 This isn't so much an attack
  as a probe to determine vulnerabilities. There's a myriad of UPnP
  vulnerabilities out there, and if any are found by this scan, you'll
  likely get hit by an actual attack shortly following this event. UPnP,
  by the way, is a Windows-only service. Just in case you weren't clear
  on that.
• WEB-IIS %2E-asp access \u2014 Microsoft introduced a new vulnerability
  to IIS with a hotfix meant to plug up another security hole some time
  ago. That new vulnerability allows remote access to Common Gateway
  Interface scripts running on the IIS webserver, such as ASP,
  ColdFusion, and Perl scripts. This is commonly done to gain access to
  usernames, passwords, and data source names, so that your webserver can
  be rooted.
• ICMP PING CyberKit 2.2 Windows \u2014 This is basically a "check to see
  if there's a target there" probe. An interesting point here is that it
  doesn't actually target Windows at all: rather, it is run from a
  Windows machine. If you're running a fairly well-featured firewall,
  such as iptables or pf, you can easily configure your system to simply
  drop all incoming ICMP requests. If you're trying to make do with
  something like Windows Firewall or ZoneAlarm, on the other hand, you're
  pretty much screwed. The world knows you're there.
• (http_inspect) OVERSIZE CHUNK ENCODING \u2014 This is indicative of an
  attempt at a denial of service attack on older webservers. It affects
  both older versions of IIS and older versions of Windows-based Apache.

I figure the PHP-based stuff is all pretty clear. These are
PHP-based forum script vulnerabilities, mostly related to phpBB and
mostly a result of some piss-poor language design and rather unskilled
coders using the language (PHP is so accessible as a language that any
knucklehead can write code in it).

With the exception of the MS-SQL and UPnP attacks and scans, and the
ICMP scan, this is all webserver-directed stuff. I don't have a
webserver running on this system, and on systems where I am running a
webserver I don't have any of these vulnerabilities. Many kitchen sink
type Linux installs, and basically all Windows installs, are however
vulnerable to some of these. I'm also not running a vulnerable MTA
(updated Postfix on this machine), and because my MTA is for handling
local mail only, it wouldn't be accessible to the POP3 attack in any
case. The PHP vulnerabilities in that list all require a webserver
running PHP scripts. The MS-SQL worm and overflow attacks all require a
Windows system running the Microsoft SQL DBMS. Since I'm running a
"lean and mean" Linux install with exactly what I want installed, and
nothing more, I lack these vulnerabilities on this system, and having
an iptables configuration that by default has DROP rules for
unsolicited incoming and forwarding connection attempts ensures that
this stuff would all just be water off a duck's back anyway.

How well protected are you?

I ran across an article and discussion from (much) earlier this year that I hadn't noticed when it was new. In particular, I refer to the suggestion that, because of spam issues, SMTP needs to be replaced. I have one word in response to this: balderdash.

Perhaps I should elaborate, though.

SMTP is a communication protocol, nothing more, nothing less. The recommendation of that article, and of many of the discussion posts that follow it, is that it needs to be replaced with a mail protocol that incorporates authentication and/or registration so that email security can be better assured. The article refers to layers of "band aids" being slapped onto SMTP in the form of spam and virus protection software. Some of the discussion posts refer to open relays and protocol field requirements that are too permissive.

A few points need to be made in response to these complaints, I think.

First, there's a distinct difference between a protocol and an implementation. A protocol contains a definition of what data must be exchanged for (hopefully quick and efficient) identification of communicating systems and format of data that is exchanged so that the receiver knows what to expect. Whether or not that communication is accepted by the receiving system is up to the receiving system's implementation configuration. In other words, issues with authentication and security, generally speaking, are the fault of a poorly locked down implementation on the receiving end. The data format exists in the form of the SMTP protocol for getting information to the receiving system: whether or not the data is accepted and passed on to users in the form of email is up to that system's setup. If you have a problem with security, perhaps you should examine your mail server software. The protocol doesn't solve the problem, just as the English language cannot be modified to solve the problem that people lie.

Second, once you start including implementation definitions in something, it's not longer a protocol: it is now software configuration. It's best to keep your software architecture definition and your protocol definition separate. There's a widely-known truism amongst software developers that modularity is good, all else being equal. It allows for greater flexibility, greater maintainability, and so on. If you want to see email fragmented into a slew of uncompatible forms that ensure people often cannot communicate with each other, go ahead and mix implementation with protocol: otherwise, leave well enough alone. SpamAssassin is data analysis, not a protocol "band aid".

Third, open relays make the Internet go 'round — not directly, but because they are the necessary symptom of allowing flexibility in a "free" Internet. The only way to prevent open relays is to actually firewall the entire Internet by routing all communications through a single, central point of failure. If you think email suffers security problems now, just try dealing with the security issues of painting a big red X on a single target that manages all email on the Internet. It'll be down more often than it's up, I guarantee it.

Fourth, permissive protocol fields are, again, an implementation matter — not a protocol matter. If you want to be able to ignore all domain names that don't fit the Internet standards, implement a mailserver that does so. No matter how you define the protocol, it must still be implemented. Even if your protocol definition clearly states that the last period-delimited segment of an FQDN suffix of the "From" field must be no more than three characters in length, that won't actually change anything directly since when implementing a mailserver package someone will figure the protocol definition covers the problem and won't hard-code the limit into the mailserver's packet acceptance definitions.

Fifth, even if you can magically (and I do mean "magically", since that's the only way it would happen) force all implementations to exactly match the standard — even Microsoft's — you'd then have the problem that you've just eliminated the flexibility needed for many small networks to be able to use TCP/IP in closed network environments in manners unforseen by the common end-user who can't see past his own nose and his direct connection to his ISP. Some of us run networks (private internets, with a small "i") that only touch the Internet (big "I") by way of a router/firewall, possibly with network address translation. Do you really want to have such a rigid definition that you can't have privately defined, privately implemented email networks that don't touch the Internet directly, but do have to deal with clients that touch the Internet? If you limit all email to officially designated top level domains, that means that suddenly you have conflicts between email from the Internet and email from a subnet on your local network, since both might have to be called blah@blah.com. As things currently stand, at least you can receive email locally from the Internet in the form foo@bar.com, your network (or internetwork) in the form foo@bar.commie, and your local computer in the form foo@localhost.

That's not to say that SMTP couldn't use an overhaul. It just doesn't need an overhaul for the reasons cited, and can't be overhauled in the manner suggested. What it needs is some adjustment to make it match up more accurately with the first letter of the acronym, which stands for "Simple".

In fact, judging by that article and its responses, it looks to me like some of these shortsighted, authoritarian-minded individuals want to replace SMTP with ICMTP: Intentionall Complex Mail Transfer Protocol. No thanks.

If you really think any of that would stop spam, anyway, you're kidding yourself. I guess I shouldn't be surprised by such misconceptions, though: the "there oughta be a law" crowd has always been more populous than those of us who have a clue why there isn't such a law.

In a previous entry to ITLOG, I mentioned that my secure iptables
configuration on this laptop provided me with protection against
attacks of the sorts indicated by my snort output. I'm following that
up now with some information on how that setup was achieved, what it
contains, and how you can manage such a setup without too much
difficulty in a networked environment.

In his own TR blog, jmgarvin provided some good tutorial information
on iptables setup and management from the command line, using Webmin,
and using Bastille. The beginning of it all can be found way back in
May of 2005 at this link target.
I'm not trying to do anything nearly that ambitious with this post, but
I am going to show how I manage my iptables setup for laptops at my
place of business. This is a quick and simple way to handle things,
generally speaking, particularly when you have a bunch of computers to
work with that all need to have the same, or similar, iptables setups.

The first thing I do, of course, is flush my iptables setup with iptables -F -t filter, iptables -F -t nat, and iptables -F -t mangle.
As jmgarvin pointed out, specifying "-t filter" isn't technically
necessary, since it's the default target for iptables commands.

Next, for the work systems, I created a reasonably secure iptables
configuration using all the various iptables commands at my disposal.
This is an operation you probably won't have to go through once you
finish reading this, because I'll provide a simple template file you
can use to set up your own iptables configuration.

After testing that configuration out for a little while, I output it
to a file using iptables management commands. As jmgarvin pointed out, iptables -L will give you information on the current state of your configuration, but I find that iptables-save
produces more useful output for many operations. This prints all the
iptables rules you've entered to STDOUT (which, typically, means your
monitor's screen). You can use a redirect to save that data to a file,
which you should save in an easy to remember location. On some setups,
the /root directory (root users's home directory) might suffice. At work, I tend to use /var/lib/iptables, though I'm considering changing that to /var/local or /var/local/iptables. Another option, of course, is to save it in /etc
somewhere, since it is arguably system configuration data. Save it
under a filename that makes sense for you and for where you're saving
it, so that you won't forget what it is based on the filename. If you
have it in a location that includes the term "iptables" in the path, a
filename like "saved" or "save.cfg" might suffice. Assuming you're
using "save.cfg", your finally path to the file might look like this: /var/lib/iptables/save.cfg.
That's not what I call it, but my choice is really fairly irrelevant.
To save the configuration data by way of iptables-save, navigate to the
directory where you want to store it, then enter a command like this: iptables-save > save.cfg. In case you're curious, you can use that ">" symbol (called a "redirect") to save the output of any command to a file.

Once you have that file saved, you can edit it to contain the
iptables rules you desire at your leisure, and use the iptables-restore
command to load them. By default, iptables-restore will flush your
iptables before rebuilding them, so you don't have to worry about
flushing the tables before using that command to ensure that everything
works the way you want it to. To configure iptables using
iptables-restore, simply use a redirect (in the other direction this
time) to point the contents of your previously saved file into the
iptables-restore command, like so: iptables-restore < save.cfg.
Because save.cfg is not a command or an executable script, but is
instead merely a file filled with data that can be used by a command,
it cannot be listed first at the shell when using iptables-restore. As
such, you must use the leftward-facing redirect to point it "backward"
at the command you are using. It might seem a little unintuitive at
first, but after using redirects a little bit you'll start to get the
hang of it.

Now, if that save.cfg file of yours is something you want to reuse
for a lot of other computers, you should save a copy of it somewhere
convenient. Then, to apply it to a new computer, all you have to do is
copy it to the new system and, on that system, run iptables-restore < save.cfg
just as you did on the first system. This completely replaces the
iptables configuration in use with the new configuration, and even
restarting the computer doesn't undo the change. It stays put until you
change it again.

Here's an example template of a save.cfg file for you, with some explanation:

*mangle
:PREROUTING ACCEPT [48436:11233990]
:INPUT ACCEPT [48436:11233990]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [29730:6162034]
:POSTROUTING ACCEPT [29730:6162034]
COMMIT

*nat
:PREROUTING ACCEPT [391:49336]
:POSTROUTING ACCEPT [1793:110951]
:OUTPUT ACCEPT [1793:110951]
COMMIT

*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [1418:147349]
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 631 -j ACCEPT
-A INPUT -p all -s 127.0.0.1 -d 127.0.0.1 -j ACCEPT
-A INPUT -j DROP
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -p tcp -m tcp --sport 22 -j ACCEPT
-A OUTPUT -p tcp -m tcp --sport 631 -j ACCEPT
COMMIT

This iptables configuration essentially causes your computer to
pretend, first off, that any incoming connections that have not been
solicited by you just don't exist. It also does the same for forwarding
packets. This is a generally good policy, using the :INPUT DROP and
:FORWARD DROP defaults. Exceptions can be created later for specific
ports, addresses, and so on. Meanwhile, for purposes of ensuring you
don't forget to allow something that your user can do, you should
probably use an :OUTPUT ACCEPT default. A more secure way to configure
it is to use :OUTPUT DROP with exceptions defined for behaviors you
want to allow, but that can get prohibitively difficult with end-user
client systems that must perform a wide variety of networking functions.

-A INPUT -i lo -j ACCEPT allows your system to accept all
incoming requests that originate at your own network adapter. This is
useful for things like testing your network configuration by pinging
yourself, getting local system mail delivered (like when your computer
wants to tell you something broke), and so on.

-A INPUT -m state --state ESTABLISHED -j ACCEPT takes
advantage of the stateful packet filtering capabilities of iptables to
allow you to function quite flexibly with default DROP policies for
incoming packets. This line basically states that any connection that
is initiated by you will be allowed to continue, circumventing
the DROP policies for incoming packets related to already established
connections. Otherwise, you might be able to send data to another
server, but never know whether it got there because when the server
tried to reply your firewall would just drop the packets without
comment.

-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT allows incoming
SSH connections. You might want to change that 22 to another,
nonstandard port number, and ensure that anyone that is supposed to
have remote SSH access to the computer in question knows to use a
different port when making such connections. For a system to which
nobody should ever have remote access, you should remove this line from
this file before using it. When you're more well versed in iptables in
the future, you might consider replacing this line with several lines
that define what sources are valid for attempts to connect to the
computer remotely, so that even on a nonstandard port no system cracker
is going to be able to use SSH to get in from the wrong source (such as
an incorrect IP address, et cetera), but that's well beyond the scope
of this article.

-A INPUT -p tcp -m tcp --dport 631 -j ACCEPT allows for
connectivity with network printers using CUPs. If that's not something
you have to worry about, delete this line. More complex, more secure
iptables rules than this can be used to do the same thing, but this is
a decent starting point.

-A INPUT -p all -s 127.0.0.1 -d 127.0.0.1 -j ACCEPT is another "allow me to talk to myself" line.

Based on the INPUT explanations I've just given, the OUTPUT lines
should be fairly self-evident. A lot more can be done with iptables to
ensure system security, but I'm only aiming to provide a starting point
for managing iptables configuration. This very short, simple
configuration is superior, security-wise, to every single
distro-default iptables configuration I've ever laid eyes on, despite
the fact they're typically a hundred lines long, give or take -- except
when they're empty or simply nonexistent.

Have at it, and good luck.

Every Linux user eventually runs across the concept of iptables. Early on, though, (s)he/it might not know what "iptables" means, or even have heard the term. I just wrote something quick about iptables management in this blog without ever explaining what it is. Bad me. I'll rectify that situation now.

I'm going to assume you've heard of a "firewall" and have some vague notion of what that means in relation to computers and networking. As a quick summation, though, a firewall basically provides and enforces rules for allowing or denying network access on specific ports, from or to specific networked computers, and so on. Most Windows users, when they think of a "firewall", think of Windows Firewall, ZoneAlarm, or what they tend to refer to as a "hardware firewall", such as the many rinky-dink router appliances that can be purchased from Circuit City, Best Buy, and so on.

Windows Firewall and ZoneAlarm (along with a slew of others), sometimes called "software firewalls", are in fact little different in concept from the so-called "hardware firewalls". In point of fact, the biggest difference between the concepts is the security that each provides. Because the "software firewall" is on the local system, it provides a reduced security potential: by the time unauthorized traffic touches the "software firewall", it has already touched the system you're trying to protect. That doesn't mean you shouldn't use such a thing, however. It's an extra layer of security, and used properly it can enhance the overall security of your network. You should never, ever consider it a substitute for a separate ("hardware") firewall, however.

Applications such as the Windows Firewall and ZoneAlarm are really pretty crappy, as firewalls go. Even ZoneAlarm Pro (much better than the free version, and staggeringly, incredibly, frighteningly better than the deeply flawed Windows Firewall on Windows XP systems) is not great as a firewall. Norton Firewall is better in some ways than the above, in that it is capable of providing better security, and worse in others, in that it is difficult to configure, hides much of what it's doing even worse than ZoneAlarm (and on a par with Windows Firewall), and in general has the potential to seriously screw things up. Ultimately, the major problem with all of these popular "software firewalls" on Windows systems is that they do not operate at a low enough level to provide really significant security. There are a couple of firewall applications for Windows that do provide a more fundamental firewalling capability, making use of Windows kernel socket APIs, but the Windows OS design and driver APIs provides for potential "leakage" that even these Windows socket-layer firewalls (such as the iSafer Winsock Firewall: http://winsockfirewall.sourceforge.net) can be worked around by a clever security cracker, depending on the sort of hardware you're using for network connectivity, what drivers you're using, and so on.

Ultimately, the problem with these Windows-based firewalls is that they're software that sits on top of the OS trying to get the OS to relinquish control of network packet control earlier than it really "wants" to so that the traffic can be filtered effectively.

Free unices tend to have a much better packet filtering model. Linux, for instance, has the netfilter project, which works on kernel-integrated network traffic filtering. The management system for that, which handles filtering rules for netfilter to apply and enforce, is called "iptables". The OpenBSD analog to iptables is called "pf". It happens that iptables (as well as pf) works extremely well as a firewalling system. While I haven't done an exhaustive survey, I'd say that probably at least half of the little "hardware firewalls" you run across in retail electronics outlets are in fact running a stripped-down embedded Linux kernel with netfilter, some running iptables and some running some wacky hybrid thing that replaces iptables just to make everything work "differently" somehow -- probably to frustrate the efforts of people who would like to have more hands-on control of how the router/firewall appliance is working behind the scenes. In any case, if you've used a store-bought router/firewall appliance, there's a reasonable chance you've used something running iptables for firewalling, even if you've never installed Linux on anything.

Because of the open, modular design of Linux (and other free unices, for that matter), kernel-integrated network packet filtering has become very easily implemented and improved over the years. This allows for a very close marriage of the firewalling capability of such OSes with the network interface itself, providing a basically impenetrable security model, in theory. The security you can get from this depends on your ability to effectively define firewall rules and the flexibility and functionality of the filtering rules management system (in this case, iptables), however.

There was a predecessor to iptables called ipchains. From what I've seen thus far, it looks like ipchains differed from iptables mostly in that it was a little more of a pain in the butt to work with, and in that it was stateless. What iptables being "stateful" means is that it can actually apply firewall rules based on the current state of network traffic: rules can exist that depend upon the amount of traffic you're receiving on a specific port, for instance, rather than simply blocking or opening that port. This makes iptables much, much more useful for ensuring system security than ipchains.

It's always a good idea to have a decent iptables configuration in place on a given machine, regardless of outside firewalls you may have. Each individual machine, depending on its uses, might have different packet filtering needs. As a result of this, the external firewall device should have a configuration that permits everything the most permissive of your local systems is going to need to do: each individual machine, then, can deny as much of that as it can get away with.

When you first set up a Linux machine, it should have an iptables configuration of some sort already in place. That might consist, in some distributions of Linux, of a pages-long set of complex rules designed to allow you to make use of hundreds of applications and services that you probably won't ever touch: this is the approach that "kitchen sink" distributions like Mandrake have taken over the years. Very minimal systems have a tendency to just set everything to "allow", giving an extremely simple, but essentially pointless, configuration with the assumption that the user will do something to change that.

There are user interface tools that can be used to manage your system's security profile in a higher-level, more abstracted, more "user friendly" manner than hacking iptables configuration yourself. These include both CLI-capable tools like Bastille, and GUI tools with pretty colors and clicky buttons like KDE's Guarddog. There are even Linux distributions whose whole purpose is to provide a GUI front-end to iptables with a fair amount of configurability, reasonably sane default configurations, and integration of the configuration interface with that for routing services and other functionality commonly implemented on a network firewall box. I find that it tends to provide a greater understanding of, and thus better ability to manage, network and system security to work with iptables directly instead. Thus, my devotion of an ITLOG post to the subject of simple iptables management.

If you've wondered what iptables is, and what you should do with it, I hope this has provided your answers.

elegant (adj.): characterized by a lack of the gratuitous

"There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies."

- C.A.R. Hoare

There's a long tradition of referring to the elegance of a system. In the IT industry, this tends most commonly to be applied to source code, and it is generally accepted that the more elegant it is, the better. Elegance is differentiated from other superficially good things in a number of ways, including the common assumption that elegance goes deeper, while these other "good" things are only good within certain constraints.

For instance, "clever" source code is good for its cleverness, but can be bad for maintainability -- mostly because clever code is often difficult to understand. Cleverness also falls short because of a simple principle first articulated in an email signature of Brian Kernighan's: "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."

Another example is object oriented programming. Probably ninety-some percent of the competent programmers out there are thoroughly sold on the concept that OOP is the holy grail of programming techniques, and any further advances in programming techniques are just fine-tuning OOP techniques. I think this common perception is an outgrowth of twenty years of corporate influence on the evolution of programming, where large numbers of mediocre programmers end up handling the same codebase over the course of its lifespan. Two orthogonal systems of minimizing the damage a mediocre programmer can do to a project have been introduced to programming practice with a great deal of success: version control and object oriented programming.

Object oriented programming isn't the holy grail, though. It doesn't in any way aid with the creation of truly excellent code. It simply aids in the avoidance of truly atrocious code, and even then only in an aggregate view of a complex project. When you start drilling down to the individual bits and pieces of a complex software project that has passed through the hands of a great many mediocre programmers, you'll start seeing atrocious bits of code that limp along just well enough to keep working, as long as they're strictly encapsulated and separated from the rest of the codebase (except for its API, of course). Encapsulation and modularity are good things in general, but they aren't immutable axioms of goodness.

One of the trade-offs with object oriented programming is that it encourages repetitive action and tedious effort in writing code. Have a look at some "enterprise" class Java source code some time and start paying attention to how much of it is actual program logic, as contrasted with how much of it is scaffolding imposed on the source by Java's object-orientedness. In fact, if you really want to understand what's going on with object oriented programming and other superficially "good" things in programming, I recommend you start comparing how easily one can produce short, elegant code in various languages, and pay attention to why one language produces a shorter, more elegant solution than another. I think you'll find some surprising facts come to light.

Of course, it's true that brevity is not strictly synonymous with elegance. In fact, Perl golf -- the practice of passing code around between programmers to see how short a given algorithm can be made -- is a thoroughly gratuitous sport, concerned little, if at all, with elegance. In pursuing elegance, it is more important to be concise than brief, though. In a general sense, however, brevity of code does account for a decent quick and dirty measure of the potential elegance that can be eked out of a programming language, with length measured in number of distinct elements rather than the number of bytes of code: don't confuse the number of keystrokes in a variable assignment with the syntactic elements required to accomplish a variable assignment. Armed with that definition of the term "shorter", you should be able to make some meaningful comparisons of the elegance possible when working with various programming languages.

In particular, you might notice that without using any object oriented techniques, Common Lisp and Perl produce much shorter examples of certain algorithms than Java and C++. Even if you cut out all the object oriented scaffolding in the Java and C++ examples, you still typically end up with a lot more code, as measured in discrete syntactic elements. Things like lexical variables and anonymous blocks (or, roughly equivalently, lambdas) tend to make for much simpler, more elegant solutions than imposing rigorous OOP structure. In fact, the more you examine the matter and make such comparisons, the more I suspect you'll come to realize that OOP itself has nothing to do with producing elegance, and everything to do with limiting opportunity for mediocre programmers to produce cruft and introduce bugs.

Elegance is about the gratuitous -- or, rather, avoiding the gratuitous. It's true that sometimes people disagree about which of two or more things is the "most elegant", but this arises from underlying assumptions rather than any true subjectivity of the principle. Each of us has a set of operating assumptions, some greater (meaning: bloated and cumbersome) than others. Where something conforms to one's expectations and assumptions, it is seen to not lack in elegance in that manner. Someone that does not have the same underlying assumptions might see the same thing as atrociously inelegant, but having a different set of assumptions would overlook similarly subjective quirks in another example that are, to the first person, inelegant.

Specifically, someone with assumptions derived from long indoctrination by the OOP crowd might overlook all the scaffolding imposed by a language like Java for using OOP techniques, and see something that takes up 50 lines of program logic and 150 lines of OOP scaffolding as elegant. Meanwhile, a long-time Perl hacker might take one look at that and see it as the inelegant monstrosity it is. This Perl hacker, on the other hand, might write 30 lines of procedural code to perform the same task, and the Java programmer might look at it and wonder why it isn't more modular, simplifying the program logic itself and making the whole thing more scalable for future code maintenance, thus rightly seeing the inelegance of the procedural hack the Perl programmer threw together.

This doesn't make elegance subjective: it only makes our individual perspectives on it subjective. If we can discard the assumptions of both the Java developer and the Perl hacker, and recognize the underlying principles of source code design that contributed elegance to each solution, we could probably turn the same set of solutions into something much, much simpler and more elegant, in terms of its program logic and cruft-weight. Unfortunately, languages like Java are not really suited to that sort of optimization for elegance: you really need a language more dynamic than that, such as Perl, Python, Ruby, or basically any Lisp. The more a language lets you define the language you're using on the fly, the more likely it is to allow an excellent programmer to produce elegance, which should really be the end goal of writing code, generally speaking: elegant solutions.

All really useful principles of programming, or systems design in general, seem to be practical, case-specific extrapolations from my fundamental definition of elegance. In short, they all seem to boil down to this one instruction: If it's gratuitous, find a way to get rid of it. For example, consider the Pragmatic Programmers' DRY principle -- Don't Repeat Yourself. In short, it is better to avoid repetitions of data and program logic in your code. Any time you find yourself having to repeat or rephrase something in your code, reinject data into your data model from wherever you have it stored, and so on, you're screwing up. Ask yourself whether DRY is really useful by reducing repetition in and of itself, or by reducing gratuitous repetition. After all, recursion and looping behavior might also fall within the definition of "repeat yourself", but I don't think anyone (sane) would ever recommend eliminating all loops and recursion from programs. Sometimes, you just need your program to perform a given set of instructions on a long list of slightly different items. Often, loops make source code more elegant.

This ties in very nicely with the more general, more philosophical concept of aesthetics, and that provides some understanding of why it is possible to look at source code and, without yet consciously knowing what's wrong with it, have an immediate intuitive reaction to its inelegance. That's not to say that something can't be aesthetically pleasing without being perfect in its elegance, of course. Instead, the ability to recognize some characteristics of elegance is what leads to an aesthetically pleasing perception of the subject.

Elegance is not about aesthetics. Rather, aesthetics is about elegance. Ostentation lacks aesthetic appeal, and is inelegant, because it's gratuitous. Simplicity is often not elegant either: if something is too simple, it is nonfunctional, and fails to achieve its aim. What makes something beautiful is not strictly simplicity, symmetry, complexity, or any other such characteristic. Instead, what makes something beautiful is that its characteristics are all appropriate to its purpose. Complexity can be exceedingly beautiful, as long as it's not gratuitous complexity, which is just chaos and confusion. Likewise, simplicity can be exceedingly beautiful, but if you make something gratuitously simple, you get dullness rather than beauty. Gratuitous simplicity is merely boring.

When you're writing source code, make it elegant. When you've written something, go back and look it over, and for each and every thing you've done you should take a moment to question whether it's really necessary, or even functionally desirable, to have it in there. You probably won't get it perfect, but you can at least make it awfully pretty, which is a good thing as long as you do so by addressing elegance rather than trying to disguise the inelegance of your code by conforming to formatting conventions without rethinking your program logic at all. Refactoring, in the end, is really just about looking with fresh eyes for any opportunities to introduce elegance by removing the gratuitous.

If you're unlucky, you may discover that making your code significantly more elegant might require rewriting it in a different language.

The following definitions are from Princeton WordNet:

elegant (adj.): of seemingly effortless beauty in form or proportion

gratuitous (adj.): unnecessary and unwarranted

Want to see who's next On the Soapbox? Find out in the Blog Roundup newsletter. Use this link to automatically subscribe and have it delivered directly to your Inbox every Wednesday.

Subscribe Automatically

Remember the Sony DRM fiasco? Of course you do. It's still going on.

Well, here's more to make you giggle. By the way, don't buy the new Coldplay album, entitled X & Y.

If you're still buying CDs from Sony and friends, you're not paying attention. Where do they find these morons to run their companies, anyway?

What are the important things about your computer setup? I'm fishing for answers. Some are obvious, of course, such as security, stability, and performance. What are the important features for you about the software you use? What's important about vendor choice? What sort of applications do you tend to need? What sort of support and documentation do you want for your software? What's important to you in a community of users? What are the things you wish were different about the computer configurations you're currently using?

In short: What's already done right, and what needs to be done better?

If at all possible, try to break it down into specifics. Just saying "Computers are too hard to work with!" doesn't tell me anything, but something like "It should be easier to change my disk partitioning!" would be useful.

I've decided, due to some discussions here and on programming
mailing lists, and as a follow-up to my own recent Soapbox post about
programming elegance, to post some examples of Hello World programs in
various languages with very brief analyses.

C:

#include <stdio.h>
main()
{
    {
        printf ("Hello World!\n");
    }
}

This program contains two or three lines of code, depending on whether
you count the include statement: I do, so I'd call it three. I'm going
to be kind to languages that require multiple lines and use braces to
enclose blocks of code, so I won't count the closing braces in this
example (or in other examples, like Java). More important than the
lines of code, though, is the number of syntactic elements used to
achieve your aim: in this program, depending on how you define discrete
syntactic elements, there are three to five syntactic elements. I'd
call it four.

Java:

class HelloWorldApp {
    public static void main(String[] args) {
        System.out.println("Hello World!");
    }
}

This program contains three lines of code. In this program, depending
on how you define discrete syntactic elements, there's anywhere between
eight and twelve syntactic elements here.

Perl, Python, and Ruby: I'm going to include all three of these in
one explanation, because the three are amazingly similar when you're
writing a Hello World, even though what's going on behind the scenes is
quite different for the three.
Perl:

print "Hello World!\n";

Python:

print "Hello World!\n"

Ruby:

puts "Hello World!"

Each of these languages provides a simple, one-line program as a Hello
World, with exactly two syntactic elements -- no more, and no less.
Perl requires semicolons as line terminators, while in Python you can't
use semicolons as line terminators, and in Ruby you can use them or not
at your leisure. Both Perl and Python require a newline escape
character (the "\n") to insert a new line after the "Hello World!"
text, while Ruby has both a "print" method that works the same way and
a "puts" method that automatically adds a new line at the end of the
string ("puts" means "put string").

Visual Basic .NET:

Imports System
Public Module modmain
   Sub Main()
     Console.WriteLine ("Hello World!")
   End Sub
End Module

This program contains either five or six lines of code, depending on
whether you include the Imports statement. I do, making it six. As for
discrete syntactic elements, this thing has between nine and fourteen
syntactic elements, depending on your definitions. I call it about
eleven to thirteen, depending on my mood on a given day -- after all,
I'm a hacker, not a linguist. C#.NET looks similar, but actually
accomplishes the same with four lines of code and about ten syntactic
elements, making it an easier language to use as well as being a
technically better language by far than VB.NET, despite all the "VB is
easy!" marketing hype to the contrary. The only thing easier about VB
is this: it's easier to write bad code, because its OOP features are
crap, and its structure is line-oriented.

In my recent essay here entitled Elegance, I made a number of statements about what constitutes elegance in programming. Because I wanted to make these statements accessible, I used examples. Java became the subject of much exemplifying. Predictably, considering the number of Java programmers and proponents out there, I started getting a lot of reaction to my comparisons of Java language features to those of other languages.

I'm actually just a touch surprised by the common assumption that I was making a categorical denouncement of Java. On the contrary: there's a lot that Java does right, and a lot of what I said that others are perceiving as pejorative is actually indicative of some positive regard for Java.

Given my druthers, I write code in other languages than Java. I don't tend to work on large, organized, long-lived software development projects. I'm not part of a corporate code factory writing "enterprise" applications, so the benefits of Java don't much apply to me, and in any case I don't find Java source code to be particularly fun to work with. I enjoy the sort of challenges that arise in programming: that can be helped or hindered by a number of factors, and I find that (for me at least) Java suffers from a lot of hindrances to that. The painful tedium of repetitive scaffolding generation required to write a Java application, for instance, gets old pretty quick. Just looking at a hundred line Java program's source can put me to sleep.

That doesn't mean the Java language is all bad. It provides some distinct benefits when working with a number of other programmers in circumstances where you don't get to choose your fellow programmers on the team, for instance: much as I love Perl for lone code hacking, it's not the language of choice for that same situation by any means. Perl does provide far greater opportunity for elegant program design, but it also provides far greater opportunity for inelegant program design. Sometimes, you're better off limiting your potential for poor, rather than maximizing your potential for good quality.

There's no law that says that Java's suitability for limiting the damage mediocre programmers can do requires that all Java programmers be mediocre. There are a great many quite excellent programmers in the world. In fact, the authors of The Pragmatic Programmer, one of the best books on programming practice ever written, use Java code quite extensively to illustrate their points in that book. They've written several other books focusing on Java, as well. I think they have more books related to Java than to any other two languages put together. Clearly, Java has its uses, and the Pragmatic Programmers prove that Java isn't solely the tool of mediocre programmers.

One of the biggest millstones around Java's neck, actually, is its implementation. The JVM implementation is broken: it combines the worst aspects of both compiled and interpreted languages (as opposed to the Perl way of handling it, with a hybrid compile/interpret-at-runtime implementation that works quite well), all in the name of Java's mythic portability. Unfortunately, as the scores of "write once, run nowhere" jokes attest, it fails in its goals rather too often.

Java is certainly not suited to "scripting" use. It's far too cumbersome as a language for that. If a good, solid compiler for Java were developed, though -- something that would put it on a level with C++ -- it would suddenly become a far more effective language in practice. In fact, it would begin to be much of what it already claims to be: an excellent application programming language. From a certain perspective, it already is an excellent application programming language in theory, but its implementation completely screws that up in practice.

Another one of the big millstones around Java's neck is the fact that it's billed, and used, as a panacea. My biggest gripe in this area is its use as a server-side web programming language. Whatever idiot first thought a language run in a bloated, clunky, slow, flaky VM optimized for portability should be used as a server-side technology was clearly sprinkling PCP in his breakfast cereal.

In short, Java has its uses, just as C, Perl, and Fortran have their uses. Just as I'd never recommend Fortran for lightweight system administration, though, Java should be used where it's best suited to be used, rather than simply everywhere. It has its strengths, but it also has its weaknesses. Don't forget that as you rush to join the knee-jerk reactions to my use of Java as an example of the differences in language design.

This is a brief bit of instruction in how to reply
intelligently to email. It is inspired by Tech Juggler's
somewhat humorous How to reply to an email
post, in which he carefully addresses how one can reply to an email
using the Yahoo! webmail interface. While I don't tend to have the
problem he has, with people simply not replying (maybe I'm just more
likable), but I do occasionally see problems with people replying
stupidly. That being the case, I have decided to give you guys some
instruction so you won't come off as a complete idjit the next time you
reply to an email. Here are ten things to do to reply intelligently to
email:

1. Don't top-post. Yes, it's true, these days most mail clients for
Windows and most webmail service interfaces place your cursor at the
top of the quoted text to which you're going to reply, but it is worth
your while to go the extra step of clicking your little mouse pointer
at the bottom of the message to add text in the correct place. The
reason you shouldn't top-post is simple: top-posting destroys logical
flow of conversation. It makes sense, at times, to add some kind of
editorial note at the top of the email about what you're going to be
saying, but for the actual reply text it makes far more sense to add it
after what the other person said. It is acceptable at times to do
inline replies, where you break up the other person's text into
sections with replies to each of them, but in this case it becomes even more critical for you to post your replies to each of these sections after
what was previously said, rather than before. There are actually
mailing lists and newsgroups out there where top-posting can get you
banned: it is that annoying. This rule ties in with the next, about
cropping text. For a demonstration of this principle in action, see
this example:
A: It reverses the normal flow of conversation.
Q: What's wrong with top-posting?
A: Top-posting.
Q: What's the biggest scourge on plain text email discussions?

2. Crop text of messages to which you're replying. Only keep quoted
text from a previous email in your reply if it is relevant. Nobody
likes to reread, for the sixteenth time, the thirty paragraphs of
preceding conversation when it doesn't even pertain to what you're
saying in your reply (or even to have to scroll past it for the
sixteenth time). Save bandwidth, annoyance, time, and reasons for
people to decide you're an aggravating nimrod. This is especially
important when your reply consists of something like "I agree."

3. Send plain text emails only, unless you absolutely positively
must send an email with other content ? especially on mailing lists.
People who use text-based email clients, like me, don't like getting
spaghetti HTML code in their inboxes. Also, you look less like a
spammer if you don't send HTML emails when you don't have to.
Furthermore, you're less likely to spread email viruses and the like if
you only send text-based emails (and you're less likely to get infected
by them if you view all emails as plain text without markup
interpretation). When all you need to do is get a collection of words
from point A to point B, there's little point in cluttering it up with
a bunch of nasty markup, and there are a lot of reasons to avoid it.
I've heard people complain about techies giving them guff about things
like HTML emails and other utterly gratuitous misuses of misfeatures,
but it strikes me as odd that people never pause to consider that,
being techies, they probably know something you don't.

4. When responding to email, and you get angry with what has been
said, take a moment to make sure you aren't misinterpreting what the
other person said. Read the email a second, and maybe third, time:
while doing so, do your level best to interpret the phrasing in a
positive, rather than negative, manner. Don't make an ass of yourself
by jumping to conclusions that simply aren't accurate. Tone doesn't
carry as well online as in person, usually, so it makes sense to be
more sure of your interpretation before flying off the handle at
someone over some perceived insult. It is permissible, and even smart,
to ask if someone means to be insulting most of the time, in case you
aren't sure. It is just downright stupid to always assume the worst
without confirmation.

5. When asking for help, be polite, gracious, and humble. Humility
isn't always necessary, of course, but it serves as a good substitute
for actually knowing the limitations of your own knowledge, and most
people aren't so good at recognizing those limitations. So: be humble.
Too often, I see someone new to a mailing list relating to some
technical matter show up and start complaining about what's wrong with
such-and-such, based on his experience with something almost unrelated,
because this new thing doesn't act the way he expects it to from that
unrelated experience. This happens with people moving to Ruby from
Python or Java, people moving to Linux from Windows, and so on. Realize
you're in new territory, and don't (yet) know everything. Act
accordingly. Realize that others are not likely to respond positively
to you if you don't take this very reasonable approach. You should also
be polite, gracious, and humble even when excoriating some troll for
his imbecilities, and perhaps enjoy using words the dimwitted,
unregenerate nematode probably doesn't understand.

6. Use a spell-checker. You should probably use a grammar-checker,
too. I don't use either, but I'm a spelling and grammar guru with the
English language, so I'm the exception. You, dear readers, are likely
to be the rule. I recommend finding better spelling and grammar
checking applications than Microsoft Word, as I see it make mistakes
all the time. You should also proofread your email by eye to ensure the
spell-checker and/or grammar-checker didn't miss something, and to
ensure you didn't do something stupid like accidentally attribute the
"Reply to an email" post to master3bs instead of Tech Juggler (What?
Me? No, I wouldn't do that!).

7. Use the reply button when actually responding to a previous
email. People using threaded email clients to help keep discussions
organized like threads to remain intact so they (we) have a better
sense of context. This is especially important when replying where your
email will be read by people who receive lots and lots of email every
day (like me). Pure chronological organization of email may be fine for
Grandma, but for someone like me subscribed to a dozen tech-related
mailing lists, some of which get more than a hundred messages per day,
threaded email is of critical importance in being able to find and
understand anything going on. These tools exist to make it easier to
parse large quantities of data very quickly. Try to avoid screwing it
up for us.

8. Don't use the reply button when you're starting a whole new
discussion. Why the heck would I want to see a thread about how to
configure Gaim to use the system speaker instead of defaulting to the
sound card crop up several layers deep in the middle of a thread about
electronic voting machine vendors having to share their source code
with state governments? I'll give you a hint: I wouldn't. That's
especially true if I decide I'm done with the voting machine thread,
but wanted to read the Gaim thread, and hit Ctrl+D in mutt (my email
client of choice). That deletes the entire thread, including your Gaim
subthread. In short, don't screw with email threading: reply when
you're actually replying, start an email from scratch when you're
starting a whole new conversation. Capisce?

9. There's a well-known rule in some circles, stating that email
signature blocks should be no more than four lines long, at no more
than 80 columns (characters) of width. There's a very good reason for
this: we don't want to spend more time reading your signature than your
email body text. Most people's signatures aren't nearly as clever as
they think. The exception, of course, is when corporate-mandated
disclaimers have to be attached to the ends of your emails, but that
doesn't mean you should create your own disclaimers that are longer
than four lines at 80 characters. Don't be a completely gratuitous
windbag, please. Now, the four lines rule is realistically a guideline
more than a rule, in most instances, but there are those who are more
strict about it than others, so plan accordingly. I find that it makes
sense to limit my signature blocks to no more than four lines at about
70 or 75 columns' width, in case someone's email client includes my
signature in quoted text in a reply so that it ends up getting shifted
to the right a little bit. The reason for the 80 columns' width, of
course, is traditionally to accomodate those who are working on
old-school 80 column displays. Be kind to your email-reading brethren
of all screen resolutions.

10. Use hard line-wraps at less than 80 characters in your emails. I
recommend something between 70 and 75 characters, to allow for padding
out when others quote your text in replies. It makes it easier to read
a plain text email when it is limited to about 80 columns' width than
when it runs all the way to the right-hand edge of the screen or, worse
yet, past it (if someone is using a mailreader that doesn't wrap lines
dynamically).

Bonus. Here's your eleventh piece of advice for replying
intelligently to email: Don't attach files, especially big files, if
they aren't necessary, or forward crap from spammers, or otherwise send
asinine content nobody wants to read. Seriously. Why would I want 47MB
of attachments and spam in my inbox because you decided to forward a
bunch of useless crap to me? Here's a hint: I wouldn't.

So, once you get the hang of the basic functionality of creating and
sending a reply, as instructed by TR user Tech Juggler, you can start
working on not coming off as a screaming imbecile when you do so. Thank
you for your time, and have a nice day.

Get your geek girl a t-shirt. She'll love it, I promise. Well, she'll love it if she has any taste. Here's an image.

For those of you too lazy to click the link, here's a smaller image:

Also for those too lazy to click the link, it says "Linux is like a bad boyfriend: It never goes down on you."

Here's where to buy it: Apotheonic Labs swag shop.