Web Development

A dirge for CPAN


On my way out the door tonight, I stopped to briefly speak with a coworker. I noticed that he was perusing CPAN. For those who are not familiar with it, CPAN is the Comprehensive Perl Archive Network, the collection of Perl libraries. As far as I know, no one has even bothered to make something similar to it for Perl try to compete with it in the Perl world. [Edited 9/18/2007 {Justin James} for clarity]

I was surprised my worker was on CPAN for two reasons. First of all, this person really does not like Perl. Second of all, his project is in VB.Net. So why in the world would he be looking at CPAN?

Because if you need to write it, there is a darn good chance someone already wrote it in Perl and put it on CPAN. Apparently, this coworker uses CPAN the way most coders use Google -- as a source for example code.

CPAN is a positively amazing resource for the following reasons:

  • CPAN code tends to be of high quality (although it's quite often of dubious usefulness).
  • CPAN is beyond comprehensive. As of 9/17/2007: 12,168 modules encompassing a whopping 3,898 MB of code.
  • CPAN is a universal resource; everyone who has written enough Perl to put it on a server somewhere will put it on CPAN because everyone else puts their code on CPAN.
  • CPAN is simple to use. Not only is the Web site so low tech you cannot get lost on it, but the Perl package itself comes with simple tools to access CPAN.
  • CPAN bundles are easy to use and access, regardless of whether you download them, view them on the screen and save the text, or use the Perl CPAN tools.
  • CPAN code is self-documenting in the Perl POD format. Say what you will about POD (it has some warts), but it is a good format for libraries since it is built into the library and source code at the same time.
  • Perl is an interpreted language so, by their nature, any libraries and modules are open source, editable, and viewable.
  • The licensing terms are quite gentle; while each module in CPAN has its own license agreement, I cannot recall ever seeing one that did not seem to say, "do what you want, I don't care."

That is enough plumping of CPAN for Perl code for the time being. I must admit -- it has been some time since I visited CPAN. Then again, it has been some time since I wrote Perl that was complicated enough to need to go beyond the built-in modules.

The whole experience with my coworker made me ask myself: Why is Perl completely unique as a once mainstream (and still not obscure) language in having something like a CPAN? Sun never gathered mass with Java as a locus for community content for a huge number of reasons. PHP could have/should have been able to reach CPAN status with PECL and PEAR, but it is nowhere close to CPAN in any respect. Microsoft has made a few attempts with GotDotNet (which is currently being dismembered) and more recently CodePlex.

What did CPAN accomplish that no one else did?

I have no answer to that question. I know that of all of the systems out there, CPAN feels the least professional. The site is probably running on 10-year-old CGI code. The site itself has not been updated in more than five years with the exception of the contents. As far as I can tell, there is no corporate sponsorship of any kind. It does not support "workspaces" or "projects" in any way. There is not a drop of advertising on it other than a tiny logo and a link to the company doing the hosting. In a nutshell, it is like an FTP site with a rudimentary search engine on top of it. At the end of the day, CPAN is something that simply "just happened" without any conscious thought or decision. The more that big sponsors try to replicate CPAN or (even worse) to foster "community," the more they seem to fail.

CPAN is probably one of the top five reasons I miss working in Perl on a regular basis; the lack of an equivalent is one reason why I will never quite adjust to the .Net ecosystem or take a shine to it the way I did Perl. I could spend hours going through CPAN, finding obscure pieces of code (Klingon language parsers, anyone?), reading through them, and learning a lot. Most of the open source stuff out there comprises an entire project, and it is both too large to make much sense from and too broad in scope to learn specific ideas from. For instance, you might need to go through thousands of lines of code that deal with the interface and validation just to find the 100 lines of interesting logic in any given project.

I know... at this point I am Ted, and Perl is Mary; it's years later, and I am still hung up on her. But, in this case, I think there is something really special about the past that no artificial attempts to recreate will ever bring back.

J.Ja

About

Justin James is the Lead Architect for Conigent.

17 comments
Sasha Chorny
Sasha Chorny

Another good point about CPAN - cpantesters. CPAN distributions contain tests and these test are executed on install and by cpantesters that tests it on many different platforms. There is JSAN - similar attempt for JavaScript by Perl programmers.

dave_cross
dave_cross

Oh, and by the way, why call this a "dirge"? That's not a very flattering word. Did you mean "eulogy"?

Justin James
Justin James

... it is the right word. Another alternative I almost chose was "lament". A "dirge" is a mournful song. Since I like CPAN a lot, and miss working with it, in this case I am mourning the loss of CPAN and the lack of a similar tool as a regular part of my life today. :( J.Ja

jklowden
jklowden

I don't know if there is a right word. "Ode" might be better. The problem with "dirge" is that it has a strong connotation: that the subject is dead. I tuned in to see why you thought CPAN was dead. Take a look at Merriam Webster's thesaurus if you don't believe me. Every related word is connected to a funeral or mass.

Justin James
Justin James

"Ode" is accurate from the perspective of CPAN actually still being quite active, useful, and online. "Dirge" is correct as well, since it is no longer a part of my life in any way. Not due to any factor of its own, just simply due to what little work I do in Perl at this point does not require CPAN. Sorry if you felt the title was misleading (I can see how it could be considered as such), I probably should have spent a few more minutes at the thesaurus than I did (I started with "requiem"). On the flip side, this post managed to generate not just one, but two grammar-related threads (the other regarding the now-struck-out sentence), which is definitely a first in this space! J.Ja [Edited 9/20/2007 {Justin James}: corrected the word "slip" to "flip"]

apotheon
apotheon

I think the misunderstanding arises because he means that, for purposes of his professional life, CPAN may as well be dead -- and he misses it. He's not saying CPAN itself has died, though. Thus, from his perspective, "dirge" is the correct term. You're viewing the use of the term from a less subjective perspective, and see CPAN as alive and well whether you're using it or not.

dave_cross
dave_cross

"CPAN is the ???Comprehensive Perl Archive Network,??? the collection of Perl libraries. As far as I know, no one ever even bothered to make something similar to it for Perl" I think that statement contradicts itself. Probably a typo. "Because Perl is an interpreted language" Nope. Perl is compiled. Ok, so it's compiled each time you run the program, but it's compiled, not interpreted. There are important differences. "Do you know of any equivalents for non-Perl languages?" The idea for CPAN was shamelessly stolen from the TeX archive CTAN (http://www.ctan.org/). Perl people have tried a number of times to take the CPAN idea to other languages that they use. The most successful one is probably the Javascript repository JSAN (http://openjsan.org/). Dave...

Justin James
Justin James

Dave - That sentence is, beleive it or not, not a typo but one of the most awkward word kludges I've ever assembled. I've struck it out on the post and replaced it with something that expresses my intentions much more effectively. You are 100% correct about the internals of Perl. It's something I've known forever and a half now, but out of a combination of habit and simiplicity, I have always (and will probably continue) to refer to it as an "interpreted" language. For the majority of Perl usage out there, it effectively is an interpreted language, but you are also right that the JIT compilation does matter quite a bit, particularly for certain usages (for example, the C preprocessing system). Good looking out on my faux pas there! J.Ja

apotheon
apotheon

"[i]I think that statement contradicts itself. Probably a typo.[/i]" I guess he probably meant s/Perl/other languages/. "[i]Nope. Perl is compiled. Ok, so it's compiled each time you run the program, but it's compiled, not interpreted. There are important differences.[/i]" It's really a little of both. In fact, in some ways, it's similar to the way Java is run -- much as I cringe at the comparison. My understanding is that Perl source is compiled to something like a parse tree, which is then executed by the perl binary. Thus, it is JIT-compiled about halfway, then interpreted the rest of the way to execution goodness. edit: I decided to use perldoc to find more information. [b]perldoc perlcompile[/b] yields the following -- "[i] Perl has always had a compiler: your source is compiled into an inter- nal form (a parse tree) which is then optimized before being run.[/i]"

Justin James
Justin James

If you used CPAN, what did it have that other resources lack? Do you know of any equivalents for non-Perl languages? J.Ja

Tell It Like I See It
Tell It Like I See It

Why not start a .NET version? Maybe TechRepublic would be willing to host it?

dave_cross
dave_cross

One of the things that (in my opinion) makes CPAN so successful is the fact that everything on there is released under an Open Source licence (generally a GPL/Artistic combination like Perl itself). I'm not sure if that's in the rules, but it's certainly current practice. I'd be interested to see if something similar could work in a community (like .Net or Java) where the Open Source ethos isn't so deeply embedded. Maybe the Mono community have something like this already. Dave...

dave_cross
dave_cross

"Raise your hand if you have ever encountered an app that runs on Mono" I've seen a number of Mono applications. The two that come immediately to mind are Beagle and F-Spot. In fact, I've never knowingly used a .Net application that wasn't running on Mono. Of course, the fact that I spend most of my time working in Linux probably explains that. Dave...

apotheon
apotheon

I've seen a few Mono apps out there. Off the top of my head, the only one that comes immediately to mind is Beagle, the desktop search application for Linux systems. Of course, every single Mono app I've seen was something that I looked at for a few moments, said "Cool," and promptly ignored. I'm just not interested in the sorts of applications that are likely to get written for the Mono framework, I guess.

Justin James
Justin James

Dave - You are very correct in wondering if the .Net ecosystem is even close enough to the Open Source mindset to support an open repository of basic code libraries. My guess is no, for the following reasons: * There is too much money in the .Net world right now. * The folks who are really dedicated to Open Source (ie: developing something on their own time expressly for the purpose of open sourcing it) probably are not coding in .Net. * Not too many people seem to write .Net code "just for the heck of it" like Perl, Ruby, Python, etc. coders. As a result, most .Net code is proprietary code written for an employer, who probably is not keen on open sourcing it. * And the final killer: if you look at the CPAN stuff, the most useful items in there are libraries that fulfill basic "beyond the language" functionality, like CGI.pm. The .Net Framework (as well as the J2EE framework, for that matter, and many of the existing C/C++ libs) are so comprehensive, typically what is missing is a function call or a property on the existing framework, so the programmer just inherits the .Net or J2EE class, adds the one thing they need, and call it a day. I also have to (still, sadly) question how much truly serious spadework type work gets done in .Net. For example, how many people are using .Net for serious math (intense graphics processing, weather prediction, geographical analysis, etc.), where a good math library would be needed? To rephrase it, I beleive that the bulk of the folks who are working on code that would be generating widely reuable libraries that are not for functionality that the Framework already covers, have an investment in other languages (and often OS's) and are not too likely to be working in .Net. Now, that being said, I could see it from the Mono community, if Mono wasn't nearly lifeless. Raise your hand if you have ever encountered an app that runs on Mono. I have, Paint.Net. The authors stated that the conversion to Mono was a PITA. At the end of the day, part of what makes the .Net Framework so useful is encapsulating the decades of Windows API development out there, code that is stable and tested. When you go to Mono, you lose enough functionality from the Framework to take away many of the reasons to work in .Net in the first place. I know for me, I like .Net for two reasons: the CLR, to let me choose the right language for the job, and the fairly comprehensive Framework. Cripple the Framework (ie Mono) and most .Net developers would not use .Net. I know, it seems pretty off topic, but I think this is a big reason why we'll probably never see a .Net CPAN. J.Ja

rafapa
rafapa

If you consider TeX as a language, there is CTAN which predates CPAN.

Justin James
Justin James

... until last night, when another commenter said the same thing. Good looking out, and thanks! J.Ja