Smartphones

Calibre: A free, open source ebook management solution that truly delivers

Donovan Colbert highlights Calibre's ability to scour web sites and compile documents that rival pay-per-view digital distributions.

One of the problems facing readers of digital content has been a lack of compelling models for delivery of digital magazines, newspapers, and other media outlets -- as well as an uneven platform distribution of those titles that have successfully made the transition to digital formats. Wired was one of the first big technology magazines to embrace distribution on Apple, and they were also a pioneer in making the digital subscription free to all print subscribers. Unfortunately, they achieved this milestone after I had already dumped the iPad for an Android platform ASUS Transformer.

With the release of the Kindle Fire, Wired finally arrived for Android -- sort of. It's more appropriate to say that Wired has arrived for the Kindle Fire. A long promised Android edition remains forthcoming and conspicuously absent. Driven by this ongoing disappointment, I searched for and found a pretty decent alternative. As a bonus, it delivers content from my favorite sites in a magazine-type format for free. If you haven't heard of Calibre, read on and find out if this solution might also work for you.

Calibre is a free, open source ebook management solution for Mac, Windows, and Linux. As an e-reader, it isn't much to write home about, mostly because the desktop OS platforms aren't well suited to e-reading. But Calibre has a trick up its sleeve. It's a surprisingly well kept secret that Calibre can scour web sites and compile versions that rival the pay-per-view digital distributions of the print periodicals published by these sites. Additionally, Calibre can be used to convert documents from and to various formats, allowing you to turn .epub documents into Kindle .mobi format documents, for example. These features work for a number of devices, including e-Ink readers, tablets, smartphones, and any other portable digital device you might want to read edocuments on.

Now, this isn't perfect. I've noticed that when I pull down USA Today by this method, I'll have stories in the table of contents that don't actually appear in my digital copy of the newspaper. Wired doesn't seem to update their web site in direct correlation with their print magazine, so I sometimes find stale and outdated articles. However, the price is right, and in the example of Wired, Calibre is really the only option that most Android users presently have available.

I Calibre on a "server" running Ubuntu 10 desktop that is always on. I've also tested it on Windows and Mac machines, and it runs virtually the same, regardless of the OS platform. When you first run Calibre, a Welcome Wizard appears (you can run this Wizard again later from the Preferences button on the main menu bar in the application). Here you will enter your language, the location of your library, the mobile device for your default reader platform, email addresses for documents delivery, and account information for your SMTP server.

To crawl web sites to create periodicals, use the "Fetch News" function along the top bar of Calibre (see Figure 1). Figure 1

Clicking the Fetch News button brings up a dialog that displays a list of pre-configured "recipes" that connect to selected web sites, along with tables of contents that links to articles from those sites. Type the name of a media outlet into the search dialog, click the "Go" button, and Calibre will return a list of all relevant recipes sorted by language and outlet.

Figure 2

In Figure 2 above, I searched for "Time," and Calibre opened up the 19 results in English to display all the recipes that come pre-bundled for media outlets that match the requestor Time. At this point, you simply highlight the periodical you want to create, and a tab appears that allows you to schedule the polling days and times for your digital magazine or paper. You can also click the "Download Now" button to immediately fetch the document (see Figure 3). Figure 3

The document that's created is frequently very complete and polished. It will generally include a cover, if available, and a table of contents that link to individual articles. In the case of USA Today, it creates a master table of content as well as individual table of contents by section (Top News, Technology, Health, Sports, etc.).

Of course, you can always choose to read your periodical on Calibre on your desktop or notebook (see Figure 4), but the real magic comes in your ability to transfer it to your mobile device to read offline later at your leisure. Figure 4

The document can be automatically sent to the destination you've selected during the initial setup wizard. If you have a Kindle, documents will be sent to your Kindle account on Amazon. Your final step is to go to Amazon, log into your Kindle Management page, and send the uploaded document from Amazon's servers to your Kindle device.

With Android, you send the document to your email account, open the email on your Android device, save the attachment, and copy it into your Kindle folder (assuming the document was created in .mobi format and you want to use Kindle to read it, because you can select the format and reader you prefer). It isn't a perfectly automated process, but once you get the hang of it, it's fairly simple to get the document to your device.

You can also choose to hook your Kindle or other device directly to the PC running Calibre and transfer the document via USB. To access delivery preferences, click the Preferences button in the main menu bar (refer to Figure 1). This will bring up a Preferences screen (see Figure 5). Figure 5

From the Preferences screen, in the "Sharing" section, click on "Sharing books by email." Another window will open up where you can select or deselect "auto-send," add or remove destination email address, and edit your mail account information (see Figure 6). Figure 6

If your favorite news site is not already in the list of news sources, you can even create your own recipe that will crawl that site and produce a custom e-document. This is really just the tip of the iceberg of Calibre's capabilities. A Google search for "Calibre Kindle News" will return a wealth of sites offering detailed advice and instructions on how to fully utilize the features I touch on here, and I encourage readers to fully explore those resources to get the most out of this application.

Calibre is probably the most polished, reliable, and useful example of Free and Open Source Software I've ever encountered, and it really delivers the goods it promises. The quality and convenience of paid subscription services generally exceeds what you'll experience doing it "on the cheap" with Calibre (it won't help you get around pay-walls), but if you're willing to put up with a few hassles and extra steps, Calibre can open up your device to a whole new world of self-generated, customized, and very inexpensive content.

Have you used Calibre or other ereading alternatives? Share your experiences, advice, and questions with us in the forums.

About

Donovan Colbert has over 16 years of experience in the IT Industry. He's worked in help-desk, enterprise software support, systems administration and engineering, IT management, and is a regular contributor for TechRepublic. Currently, his profession...

11 comments
CharlieSpencer
CharlieSpencer

I haven't tried this method of content delivery yet. Would Calibre be a good tool to get my feet wet, or does it assume the user is familiar with e-published content and knows the basics?

JimmacNOV
JimmacNOV

I use Calibre to organize pdfs, word documents, and books on my computer and from there my ereader (a nook in my case). Works very well. If you download a freebook, it helps search for covers and summary information. Excellent program. I don't use it much as an ereader, but have used the conversion functions. My experience with both operations is as described in this article.

dcolbert
dcolbert

Time is behind a paywall - so the magazine I created above actually had excerpts/summaries of the actual articles but did not offer the full article - this is an example of the kind of little frustrations you might encounter with Calibre. It isn't going to help you get a subscription for free if the online site requires authentication - and it isn't going to be a match for digital subscriptions that add enhanced or interactive features to their online editions. For what it offers to do, though - it is the best solution I've been able to find.

YetAnotherBob
YetAnotherBob

Thanks for the tip to this application. I do some reading on Android (phone) and I have wanted to be able to get more things in as epubs. Project Gutenberg has a lot of books from around a hundred years ago that I still want to read. they usually come in as plain text or HTML. Use of an epub is more convenient on the reader I have installed. I will definitly plan in installing Calibra. More articles like this please.

dcolbert
dcolbert

I didn't have room for this in the article, but you can also use Calibre to convert a document from one format to another. For example, if you have a non-copyright DRM-free or public domain book in ePub format and you want to convert it to .mobi format and send it to your Kindle, Calibre will assist you in this process. It is pretty straight forward. Add your .epub document to the Calbire library, right click on the title and select Convert Books from the pop-up menu. A submenu with options will come up, they're pretty self-explanatory. You can select Convert Individually or Bulk Convert. Select Convert Individually and a new screen will come up and you can select a bunch of options to fine-tune the output. Select your options (the defaults will generally be fine, you'll want to change the output format to your desired destination format) and click OK. You can also convert from .mobi to .epub and from and to about a dozen other formats. It is very flexible and powerful. Once the conversion is done, send it either to your Kindle address and then log into your Kindle account on Amazon to send it to your device - or e-mail it or copy it via USB to your non-Kindle device as described above. (You can also transfer via USB to a Kindle, bypassing sending the document(s) to Amazon's servers first and the step to send it from Amazon to your device wirelessly). The thing I like is that it opens up your device to allow you to populate it with content you control more directly. That is a huge limitation with Kindle as a device or an app. It gives you more granular control over what is in your library and where you obtain that content. This is a user-empowering application, and I think we need more ladders that help users climb over the boundaries of the walled gardens their device and platform manufacturers have erected. I think Calibre is a great triumph for the concepts of fair-use.

dcolbert
dcolbert

While Calibre is a great tool for the user experienced with eDocument formats - it does require some basic familiarity with the various formats and where and when they are appropriate depending on your e-reader platform of choice. It assumes you understand the difference between ePub, Mobi and other digital document formats. My gut feeling is that it might be a little overwhelming if you're just starting to expose yourself to the world of eReading - it is a pretty powerful tool. On the other hand, if you're involved in a technical industry like I.T. and you understand the basic concepts of file formats and compatibility - it shouldn't take you a lot of effort to get up to speed, and Calibre does offer all of the basics that will allow you to download, organize, convert and read a variety of inexpensive e-format documents for a variety of digital devices - plus it is a full fledged eReader platform for your desktop PC. You could inexpensively get your feet wet, copy documents to your Android or iPhone handset and save some money while deciding if your usage justified buying a Kindle, Nook or tablet. Provided that you're technically adept - it isn't the most complex application platform to jump into. I'd suggest you give it a try, and if you have any questions, go ahead and post here and I'll do what I can to help you out.

dcolbert
dcolbert

And you'll find that Project Gutenberg is exactly the kind of application of this software where Calibre shines the most. I've read a number of old HG Wells books and other old books over the last year or so - a lot of the classics I skipped when they were required reading in high school. Calibre makes it very easy to take those open documents and move them where you would like and use them how you want to. Thanks for the positive feedback!

CharlieSpencer
CharlieSpencer

For some reason, I just received the e-mail alert this morning regarding your post of ten days ago. Any chance you could elaborate on the differences between the file formats, or point me toward a good resource? This will be strictly a desktop PC experiment; I don't have an Android or iPhone, and I suspect the start-up costs for those will be more than purchasing one of the $79 Kindle currently being advertised.

dcolbert
dcolbert

I just took a graphic intensive PDF document and converted it via Calibre to .mobi and then sent it to my Transformer to load into the Kindle for Android app. (*phew, that was a mouthful*) I've got to say that for a graphic intensive kind of PDF document, it probably makes sense to leave the document in PDF format. Conversion to .mobi causes all kinds of formatting issues. But you're going to find this is a hit and miss proposition with the Kindle. PDF doesn't work awesome on Kindle devices, but conversion of PDF causes problems too. Document by document, you're going to have to decide which should be converted and which are best left in their native format.

CharlieSpencer
CharlieSpencer

I swap a lot of books with my father, and he haunts the used book stores for bargains for us. If he ever kicks off, I probably go digital within a year. Until then, I suspect we'll continue to kill trees.

dcolbert
dcolbert

If you plan on reading on a Kindle - that solves a big part of the problem. You'll be using the .mobi format if you create, convert or transfer documents from Calibre to your Kindle. Kindle uses a format called .azw, which is a modified .mobi format. http://wiki.mobileread.com/wiki/MOBI As far as I can tell, the details are all technical and probably transparent to the end user. .epub is the most common OPEN format for eBooks. ePub can support DRM encrypted books (you have to enter the credit number you used to purchase the book in order to unlock it, in every case I've encountered). Barnes & Nobel has been behind ePub since the start. Kindle does not support ePub as of this time. Almost every platform out there supports epub *except* the Kindle. You can read more at the following link: http://en.wikipedia.org/wiki/Comparison_of_e-book_formats With all that said, there is a reason why Kindle is the dominant reader and why Kindle can ignore the growing momentum behind .ePub (at least, at the present time). Amazon will allow you to send .PDF documents to your Kindle via .usb, WiFi or Whispernet (depending on which is applicable). But .PDF doesn't publish as well as you would hope on most Kindle devices. I've got a Kindle DX (the big ol' honkin' sized one) and it is still difficult to manage PDF documents on it. You can't resize the font and pages don't reformat right when you rotate from portrait to landscape and there are other inconveniences. I had a couple of tech books that included digital versions on .PDF. I sent them to my Kindle and found them nearly unusable. Best case scenario, it is far less enjoyable and far more limited to read a PDF format document on a Kindle, and Amazon only officially supports Kindle .azw or PDF at this time. You're better off in many cases converting from .PDF to .mobi to send and read on your Kindle device (remember, .azw and .mobi are interchangeable. .AZW is Kindle's version of .mobi.) Confused yet? Don't worry, once you get a device, set up Calibre, and start sending files around, it'll make sense to you in no time. It doesn't matter much if you go with a Nook, Kindle or most other brands of eReaders - Calibre should be able to help you get documents to the device from alternate sources. Basically, if you're going to go with some other brand than Kindle or Nook, do your research before you buy.