Skip to content

TechRepublic

  • Top Products Lists
  • AI
  • Developer
  • Payroll
  • Security
  • Project Management
  • TechRepublic Academy
  • Innovation
  • Cheat Sheets
  • Cloud
  • 5G
  • Big Data
  • TechRepublic Premium
  • Top Products Lists
  • AI
  • Developer
  • Payroll
  • Security
  • Project Management
  • TechRepublic Academy
  • Innovation
  • Cheat Sheets
  • Cloud
  • 5G
  • Big Data
  • See All Topics
  • Sponsored
  • Newsletters
  • Forums
  • Resource Library
TechRepublic Premium
Join / Sign In
Software

Building an RSS mashup with Yahoo! Pipes

By philwainewright February 12, 2007, 7:39 AM PST

Image
1
of 14

Building an RSS mashup with Yahoo! Pipes

Start from 'my pipes'

Start from 'my pipes'

Yahoo! Pipes makes it really easy to build a mashup of your favorite RSS feeds, even if you’re clueless about programming. Anyone can follow this step-by-step guide to see how it’s done.

First of all, log in to Yahoo! Pipes and go to the ‘my pipes’ page.

Click on ‘Create a New Pipe’ (I’ve circled it in red on the screenshot).

Building an RSS mashup with Yahoo! Pipes

A blank canvas

A blank canvas

Next, you’ll see a blank canvas. Don’t you just hate it when that happens? But don’t worry, it won’t take long to finish the job.

Notice the list of sources on the left-hand side. You can build pipes using any of these sources.

For your RSS mashup, you’ll be using ‘Fetch’, as shown in the next image.

Building an RSS mashup with Yahoo! Pipes

Drag-and-drop

Drag-and-drop

Click on the ‘Fetch’ module and drag it onto the canvas.

Fetch is a generic module that can read any RSS feed on the Web. You just need to tell it which ones.

Once you’ve dragged the module onto the canvas, you can fill in the URLs for the RSS feeds you want in your mashup (you’ll need to have the URLs ready to paste in).

Building an RSS mashup with Yahoo! Pipes

Fill in your feed URLs

Fill in your feed URLs

You can add any number of URLs to the Fetch module you’ve created (there is a limit, but it’s not documented yet).

To add a URL, simply click on the plus sign next to the word URL at the top, and a new blank field will open at the end of the list. Then type or paste the URL into the space where it says url.

If you make too many blanks, you can delete any spares by clicking on the x to the left of each one.

You’ll notice Pipes also added a module called ‘Pipe Output’ to the canvas. You can ignore this for now. We’ll make use of it later on.

Building an RSS mashup with Yahoo! Pipes

Save early and often

Save early and often

This is a good moment to save what you’ve done so far. This is the Web, after all. You never know when you’re going to lose your connection. ‘Save early and often’ is a good motto.

When you click on the ‘Save’ button at the top right of the canvas, a blue box appears top left (I’ve circled it in red on the screenshot). Fill in the name of your mashup and then click on the ‘Save’ button right next to it.

Building an RSS mashup with Yahoo! Pipes

Now you're programming

Now you're programming

Now that you’ve assembled your feeds in the ‘Fetch’ module, Pipes makes it easy to test and build your mashup. In the menu to the left of the canvas, clicking on the word ‘Operators’ pulls down a list of modules you’ll use to build the mashup.

But first of all, look at the bottom of the canvas and you’ll see a gray area labeled ‘debugger’. That’s where you test your modules. Click on the word ‘Refresh’ and it’ll test the output from whichever module is highlighted in orange. In this case, that’s the ‘Fetch’ module, and as you can see, it works just fine.

This would be a good point to see if you can get someone in the office to ask you what you’re doing. Then you can casually reply, “Oh, nothing. I’m just debugging a new software program I’ve written.”

Building an RSS mashup with Yahoo! Pipes

Selecting an operator

Selecting an operator

When you move your mouse over the list of operators, a help bubble appears explaining what each one does. You can filter using search terms or other criteria, do a Babel Fish translation into or out of a dozen or so languages, or even use advanced ‘For Each’ operators to add or change feed items.

We’ll be using the ‘Sort’ operator to rearrange the items in our feed mashup into date order.

Drag the ‘Sort’ module into an empty space on the canvas.

Building an RSS mashup with Yahoo! Pipes

Connecting modules

Connecting modules

Now you’ll see the pipes concept come to life. You need to connect the ‘Fetch’ module containing all your feeds to the ‘Sort’ module.

Click on the small circle at the bottom of the ‘Fetch’ module and drag it towards the small circle at the top of the ‘Sort’ module. You’ll see a pipe appear and drop into place, as shown in the next picture.

Building an RSS mashup with Yahoo! Pipes

Pipes dynamically adjusts

Pipes dynamically adjusts

Now that your modules are connected, wait for a moment and Pipes will automagically reconfigure the sort options to match what it finds in your selection of RSS feeds.

For example, notice how the first ‘Sort by’ option has changed from ‘Title’ to ‘title’ with a lower-case ‘t’. Pipes has replaced its standard option list with the actual field names coming from the connected module.

Building an RSS mashup with Yahoo! Pipes

Houston, we have a problem

Houston, we have a problem

Clicking on the down arrow shows you the full list of options. In this case it highlights one of the pitfalls of mashing up RSS feeds, because there are several different date tags in use, which I’ve circled in red.

The most popular RSS formats use a ‘pubDate’ tag to date feed items. But RSS 1.0 mostly uses ‘dc:date’ and the up-and-coming Atom format, favored by Blogger, uses ‘published’.

Pipes can’t sort feeds on all three of these date formats at the same time, although the design team are working on it and hope to have a solution soon.

Building an RSS mashup with Yahoo! Pipes

Sorting into date order

Sorting into date order

My workaround to the date format problem is simple but not wholly satisfactory: I’ve deleted from my ‘Fetch’ module the two feeds that don’t use ‘pubDate’. Once the Pipes team come up with a way of normalizing date formats I’ll be able to add them back in.

So I’ve selected ‘Sort by’ ‘pubDate’ in ‘descending’ sort order. This will put the most recent date at the top, creating a “river of news” aggregated feed.

Building an RSS mashup with Yahoo! Pipes

The final step

The final step

To finish off, connect the ‘Sort’ module to the ‘Pipe Output’. Click on the ‘Save’ button and you’re done.

You can always come back and change things later on. If you hover over a connection, you’ll see a pair of scissors appear, which you can click on to sever the connection. Then you can drag in a new module and connect it in place. Popular modules include ‘Truncate’, which restricts the length of the feed, and ‘Unique’, which eliminates duplicates.

For now, though, click on ‘Back to my pipes’ (which I’ve circled in red) to see your handiwork in action.

Building an RSS mashup with Yahoo! Pipes

Ready to run

Ready to run

Back on the ‘My Pipes’ page, you can now select your new creation in the list of pipes, and you’ll see a screen something like this.

Whenever you want to make changes, click on the schematic or ‘View/Edit’ to go back to the modules canvas.

To see how your RSS mashup has turned out, click on ‘Run this Pipe’ (which I’ve circled in red).

Building an RSS mashup with Yahoo! Pipes

See the results

See the results

Anytime you ‘Run this Pipe’ it’ll re-run the mashup based on the latest data collected from all of the feeds. This screenshot shows the results from the night before this gallery was published.

But as you can see from the ‘Get as RSS’ pulldown on the right, you can also feed it into your favorite feedreader or start page. You can even pipe it into any other Yahoo! Pipe that you build.

To see or pipe the latest results, or to view or clone the underlying modules, go to Simple RSS mashup – Loosely Coupled’s SOA blog feed (Yahoo! login ID required). I’ve changed the title, but everything else is exactly as shown in these screenshots.

  • Software
  • Account Information

    Share with Your Friends

    Building an RSS mashup with Yahoo! Pipes

    Your email has been sent

Share: Building an RSS mashup with Yahoo! Pipes
Image of philwainewright
By philwainewright
  • Account Information

    Contact philwainewright

    Your message has been sent

  • |
  • See all of philwainewright's content

TechRepublic Premium

  • TechRepublic Premium

    Checklist: Network and systems security

    Cybersecurity demands and the stakes of failing to properly secure systems and networks are high. While every organization’s specific security needs form a unique and complex blend of interconnected requirements, numerous security fundamentals almost always apply to each of these groups. It stands to reason that cybersecurity pros who effectively identify network and systems risks ...

    Published:  June 1, 2023, 4:00 PM EDT Modified:  June 2, 2023, 7:48 AM EDT Read More See more TechRepublic Premium
  • TechRepublic Premium

    How to host multiple websites on Linux with Apache

    In this guide from TechRepublic Premium we’re going to explore the various things you can do with a Linux server. We won’t leave out any steps, so you won’t have to refer to another tutorial to complete the process. The only step we will leave out is the installation of Linux, as we’ll assume you ...

    Downloads
    Published:  May 31, 2023, 4:00 PM EDT Modified:  June 1, 2023, 7:45 AM EDT Read More See more TechRepublic Premium
  • TechRepublic Premium

    How to deploy an application with Kubernetes

    If you want to deploy applications into a Kubernetes cluster, be warned — it’s not the easiest task. There are a lot of moving pieces that go into these scalable containers. Don’t you wish you had a complete roadmap, from start to finish, to walk you through the process of deploying the Kubernetes cluster, deploying ...

    Downloads
    Published:  May 31, 2023, 4:00 PM EDT Modified:  June 1, 2023, 7:51 AM EDT Read More See more TechRepublic Premium
  • TechRepublic Premium

    Cross-training tool kit

    The more flexibility you can create in your technology workforce, the better you’ll be equipped to manage tomorrow, whatever the future brings. Too often, we focus on helping our teams become technical specialists who know volumes about a single technology, but quickly lose sight of how that technology connects with others. This makes their skills ...

    Downloads
    Published:  May 30, 2023, 4:00 PM EDT Modified:  May 31, 2023, 4:50 PM EDT Read More See more TechRepublic Premium

Services

  • About Us
  • Newsletters
  • RSS Feeds
  • Site Map
  • Site Help & Feedback
  • FAQ
  • Advertise
  • Do Not Sell My Information

Explore

  • Downloads
  • TechRepublic Forums
  • Meet the Team
  • TechRepublic Academy
  • TechRepublic Premium
  • Resource Library
  • Photos
  • Videos
  • TechRepublic
  • TechRepublic on Twitter
  • TechRepublic on Facebook
  • TechRepublic on LinkedIn
  • TechRepublic on Flipboard
© 2023 TechnologyAdvice. All rights reserved.
  • Privacy Policy
  • Terms of Use
  • Property of TechnologyAdvice