Filed under: Developer, Internet, Blogging, Web services, Yahoo!, How-Tos, Social Software
Aggregating myself with Yahoo! Pipes
I sat down to take all the data I spew at places like my own blog, del.icio.us, Twitter, and Flickr into one aggregated bowl of RSS love. The result of my feed union isn't an hour by hour diary of my day, but does create an ephemeral stream of things that are undeniably me. The same techniques could easily be tweaked to document your day to near pinpoint perfection, if you're so inclined.
The Pipes components you'll need are Fetch, Union, and Sort. Drag your first Fetch onto the sheet to get started.
Paste the URL of your first victim feed into the Fetch box. If the feed is valid, you should see the universal RSS orange logo pop up next to the URL.
Fetch doesn't work exactly the way I'd guessed, and because of that it took a little experimentation to make this work. For some reason, if you add all your URLs into one Fetch, the sort doesn't process them correctly and they will never meld into something that makes chronological sense. Drag a new Fetch to the sheet and repeat until you have all of your feeds entered.
Next, we'll drag in Union. Union allows you to put the feeds together in a way that retains any extended specific data contained in only one feed. For RSS newbies, think about the Union step like shuffling together a few decks of cards at once; All the cards will end up in one stack, but not in the specific order you want them.
Attach the Fetch blocks we created earlier to the top of the Union. A Union will allow you to connect up to 5 feeds. I haven't tried to connect more (yet) but I'm guessing that multiple layers of Unions could be used.
Now that we've unified the feeds, we'll use Sort to straighten them out. Drag in Sort and connect it to Union. You should also connect the bottom of Sort (the output) to Pipes Output.
The drop-down box to the left of Sort will show "Updating" for a few seconds. This is Pipes attempting to catalog all of the feed elements which you could use to perform your sort. Once the drop-down fills in, we'll select PubDate, the date and time which the element was published. We should also set the order to descending, so that the resulting feed contains every item from our originating feeds, ordered from new to old.
That's it! If we click on Pipe Output, Pipes should grab and massage all the data and display it in the preview pane.
My resulting feed takes information from my blog, Flickr, Twitter and del.icio.us and creates Aggregated Grant, a meta feed my friends and family can watch to get a better idea of what I do everyday.
With Halloween fast approaching, it's a great time to get in some practice defending your territory against zombies. In Graveyard Shift, you take aim at zombies and other creepy-crawlies, blasting them into splatters of cartoony green guts. It's a casual first-person shooter, and it's very easy to get the hang of - use the mouse to aim, click to fire. Graveyard Shift has at least 15 levels, and it might even have some secret stages I haven't unlocked yet.
They key to getting good at Graveyard Shift is learning to use ...
Reader Comments (Page 1 of 1)
robotrock@gmail.com said 1:48PM on 2-20-2007
Or you could do this much easier with aim...
http://buddyupdates.aim.com/settings
Reply
Grant Robertson said 2:07PM on 2-20-2007
Similar idea but, Pipes is way more flexible that that AIM buddy notifications system.
Reply
Carl said 5:00PM on 2-20-2007
Or, you could just use google reader, organize feeds into folders/tags, and click on the folder and walla...combined feed.
Reply
e-tat said 4:38PM on 2-20-2007
Pipes looks like fun - but if all that work is just so you can combine a few incoming feeds, then RSS Owl does that with a single click. http://www.rssowl.org/
On the other hand, if you are spewing a load of feeds, then the pipes method looks pretty simple.
Reply
Grant Robertson said 4:57PM on 2-20-2007
Pipes is quite a bit of fun. My main goal writing this tutorial was to give a foundation from which to jump off when writing more in depth examples of how to use Pipes. We'll be doing more cool stuff with Yahoo!'s new baby, soon!
Reply
hubs said 6:08PM on 2-20-2007
a couple of things i don't like about pipes: no links. pipes strips the link from your entries. this means any links referenced in your text will be taken out in you pipes feed. also pipes only aggregates excerpts from your feeds. i believe anything above 200 words is eliminated from your aggregated pipe feed.
other options i found that are better:
xfruits
feeddigest
feedjumbler
however, none of these options allow you to show the aggregated feed on your own website like lifestreams (www.artifacting.com/blog/life) do.
Reply
Dave Chartier said 6:00AM on 2-21-2007
Pipes is capable of a *lot* more than simply combining feeds. Some of Yahoo's example feeds demonstrate its power best: one of them looks at lead stories from New York Times.com, then filters Flickr for pictures that relate to, or are about, those lead stories from the NYT. So it isn't just combining two feeds - it's actually performing quite a bit of logic to filter and combine content from two sources into a new RSS feed.
Hot.
Reply
Peter Cooper said 3:51PM on 2-23-2007
hubs: FeedDigest lets you put together things like that artifacting page you linked to. The templating is totally dynamic. In fact, I made something like that for myself combining my Twitter, del.icio.us, and blog feeds into a single "life log" :)
Reply