Mashup with Yahoo Pipes

Following my previous post on Mashup, I will be documenting the steps to create a mashup using Yahoo Pipes. For those who never heard about Pipes, it is “a powerful composition tool to aggregate, manipulate, and mashup content from around the web.”

Yahoo provides the platform for anyone to create own pipes, all you need is Yahoo account and you can start piping. In this tutorial I will combine data from USGS and Yahoo Map to show recent earthquake around the world.

Step 1: Login to Yahoo Pipes

Step 2: On the top navigation bar. Click on the “Create a pipe” button/link

Yahoo Pipes Navigation
Yahoo Pipes Navigation

Step 3: Now you will see the pipes workspace. It looks like any IDE. On the left hand side, you will all the modules available and on the right hand side it is the canvas for development. All you need to do is drag -n-drop the require component and wire them together.

Yahoo Pipes IDE
Yahoo Pipes IDE

Step 4: Drag-n-drop the “Fecth Feed” module and enter the following URL: http://earthquake.usgs.gov/earthquakes/catalogs/shakerss.xml. Ignore the output pipe for now.

Add Fetch Feed Module
Add Fetch Feed Module

Step 5: If you look into the RSS content, you will find the geo-location for each of the earthquake data. In this step, I will use “Location Extractor” module to extract the geo-location. It is located in the “Operators” group.

Add Location Extractor
Add Location Extractor

Step 6: In Yahoo Pipes, once location extracted you can out the information for the pipes to automatically link it with Yahoo Map. Feature or limitation? up to you to decide :).  With all the necessary modules in place, all you have to do is wire then together.

Wire all modules
Wire all modules

Step 7: You must save the newly created pipes before you are allowed to run it. Let’s name it “Earthquake Map by USGS”

Save & Run Pipe
Save & Run Pipe

Step 8: You will see the output from your pipe in a new screen. Yeah!! you’ve done it.

Output from Yahoo Pipes
Output from Yahoo Pipes

Anyway, this is just a simple pipe. You can see what other has created, some are simple, some are complex… some are very complex. It is really up to you and your creativity to combine various module and create new service.

So, you may ask “what can I do with the pipe?”

  1. You can publish it. It will be publicly available within the pipes communities.
  2. You can include it in your own site, by generating the badge. (see the script below below).
  3. You can add it to your “My Yahoo” or Google

Add this to your site/page to try it out.

<script src=”http://l.yimg.com/a/i/us/pps/mapbadge_1.1.js”&gt;{“pipe_id”:”48a0ce7eeece808ea6cece6bd70abc64″,”_btype”:”map”}</script>

Reference:

  • Yahoo Pipes
  • USGS – Earthquake Data

So, happy piping…. if you have created something in Yahoo Pipes, share with me via the comments box 🙂

Advertisements

Mashup

Did a bit of research on mashup couple of months ago and would like to share a short write up about it.

The term mashup originated from composing songs or musics. Wikipedia defines it as “a song or composition created by blending two or more songs, usually by overlaying the vocal track of one song seamlessly over the music track of another”

The mashup I will be talking about here is about IT applications and Wikipedia defines it as “a web page or application that combines data or functionality from two or more external sources to create a new service”. The data or functionality are typically from other sites and accessible via APIs, XML feeds, screen-scraping, mashup server, etc. The diagram below should help to illustrate the definition.

Mashup
Mashup

For a long time we have seen web portal, starting with Yahoo! and spreading to enterprise portal. So, how is mashup different from portal? I did a screen capture from wikipedia to illustrate the difference. (http://en.wikipedia.org/wiki/Mashup_%28web_application_hybrid%29). Click the figure below to see more…

Mashup vs Portal
Mashup vs Portal

Mashup itself can be implemented at any of the 3 layers, namely

  • Presentation
  • Business Logic
  • Data
Mashup Technology Domain
Mashup Technology Domain

Mashup at presentation layer is the most common one we’ve seen. Many sites combining Google Maps and own data to provide new services. Those services will then rendered as gadget (Google and Open Social Standard), widget (IBM term), Badges (Yahoo! term).

Mashup at Presentation Layer
Mashup at Presentation Layer

Mashup at logic/process layer and data layer typically done with the help of mashup server or application server. The definition of mashup at business logic layer is a little unclear to me because it looks like any server-side integration (using application server or service bus).

Data mashup can be implemented at client and server-side. Like business logic, server-side data mashup require mashup server or application server. While client-side can be implemented using on-demand Javscript.

Each of the components from the “smallest”, such as APIs, XML feeds can be assembled to create a new service or application by using a mashup platform or on-demand JavaScript. The diagram below sh0ws how each component connected.

Mashup Stack
Mashup Stack

Reference site

  • Yahoo Pipes – Mashups Development Tool
  • Google Sites – Mashups Platform
  • IBM Lotus Mashups – Mashups Platform. See the video tutorial below. It is still at early stage.  When I tried it out, it does not support IE 8 and FF 3.5.x. Future update should support them.
  • Mashups Directory
  • Kent County Council (UK)
  • Boeing Mashup – See the video below. It shows how Boeing uses IBM Lotus to develop a new services.
  • Kapow Technologies – Known for its screen-scraping product. However it has more.
  • My Twitter Nearby – did a simple mashup combining twitter data and google map to create twitter nearby. If you want to try, just click somewhere in the map and wait for awhile. You will see tweet nearby the area you click.

If you are curious about the IBM Lotus Mashup then watch the following videos:

Do share with me your thought on Mashup and how your organization make use of it.