Originally
from Public on NewsGator Online

by Robert Scoble


reBlogged

by Hashbrown

on Jan 10, 2007, 10:21PM

Originally by Robert Scoble from Public on NewsGator Online on January 10, 2007, 5:21pm

By Brady Forrest

Looking for a present a couple of days late? Think about getting the computer-savvy muggle in your life the new Lifehacker book, Lifehacker: 88 Tech Tricks to Turbocharge Your Day. I gave a copy to someone for the holidays. She uses Gmail. She can install software and defragment her hard drive. She’s a whiz with Excel, Word, and Publisher, but she needs help occasionally and doesn’t pay attention to all of the new things that you can do with a computer. She’s too busy working to take the time to find something that might be worthwhile.

That’s where Gina Trapani’s book comes in. It lists 88 useful, well-described tips for getting the most out of a computer and the Internet. It includes great bits like how to join Flickr, use Del.icio.us, back-up your computer, and install a local copy of Wikipedia. She is going to learn things from this book that she would have never known to ask me and then have the instructions to do it herself. It also had a ton of great tips that I am going to use for myself (I got a copy for myself). The Table of Contents and some sample tips have been posted

Now for an additional tip. Throw in a set of book darts. Book darts are small pieces of metal folded over like paperclips with an arrow on one end and they are perfect for research. Just like paperclips or Post-it notes, they can be used mark exact spots on a book page. I like them better because they don’t crease the pages and are reusable. I did exactly this and I was able to mark the chapters of Lifehacker that I thought would be most useful. [Via Scott Berkun]

Here’s a gift idea for me!

Originally
from Public on NewsGator Online

by Brady Forrest


reBlogged

by Hashbrown

on Dec 28, 2006, 8:03AM

Originally by Brady Forrest from Public on NewsGator Online on December 28, 2006, 3:03am

By nat

Kudos to Yahoo!, who today released two pieces of goodness into the commons. The first is their UI library, and the second is their Design Patterns Library. The UI Library is a collection of DHTML/Ajax/Javascript (pick your favourite term) controls and widgets. The Design Patterns Library is “intended to provide
Web designers prescriptive guidance to help solve common design problems
on the Web”. I swear I didn’t know they were going to do this when I wrote “I’d like to see some HIGs for Ajax interfaces” yesterday, but boy am I impressed! Best of all, Yahoo! gets open source licenses: the UI library is BSD and the Design Pattern library is Creative Commons Attribution 2.5.

Originally
from Public on NewsGator Online

by nat


reBlogged

by Hashbrown

on Feb 14, 2006, 3:04AM

Originally by nat from Public on NewsGator Online on February 13, 2006, 10:04pm

Given a search term, it can return code examples or techincal references.

Originally
from Public on NewsGator Online

by Robert Scoble


reBlogged

by Hashbrown

to scoble

on Feb 14, 2007, 5:15PM

Originally by Robert Scoble from Public on NewsGator Online on February 14, 2007, 12:15pm

Originally
from Public on NewsGator Online

by Robert Scoble


reBlogged

by Hashbrown

on Feb 14, 2007, 1:09AM

Originally by Robert Scoble from Public on NewsGator Online on February 13, 2007, 8:09pm

By Brady Forrest

Yahoo! has released a new RSS remixer named Pipes. Right now it’s got some performance issues, but once those are worked out you should go play with the service.
These are some facts and tips about Pipes that I learned while talking with the developers, playing with the app as they were making changes and from reading the documentation.

• To change the name of a Pipe, open it in edit mode and click on the tab. It will become a text field that you can modify and save.
• Pipes will output GeoRSS. They plan to release a map viewer soon. In the meantime you can consume GeoRSS over at Mapufacture.
• You can use Dapper and OpenKapow to scrape sites. They both output RSS feeds that can be consumed by Pipes.
• They have a slick debugger that allows you to step through your modules one at a time and view the data at each step.
• The following browsers are suported: Firefox, Webkit, and IE6. Known Problems: Pipes is known not to work in Opera. Safari has minor issues. IE6 currently displays problems with the For Each module. IE7 has not been extensively tested.
• In addition to RSS, Pipes also output in JSON.
• The Pipes crawler respects the Robot Exclusion Standard. It has the User-agent: Pipes-Crawler Yahoo Pipes.

This is post is the fourth in a series on the Pipes product. Here are the other posts on Radar:

• Pipes and Filters for the Internet
• Yahoo! Pipes: The Modules For Building Pipes
• Yahoo! Pipes: Deconstructing a Pipe

Update – Daniel Raffel of the Yahoo! TechDev Team, the creators of Pipes, wrote to me:

I just saw your latest post on radar about pipes and wanted to point out
that the correct user agent to block is “Yahoo Pipes” NOT “Pipes-Crawler”.

Even more details on that are listed here:
http://pipes.yahoo.com/docs/troubleshooting

Another cool feature I thought I’d mention is that you can search for feeds
in the feed finder in the top right corner, I’ve attached a pic. You can
drag the results to the canvas.

Finally, we have some “easter eggs” for folks to discover but they are
hidden as usual, and, well, we can’t spoil the fun.

Pipes looks cool

Originally
from Public on NewsGator Online

by Brady Forrest


reBlogged

by Hashbrown

on Feb 9, 2007, 3:10PM

Originally by Brady Forrest from Public on NewsGator Online on February 9, 2007, 10:10am

By Brady Forrest

Yahoo! Pipes was released today with the goal of allowing people to easily mix, match, filter, sort and merge data sources into RSS feeds. These resulting RSS feeds are called Pipes and they allow you to do things like find all of the parks in your city or convert the news to Flickr photos. The product allows you to browse pipes, search for pipes, share pipes, or clone somebody else’s pipe. Tim wrote the launch piece, in this piece I am going to delve into the tools that they have made available to construct these pipes.

Pipes  are created in an editor on the website and are made up of modules. These modules are the tools that enable the mixing, sorting and merging of different data sources. The data sources are RSS feeds. Once created a Pipe is hosted at Yahoo! and can be accessed by any RSS reader via it’s unique URL.

The Pipes editor/creator is a really amazing piece of software. You are presented with a white, graphpaper-esque canvas to build your pipe on. A toolbox with a slew of potential modules is on the left, tabs for working with multiples pipes are across the top, and a debugger on the bottom. To build the Pipe you drag modules onto the canvas, enter the relevant data and connect them together via “wires”.

A good example Pipe is “Apartment Near Something“.  It allows you to input what you would like to be near (for example: “parks”), what city (for example: “Palo Alto, CA”), and how far (for example: 2 miles). It outputs GeoRSS of available apartments in a Palo Alto, CA that are near parks.

Click thru for a breakdown of the available modules that are used to create Pipes like this.

The modules for building Pipes are broken into these categories:

Data Sources: This is where you specify your RSS-formatted data inputs. Using the Fetch module you can pull in RSS or Atom feed that Yahoo! servers can access via HTTP. They have created wrappers for several services that did not expose their data as RSS. The included services are Yahoo! Local, Yahoo! Search, Flickr, and Google Base (complete with a field for your API key).

User Inputs: This where someone can enter input values before running a pipe. Y! Pipes provides five different types for easy structuring of data: Numbers, URL, Location, Text, and Date.

Operators: This is where the Unix of the system really comes out. You have the following commands at your disposal:

• For Each: Annotate – Annotates each item in the feed with the results of executing a nested module or pipe.
• For Each: Replace – Replaces each item in the feed with the results of executing a nested module or pipe. (Example Pipe)
• Count – Counts the number of items in the feed (Example Pipe)
• Filter – Blocks or permits items that match the defined rules.
• Truncate – Limits the number of items that passes through it.
• Sort – You can sort items in a feed.
• Babelfish – Translates to and from select languages
• Location Extractor – Detects location entities in text and geocodes them. This is using technology from their WhereOnEarth acquisition.
• Union – Merges two feeds
• Unique – Filters out duplicates

URL – This category only contains the URL Builder Module. With this module you can construct custom urls that pass along data from the User Input module or other module. For example, the URL http://sfbay.craigslist.org/search/aap?query=Palo+Alto&format=rss is constructed by this URL Builder Module and this Location User Input Module.

Palo Alto is the default, but when running this Pipe you could put in any city you wished. To fetch a feed with this module you must connect it to a Fetch Module.

String: This category only contains the module String Concatenate which joins multiple item attributes into a single string.

Date: This category only contains the module Date Formatter which formats a date-time value piped in from another module as a string.

My Pipes: This is where we see some Unix resurfacing in the design. Your own pipes and other people’s pipes can be called as data sources or as functions in your own pipe. Very slick and powerful.

In my next post I am going to deconstruct the Near My Apartment Pipe.

Upate: Yahoo! Pipes now has a listing of all the modules in the help section; it wasn’t up when I was writing this.

This is post is the third in a series on the Pipes product. Here are the other posts on Radar:

• Pipes and Filters for the Internet
• Yahoo! Pipes: Some Tips
• Yahoo! Pipes: Deconstructing a Pipe

Originally
from Public on NewsGator Online

by Brady Forrest


reBlogged

by Hashbrown

on Feb 9, 2007, 7:17AM

Originally by Brady Forrest from Public on NewsGator Online on February 9, 2007, 2:17am

By Brady Forrest

Yahoo! has just released a site for hosting and mixing RSS feeds; read Tim’s launch post for details. In my previous post Yahoo! Pipes: The Modules For Building Pipes I explained the tools available for building a pipe and mentioned the Apartment Near Something pipe. It allows you to input what you would like to be near (for example: “parks”), what city (for example: “Palo Alto, CA”), and how far (for example: 2 miles). It outputs GeoRSS of available apartments in a Palo Alto, CA that are near parks. In this post I am going to step through this pipe. If you want to follow along you can check the debugger in the editor to see the output at each step.

1. You are prompted to fill in 3 fields: your location (palo alto), what you would like to be near (parks), and how far away they can be (2 miles).
2. The Location User Input module takes the location field and passes it to the URL Builder. It constructs the following URL: http://sfbay.craigslist.org/search/aap?query=Palo+Alto&format=rss
3. The URL is passed to the Fetch Module.
   
4. The resulting RSS feed is passed to the Location Extractor Module. The entries that have text relating to location get geotagged.
   
5. The updated feed get passed to a Filter Module. Only locations that have been extracted with a confidence of 80 or higher can pass through.
6. The filtered feed is passed to a For Each: Annotate Module; it has a nested Yahoo! Local Search Data Source Module. The Local module accepts “parks” as an input from the Text User Input Module.
   
7. Then the each item of the feed is checked to see how close it is to area parks.
8. The updated feed is passed through another Filter Module. This Filter Module takes input from a Number User Input Module to filter out rental properties that aren’t within 2 miles of a park.
   
9. The Pipe is complete and can be consumed as GeoRSS
   

This was a relatively simple Pipe that was created by the Pipes team. It works well as an example because it pulls in most of the common features.
I want to thank Pasha Sadri, the principal software engineer of Pipes, who answered my questions and explained some concepts along the way.

Originally
from Public on NewsGator Online

by Brady Forrest


reBlogged

by Hashbrown

on Feb 8, 2007, 8:17AM

Originally by Brady Forrest from Public on NewsGator Online on February 8, 2007, 3:17am

By Tim O’Reilly

Yahoo!’s new Pipes service is a milestone in the history of the internet. It’s a service that generalizes the idea of the mashup, providing a drag and drop editor that allows you to connect internet data sources, process them, and redirect the output. Yahoo! describes it as “an interactive feed aggregator and manipulator” that allows you to “create feeds that are more powerful, useful and relevant.” While it’s still a bit rough around the edges, it has enormous promise in turning the web into a programmable environment for everyone.

Before I get into the details of what it is and how it works, I want to give a little background on why I’m so excited. This is something I’ve been waiting nearly ten years for.

Back in the summer of 1997, at our first Perl conference, Jon Udell (who is among the most prescient of technology visionaries and the prototype for my concept of the alpha geek) gave a talk that electrified me. Jon expressed a vision of web sites as data sources that could be re-used, and of a new programming paradigm that took the whole internet as its platform. This was well before web services were au courant. We don’t have a record of that talk, but a few years later, in a keynote at the 8th International Python conference, he said much the same thing:

To a remarkable degree, today’s Web already is a vast collection of network services. So far, these services are mainly browser-oriented. My browser “calls” a service on Yahoo to receive a page of a directory. Or it “calls” a service on AltaVista to receive a page of search results.

 

One of the nicest things about the Web, however, is that browsers aren’t the only things that can call on the services offered by websites. Programs written in any URL-aware language — including Python, Perl, JavaScript, and Java — can “call” these Web services too. To these programs, the Web looks like a library of callable components. What’s more, it’s very easy to build new Web services out of these existing components, by combining them in novel ways. I think of this as the Web’s analog to the UNIX pipeline.

I picked up Jon’s theme in my own keynote at JavaOne that year, a talk entitled The Network Really is the Computer:

I want to talk about the implications for that marvelous aspect of the fundamental UNIX design: the pipe, and its ability to connect small independent programs so that they could collectively perform functions beyond the capability of any of them alone. What is the equivalent of the pipe in the age of the web? …This is one of the REALLY BIG IDEAS that is going to shape the next five or ten years of computing.

Now, many of you may think that mashups are already the equivalent of pipes. They certainly satisfy many of the requirements that Jon and I were talking about back in 2000. They allow developers to use two websites in a way that their creators didn’t quite intend, which extends them and makes them more useful. But mashups have generally been limited in their scope, pairwise combinations with their output typically being simply another web site. That is, the pipes and filter mechanism had not been generalized.

But perhaps more significantly, to develop a mashup, you already needed to be a programmer. Yahoo! Pipes is a first step towards changing all that, creating a programmable web for everyone.

Using the Pipes editor, you can fetch any data source via its RSS, Atom or other XML feed, extract the data you want, combine it with data from another source, apply various built-in filters (sort, unique (with the “ue” this time:-), count, truncate, union, join, as well as user-defined filters), and apply simple programming tools like for loops. In short, it’s a good start on the Unix shell for mashups. It can extract dates and locations and what it considers to be “text entities.” You can solicit user input and build URL lines to submit to sites. The drag and drop editor lets you view and construct your pipeline, inspecting the data at each step in the process. And of course, you can view and copy any existing pipes, just like you could with shell scripts and later, web pages.

Now, while I say Pipes opens up mashup programming to the non-programmer, it’s not entirely for the faint of heart. At minimum, you need to be able to look at a URL line and parse out the parameters (so, for example, you can use Pipes’ “URL builder” module to construct input to a site’s query function), understand variables and loops, and so on. But you don’t really need to know these things to get started.

Pipes can simply be used as a kind of “power browser.” (something Dale Dougherty has been looking for even longer than I’ve been looking for pipes and filters for the web). For example, you can build a custom mashup to search for traffic along your own routes every morning, or a news aggregator that searches multiple sites for subjects you care about. All you have to do is start with one of the existing modules. (And presumably, once pipes is opened to the public tonight, there will be many more, as anyone can publish their own modules.)

Brady Forrest is writing a separate post to dig more deeply into the how-to side. But to get the concept across, let’s look at Aggregated News Alerts, a pipe that aggregates news alerts from bloglines, findory, Google News, Microsoft Live News, and Yahoo! News.

A preview of the pipe’s output is shown below, after I’ve used the text input field to search for Linux:

But I don’t just get to look at this output on the pipes web site. I can pipe it further, as an RSS feed in its own right, I can send it to my feed aggregator of choice, and I can even get results by email or SMS.

OK. That’s nice. But what’s nicer is that even if I’m not much of a programmer, I can start to copy and paste to modify this pipe even further. I start by cloning the pipe. Now I have my own copy to play with. I can start by subtracting some feeds, and adding some others. For example, if I like to search for open source topics, I might want to subtract some of the general news sources and instead point to news sources like slashdot and the O’Reilly Network.

It’s not quite as easy as drag and drop. I have to understand the query syntax of the sites I want to search, and modify the URL-builder modules to use that syntax rather than the syntax of the sites I’m replacing. But it’s relatively easy once you play around a bit.

What’s really lovely about this is that, like the Unix shell, Pipes provides a gradual introduction to web programming. You start out by modifying someone else’s pipe just a bit, then branch out into something more adventurous.

As I wrote in Unix Power Tools back in 1993:

It has been said that Unix is not an operating system as much as it is a way of thinking. In The UNIX Programming Environment, Kernighan and Pike write that at the heart of the Unix philosophy “is the idea that the power of a system comes more from the relationships among programs than from the programs themselves.”

 

Most of the nongraphical utility programs that have run under Unix since the beginning, some 30 years ago, share the same user interface. It’s a minimal interface, to be sure — but one that allows programs to be strung together in pipelines to do jobs that no single program could do alone.

Most operating systems — including modern Unix and Linux systems — have graphical interfaces that are powerful and a pleasure to use. But none of them are so powerful or exciting to use as classic Unix pipes and filters, and the programming power of the shell.

A new user starts by stringing together simple pipelines and, when they get long enough, saving them for later execution in a file (Section 1.8), alias (Section 29.2), or function (Section 29.11). Gradually, if the user has the right temperament, he gets the idea that the computer can do more of the boring part of many jobs. Perhaps he starts out with a for loop (Section 28.9) to apply the same editing script to a series of files. Conditions and cases soon follow and before long, he finds himself programming.

On most systems, you need to learn consciously how to program. You must take up the study of one or more programming languages and expend a fair amount of concentrated effort before you can do anything productive. Unix, on the other hand, teaches programming imperceptibly — it is a slow but steady extension of the work you do simply by interacting with the computer.

Before long, you can step outside the bounds of the tools that have already been provided by the designers of the system and solve problems that don’t quite fit the mold. This is sometimes called hacking; in other contexts, it is called “engineering.” In essence, it is the ability to build a tool when the right one is not already on hand.

I think that’s a pretty good description of Pipes as well. It democratizes web programming, making it easier for people to have more control over the internet information services they consume, and providing a general-purpose platform for interacting with sites that is more powerful than the browser or feed-reader alone, but without requiring full programming skills.

Pipes still has a ways to go in the ease of use department. Parsing and filtering a stream of xml isn’t as easy as parsing Unix’s ascii stdout. And the user interface of the editor itself needs a lot of work to make it easier to use. But it’s a great start. Kudos to Pasha Sadri and the rest of the Pipes team!

Originally
from Public on NewsGator Online

by Tim O'Reilly


reBlogged

by Hashbrown

on Feb 8, 2007, 4:16AM

Originally by Tim O’Reilly from Public on NewsGator Online on February 7, 2007, 11:16pm