I had the pleasure to interview Jeff Lindsay who is passionate about doing "crazy" stuff and share it with the world. His invention, "webhooks" are gaining more and more attention in the era of the open web. This exciting technology will facilitate interconnections between individuals and organizations.

Roberto: What is a webhook?

Jeff: A webhook is a sort of extension point for web applications. In general, webhooks can be described as an architectural pattern for web developers to provide user-defined callbacks over HTTP in their applications. These callbacks let users do more with the web applications they use. Depending on what the developer exposes with webhooks, users can do a number of things, like set up custom notifications just the way they want, integrate the app with other apps, or even change the behavior of the app.The idea is based on a pattern in software engineering called hooking.

This is where a software library allows the programmer using it to modify its behavior by giving it a pointer to a function known as a callback. This callback is used inside the library as if it were a function inside the library, only it was written by the programmer using the library. In this way, hooking allows programmers to modify the behavior of a library without modifying the code of the library. Webhooks present the same pattern, only at the application level and over the web.

It proposes that web applications allow users to register callbacks in the form of URLs that the application invokes using standard HTTP POST, passing along relevant data. The magic is that a URL can point to a CGI script that can do ... pretty much anything. So this would be taken advantage of by advanced users more often than not, but being accessible by any user has some interesting long-term implications.

R: What is a webhook for civilization?

J: Hah! So this gets into those long-term implications. If you get the core idea of webhooks, with a bit of imagination you can see how some really neat emergent things can happen. First, it introduces a more event-driven web ecosystem. When web APIs started getting popular, the web turned into a programming platform. You didn't have to build applications from scratch all the time because you could use existing applications like libraries. However, there came all these limitations because there was no great way for the application as a library to talk back to you. Because webhooks are about having applications make requests to a URL you specify, it suddenly means they can talk back to you without you having to poll them all the time.

That in itself is a great benefit, but this leads to the web as a programming platform that no longer needs to a have single CGI script making things happen. Event-driven programming is where processes running in parallel can trigger events that will affect others or start executing new ones. It's a great programming paradigm that's been around and more recently has become popular with Javascript programmers for frontend programming. Webhooks creates the infrastructure for this on the backend at the web ecosystem level... where any web app can listen to the events of any other web app.

The more interesting long-term implication is in the fact that they're accessible to the user. And I mean this in the same way that most web APIs are accessible to the user. In most cases, any user, whether a developer or not, can get access to the web API for the web applications they use. If they're so inclined, they can use it from their own programs. If they're not a programmer, it's likely they'll use tools written by programmers that do take advantage of the API. But in a strange way, webhooks make doing cool programmatic things with applications much easier than working with, say, a REST API.

I think this is probably from a couple of factors. First, your hook scripts are triggered by actual events in near real-time, which has a sort of instant gratification and coolness to it. Second, the data just comes to you ... as in, you don't have to look up API docs just to know what to ask for. Finally, getting code on the web to use for these hook scripts has never been easier. I don't just mean things like Google App Engine, or the nearly free PHP hosting out there, but Scriptlets.org lets you write code right there on the homepage, hit a button and get a URL to run it. Over time, with the help of some standards and tools built by the community on top of the simple infrastructure of webhooks, I think it will be very easy for regular users, not just advanced users, to start extending and customizing the applications they use on the web.

R: Got it. Ok, so there are three basic functions that we can use, including push, pipes and plug-ins. What can we do with each of those?

J: Right. So, so those are three high level categories of use-cases I've come up with for webhooks. Push is about real-time notifications from web apps. Pipes are about the integration of web apps. And plugins are about creating extensions or modifying web apps.

R: Interesting, so that means that I can use webhooks to get notifications from your server in real-time, not by polling. RSS feeds will notify me, but are limited to how often they're polled. Is that right?

J: Yes, this is a big draw for webhooks. People are talking about the "real-time web" and because webhooks are intended to be real-time, they are a great vehicle for anything related to push or real-time on the web. Notifications seem to be the big draw ... as in, tell me when something new has been posted, or when something has changed. But with your code on the receiving end of the notification, you can decide exactly how you get notified. Forexample, tell me about changes over Twitter, not email. In fact, no, use this other hook script that uses cloud telephony to call my cell phone and use text-to-speech to tell me.

Real-time notifications, exactly how you want them.

R: You mentioned one can use webhooks to integrate web apps. How is that?

J: Yes, that's the "pipes" use-case. Conceptually it's supposed to be similar to command line piping. The idea is that with webhooks, when some event happens in one app, you can take that data, reformat it a bit, and pass it into another app using its web API. This means you can chain applications together, potentially having one event cascade into many in a system of integrated apps glued together by code triggered by webhooks.

This also suggests a way to have data synchronized across web applications. It's very common now to have data duplication across the various applications you use. Whether you have photos spread across Flickr, Facebook and MySpace, or todos spread across Basecamp, SlimTimer and Remember the Milk, or status updates across Facebook and Twitter ... having webhooks for various events related to those things in all those apps allows the user to write code or use a middleman service that glues these all together. So if you finish a todo in Remember the Milk, it's finished in Basecamp. If you remove a photo in Facebook, it gets removed in Flickr. Post a status update in Facebook that has certain keywords, it gets posted to Twitter.

R: This is really interesting. Can you mention two a couple more examples where this technology is being used?

J: Ah, so far I've been giving hypothetical examples. This is because a lot of the really cool stuff you can do with webhooks requires more people to implement them. Right now, there are a fairly large number of examples, but they're limited to niche uses and are mostly intended for developers. A popular place webhooks have been around is in ecommerce solutions. PayPal has a feature called InstandInstant Payment Notification. This is a webhook (although it's been around much longer than the term webhooks). They let you put in a URL that they POST to whenever a transaction happens, a payment fails, a subscription is created or canceled, and a number of other events. Most competitors like Google Checkout and Amazon Payments have the exact same feature because it's terribly useful for integrating their system with the rest of yours.

Another popular example of webhooks can be found in software project hosting services like GitHub, Google Code, and Beanstalk. They have a webhook for when code is committed to the repository. This lets you do things like run automated tests or try to build the project whenever new code is added to the repository. You can also do simple things, like notify your company IRC channel that new code as been submitted. But there are lots of examples I've been finding (and encouraging), with all sorts of different use-cases. But those two are probably the most popular.

R: There is just one thing left that I would like you to talk more about with webhooks, and that is plugins. As far as I understand webhooks plugins make the web app world extensible. Is that right?

J: Yes. Depending on how you implement webhooks in your application, you can use them to allow your users to write plugins or extend your application in ways you never thought of.

R: So that means that I can extend apps to do whatever I want? Can you explain that more.

J: Well, you're limited to what kinds of things are made extendable by the developer. It's very similar to the way plugin APIs are designed in desktop or open source software like Firefox. Firefox is an example that lets you do almost anything from a plugin, but there are probably better examples where you are more limited in what you can extend in the application.

Perhaps Google Wave, which was recently announced, is a good example. In fact, it uses webhooks for its extension API. One type of Google Wave extension you can make are called Robots. This lets you program your own "robot" that could do anything a real collaborator could do in Google Wave. However, for example, you can't change all the text in Google Wave to pig-latin, and you can't change user settings, because the API only lets you do things a real collaborator could do.

So it's up to the developer to decide what they'll let users extend, but the hope is as much as possible. :)

R: This is really interesting and seems to mean it will revolutionize how the web interacts. More potential for those that who want to empower their users. Right?

J: Yes! It's all about empowering users to do more and do what they want. A friend of mine compared having webhooks to jail-breaking an iPhone. Apple sells you the iPhone and they let you install applications and change settings, but if you jail-break the iPhone, the possibilities of what you can do become much greater.

R: I know that you have worked before with some folks from Hermosillo, MX on encouraging creativity and spreading the culture of innovation in technology. Can you tell me more about this?

J: Oh! Yes, another one of my side projects. In 2005, me and a friend, David Weekly, started an event here in the Silicon Valley called SuperHappyDevHouse. The idea was to just get a bunch of brilliant people together in a social, party-like atmosphere juxtaposed with the idea that you could work on your technology projects. It attracts mostly software developers, but anybody interested in technology is welcome. Some describe it as a microcosm of Silicon Valley culture. We hold them about every six weeks, are up to event number 33, and have had up to 400 people attend at a single event!

Not long ago, myself and a couple of others involved in organizing SuperHappyDevHouse were invited down to Hermosillo in Sonora, Mexico to help start an SHDH event there. A, and it was great! I'm not sure a lot of people would think of Mexico as having many big tech scenes, and perhaps there weren't... but it seems like this event has really brought out a lot of excitement and built a lot of community around technology down there.

And you know Nearsoft has also played a big part in this. It was you guys that paid for us to go down there! I believe since then there have been about 3 three or four other recurring events that have sprung up around Mexico based on the SuperHappyDevHouse model.

It's really quite amazing to see as much passion for ideas and building new things down there as there is here in the Valley. And Mexican culture is so welcoming that I can't help but to go back and see how some of these events and communities have evolved. And eat awesome food!

R: Well, Jeff it was an honor to talk to you again and get more insight about this exciting technology called webhooks.

J: Thanks! If you want to find more resources on what is webhooks and more technical information go to:

• http://webhooks.org/ 
• http://www.slideshare.net/progrium/web-hooks-google-tech-talk
• http://vimeo.com/4537957