Abstractioneer Archive

This is an EX-blog!

2008-01-18T16:32:00.000-08:00

If you want to read further, you might want to check out http://abstractioneer.org which is a good bit and has OpenID in it.

Tags: openid, abstractioneer.org

Moving on to abstractioneer.org

2008-01-11T17:51:00.000-08:00

I've decided to take the plunge and move on to publishing at http://abstractioneer.org, powered by Blogger. Since I'm the tech manager for Blogger it seems only fitting, and we've recently been adding a whole host of cool features that make it more and more attractive (OpenID commenting being just the latest).

I also have a semi-new Feedburner blog feed; some people were already subscribed through this feed, so you may notice no disruption in service as I re-point it at abstractioneer.org... just a moment... there! (http://feeds.feedburner.com/aol/SzHO). Feel free to re-subscribe there, if you are in the mood.

Tags: moving on, new blog, abstractioneer

[Abstractioneer] Warumungu Norms, Privacy, Facebook, and Useful Friction

2008-01-11T17:41:00.000-08:00

We could learn something from the Warumungu. Wendy Seltzer's Mukurtu Digital Archiving: digital "restrictions" done right is about DRM, freedom, and controls; I think it's also about privacy. What's private, and what's public, and what's semi-private are culturally determined no less than the Warumungu rules around who is allowed to see what artifacts:

...the Warumungu have a set of protocols around objects and representations of people that restrict access to physical objects and photographs. Only elders may see or authorize viewing of sacred objects; other objects may be restricted by family or gender. Images of the deceased shouldnât be viewed, and photographs are often physically effaced. When the Warumungu archive objects or images, they want to implement the same sort of restrictions.

With an interesting twist:

People can also print images or burn CDs and thus allow the images to circulate more widely to others who live on outstations or in other areas. In fact, one of the top priorities in Mukurtuâs development was that it needed to allow people to take things with them, printing and burning were necessary to ensure circulation of the materials.

What, then, prevents people from violating these norms?

Because the Murkurtu protocol-restrictions support community norms, rather than oppose them, the system can trust its users to take objects with them. If a member of the community chooses to show a picture to someone the machine would not have, his or her interpretation prevails â the machine doesnât presume to capture or trump the nuance of the social protocol.

People, relationships, and norms are fuzzy and messy, so maybe it's reasonable that a system to deal with them is fuzzy and messy too. What Murkurtu does is put enough useful friction in the way of disclosure to give community norms a chance to operate. You can't email an image out to a mailing list, but you can print it and show it to a reasonably small number of people at a time. The point is not to control distribution perfectly, but to give human-scale trust mechanisms a chance to operate correctly.

Who owns the data?

L'Affaire Scoble raised the question, who owns relationship data? Dare Obasanjo argues that his contact data is his, not Robert's. And he wants Facebook to enforce this.

I'd argue that we should un-ask the ownership question. As long as we're talking about ownership, we're heading down the road towards DRM that has worked out so well for the music business. I'd like to talk about community norms, and what kind of useful friction we should be thinking about in the pure digital realm to give community norms a chance to operate. Reputation and portable identity is part of this, as are things like limited access (E.g., OAuth), rate limits, soft constraints, and user centric norm enforcement. (What would happen if the people on Robert's friends list were simply informed, in real time, that he was copying their data for an unknown purpose?)

(Nick Carr has a great post on this subject as well.)

--
Posted By John Panzer to Abstractioneer at 1/11/2008 02:23:00 PM

Shindig!

2007-12-12T08:40:00.000-08:00

We've just made our first commit to the Apache Shindig project! This first version provides the basic substrate for running gadgets, which is useful by itself and is a prerequisite for running OpenSocial gadgets.

Tags: shindig, opensocial, apache

Singularity to Launch from Adult Chat Room

2007-12-09T14:12:00.000-08:00

You heard it here first. Based on this story about a chatbot passing the Turing Test, clearly the Vingean Singularity is just around the corner. CyberLover will acquire self-awareness soon after the Russian identity thieves deploy it on existing Russian botnets. Transcendence, and a technological singularity, is just a short hop and a jump from that point. Have fun chatting!

Tags: singularity, chatbot, social engineering, vinge

OpenID 2.0 Released!

2007-12-05T22:57:00.000-08:00

Announced at IIW2007b today (and blog post by David Recordon). Congratulations to all! It's actually two specifications, OpenID Authentication 2.0 and OpenID Attribute Exchange 1.0. Attribute exchange in particular allows for some very interesting integration possibilities.

Tags: openid, iiw2007b, attribute exchange, openid20

OAuth 1.0 Core Released!

2007-12-04T15:39:00.000-08:00

December 4, 2007 – The OAuth Working Group is pleased toannounce publication of the OAuth Core 1.0 Specification. OAuth (pronounced"Oh-Auth"), summarized as "your valet key for the web," enables developers ofweb-enabled software to integrate with web services on behalf of a user withoutrequiring the user to share private credentials, such as passwords, betweensites. The specification can be found at http://oauth.net/core/1.0and supporting resources can be found at http://oauth.net.

Tags: iiw2007b, oauth

IIW2007b Updates

2007-12-04T11:11:00.000-08:00

First session set up by Terrell of ClaimID: Open Life Bits, some interesting discussion about how to control one's one data and deal with data about one's self. The distinction is interesting and useful; every transaction that involves a second party potentially generates data about you controlled by that party, but you do want to be able to deal with that data, correct inaccuracies, etc. Notes here.

Next session, Joseph Smarr of Plaxo, OpenID user experience. Good walkthrough of UI issues. Note that with directed identity in OpenID 2.0, can simply ask to log in a user given their service. Notes here. Using an email address is a possibility as well; clicking on a recognizable icon (AIM) to kick of an authentication process is probably the most usable path right now.

Session: OAuth Extensions; notes here.

Session: OAuth + OpenID. Use case: I have an AOL OpenID. I go to Plaxo and am offered to (1) create an account using my AOL OpenID and (2) pull in my AOL addressbook, all in one step.

Proposal: I log in via OpenID and pass in an attribute request asking for an OAuth token giving appropriate access, which lets AOL optimize the permissions page (to one page, or organize all data together). Then get token, and use token to retrieve data.

Tags: iiw2007b, openid, authorization, openlifebits, oauth

Internet Identity Workshop 2007b

2007-11-30T17:17:00.000-08:00

I'll be at IIW next week, talking about Blogger, OpenID, OAuth, OpenSocial, and anything else that seems interesting. I'm anticipating a great event.

Tags: openid, blogger, oauth, opensocial, iiw, iiw2007b

OpenID Commenting for Blogger!

2007-11-30T11:03:00.000-08:00

We've just enabled OpenID signed comments for Blogger in Draft. There are a few rough edges still (which is why you have to enable it for your blog by going to draft.blogger.com), so we're looking for feedback. We're also working on enabling Blogger as an OpenID Provider, meaning that you can use your blog URL to identify yourself on other services.

What's particularly fun about this is that it's been a very collaborative project, bringing together Blogger engineers, 20% time from a couple of non-Blogger engineers, and last but not least some of the fine open source libraries provided by the OpenID community. Thanks all!

Tags: openid

Essential Atom and AtomPub in 30 seconds

2007-11-09T11:24:00.000-08:00

Atom is this: You have a bunch of things, or sometimes just one thing. They always have unique ids, they have timestamps, and tell you who created/is responsible for them. Oh yeah, if you can, please provide a short snippet of text describing each thing.

AtomPub is how to discover, create, delete, and edit those things.

Everything else is optional and/or extensions.

Tags: Atom, AtomPub

OpenSocial Ecosystem

2007-11-02T09:12:00.000-07:00

OpenSocial has been... extensively covered in just about all media over the past few days. The official site is up, and the video from the Campfire 1 announcement as well.

Obviously this is just a first step. We're all trying to build a self-sustaining ecosystem, and right now we're bootstrapping. It's a bit like terraforming: We just launched the equivalent of space ships carrying algae :).

A key next step is making it easy to create social app containers. It's not hard to build a web page that can contain Gadgets, though it could be easier. Adding the social APIs, the personal data stores, social identity, and authentication and authorization makes things a lot more complex. This is the part I'm working on, along with a lot of other people. It's a problem space I've been working in for a while on the side. Now it's time to achieve 'rough consensus and running code.'

Tags: OpenSocial, Google, containers, rough consensus and running code, authentication, authorization, identity, mashups, ecosystem, terraforming, OpenID, OAuth

Blogger #1 "Social Networking" Site Worldwide

2007-10-25T22:50:00.000-07:00

The folks over at Windows Live Spaces just crunched some ComScore worldwide numbers. Their headline was "Windows Live Spaces at a Crossroads", but I think my headline fits their graphs better.

According to them, Blogger did140,000,000 worldwide unique visitors in September, and has been on a tear since June. Nice! And to all those Blogger users, thank you!

Of course, whether Blogger is a "Social Networking" site depends on your definitions; Dare wants to disqualify the front runner. Me? I think 140 million people can speak for themselves.

Tags: Social Networking, Blogger, Windows Live Spaces

Fireblog

2007-10-23T17:15:00.000-07:00

The San Diego Union-Tribune has been posting wildfire-related updates in real time to a site, http://fireblog.signonsandiego.com/, but their servers melted under the load, so they moved over to Blogger yesterday, and have been up and running and helping people out since last night. It was great to be able to tell them that load isn't a problem for us :).

Tags: fireblog, blogger

A Four Year Mission... to Boldly Go Where No Protocol has Gone Before

2007-10-22T22:26:00.000-07:00

Today's message from the IETF:

The Atom Publishing Format and Protocol WG (atompub) in the Application Area has concluded.

...

The AtomPub WG was chartered to work on two items: the syndication format in RFC4287, and the publishing protocol in RFC5023. Implementations of these specs have been shown to work together and interoperate well to support publishing and syndication of text content and media resources.

Since both documents are now Proposed Standards, the WG has completed its charter and therefore closes as a WG. A mailing list will remain open for further discussion.

Congratulations and thanks to the chairs Tim Bray and Paul Hoffman, to the editors of the two WG RFCs (Mark Nottingham, Robert Sayre, Bill de Hora and Joe Gregorio), and to the many contributors and implementors.

Tags: atom, atompub, rfc5023

Widget Summit

2007-10-14T21:42:00.000-07:00

I'm headed to the Widget Summit tomorrow; let me know if you're going on Monday and want to sync up. (Unsure whether I can make Tuesday or not yet.)

Tags: widget, widget summit

The Atom Publishing Protocol is here!

2007-10-08T19:42:00.000-07:00

The Atom Publishing Protocol, aka APP, aka AtomPub, now has another alias: RFC 5023. Congratulations to the WG, and to Joe and Bill for many person-years of effort!

Tags: Atom, AtomPub, RFC5023

OAuth: Your valet key for the Web

2007-09-21T14:23:00.000-07:00

Just published at http://oauth.net/documentation/spec: Draft 1 of the OAuth specification. As my day job allows, I've been contributing to the OAuth working group. We'd love feedback.

What is it?

OAuth is like a valet key for all your web services. A valet key lets you give a valet the ability to park your car, but not the ability to get into the trunk or drive more than 2 miles or redline the RPMs on your high end German automobile. In the same way, an OAuth key lets you give a web agent the ability to check your web mail but NOT the ability to pretend to be you and send mail to everybody in your address book.

Today, basically there are two ways to let a web agent check your mail. You give it your username and password, or it uses a variety of special-purpose proprietary APIs like AuthSub, BBAuth, OpenAuth, Flickr Auth, etc. to send you over to the web mail site and get your permission, then come back. Except that since mostly they don't implement the proprietary APIs, and just demand your username and password. So you sigh and give it to them, and hope they don't rev the engine too hard or spam all your friends. We hope OAuth will change that.

OAuth consolidates all of those existing APIs into a single common standard that everybody can write code to. It explicitly does not standardize the authentication step, meaning that it will work fine with current authentication schemes, Infocard, OpenID, retinal scans, or anything else.

And yes, it will work for AtomPub and other REST services, and I hope it will be the very last authorization protocol your client ever needs to add for those things.

For more information and ongoing updates, go to http://oauth.net/.

Note: I picked up the "valet key" metaphor from Eran's postings. Thanks Eran!

Tags: oauth, delegation, authentication, web services, valet keys

"We have lost control of the apparatus" -- Raganwald

2007-09-16T13:06:00.000-07:00

Yet another great post from Raganwald: Wehave lost control of the apparatus.

Our users are being exposed to applications we don’tcontrol. And it messes things up. You see, the users get exposed toother ways of doing things, ways that are more convenient for users,ways that make them more productive, and they incorrectly think weought to do things that way for them.

I sure hope this part is true:

You would things couldn’t get any worse. But they areworse, muchworse. I’ll just say one word. Google. Those bastards are practicallythe home page of the Internet. Which means, to a close approximation,they are the most popular application in the world.

And what have they taught our users? Full-text search wins. Please,don’t lecture me, we had this discussion way back when we talked aboutfields. Users know how to use Google. If you give them a search pagewith a field for searching the account number and a field for searchingthe SSN and a field for searching the zip code and a field forsearching the phone number, they want to know why they can’t just type4165558734 and find Reg by phone number? (And right after we make thatwork for them, those greedy and ungrateful sods’ll want to type (416)555-8734 and have it work too. Bastards.)

Play

2007-09-14T15:53:00.000-07:00

Just launched today: Blogger Play. Wheee!

Tags: play, blogger, launch

Do you trust your friends with your URLs?

2007-08-16T09:33:00.000-07:00

"Facebook's data feed a data leak?" over at Lawgarithms:

Please correct me if I’m wrong about this; I want to be wrong aboutthis. Or I want to learn that Facebook has already considered and dealtwith the issue and it’s just not readily apparent to me. But I’mthinking that Facebook’s feeds for Status Updates, Notes, and PostedItems must in many instances be at odds with privacy settings thatattempt to limit users’ Facebook activities to “friends only” (or areeven more restrictive).

Denise is both right and wrong. The basic issue is that once you give out a feed URL (which is not guessable) to a friend, they can then give it out to their friends and their friends... ad infinitum. These people can then get your ongoing updates, without you explicitly adding them.

Of course, this requires your friends to breach the trust you placed in them to guard your bits. Notice that even without feeds, your friends can easily copy and paste your bits and send them on manually. It's a simple matter to automate this if a friend really wants to broadcast your private data to whoever they want. So as soon as you open up your data, you are vulnerable to this. To prevent it you'd need working DRM; not a good path to go down.

It would be possible to access control the feeds; there's even a nascent standard (OAuth) for doing this in a secure and standards compliant way. But even this doesn't prevent your friends from copying your bits.

A much simpler approach is to hand out a different URL for each friend. They're still obfuscated of course. You can then block a friend (and anyone they've shared the URL with) from seeing future updates at any time. This is about the best that can be done. Update: This is apparently exactly what Facebook has done. Denise is still concerned that friends could accidentally or purposefully re-share the data, since the feed format makes it easy to do so.

Facebook's messaging could definitely be improved. Suggestions?

Tags: facebook, feeds, security, friends, oauth

RESTful partial updates: PATCH+Ranges

2007-08-07T23:43:00.000-07:00

Over the past couple of months, there's been a lot of discussion aboutthe problem of partial updates in REST-over-HTTP[1][2][3][4][5]. The problemis harder than it appears at first glance. The canonical scenario isthat you've justretrieved a complicated resource, like an address book entry, and youdecide you want to update just one small part, like a phone number. The canonical way to do this is to update yourrepresentation of the resource and then PUT the whole thing back,including all of the parts you didn't change. If you want to avoid thelost update problem,you send back the ETag you got from the GETwith your PUT inside an If-Match: header, so that you know that you'renot overwriting somebody else's change.

This works, but it doesn't scale well to large resources or highupdate rates, where "large" and "high" are relative to your budget forbandwidth and tolerance for latency. It also means that you can'tsimply and safely say "change field X, overwriting whatever is there,but leave everything else as-is".

I've seen the same thought process recapitulated a few times now on howtosolve this problem in a RESTful way. The first thing that springs tomind is to ask if PUT can be used to send just the part you want tochange. This can be made to work but has some major problemsthat make it a poor general choice.

A PUT to a resourcegenerally means "replace", not "update", so it's semanticallysurprising.
In theory it could break write-through caches. (This is probablyequivalent to endangering unicorns.)
It doesn'twork for deleting optional fields or updating flexible lists such asAtomcategories.

The next idea is generally to simply use POST to update the resource. This does work in many cases, but conflicts with the use of POST to adda resource to a collection. That is, if you POST to a collection, areyou trying to add an element to the collection, or perform some otherupdate to the collection's metadata? It's possible disambiguate usingMIMEtypes but it feels fragile. It also doesn't capture the fact that theoperation is retryable; POST in general is not retryable.

A good solution to the partial update problem would be efficient,address the canonical scenarioabove, be applicable to a wide range of cases, not conflict with HTTP,extend basic HTTP as little as possible, deal with optimisticconcurrency control, and deal with the lost update problem. The methodshould be discoverable (clients should be able to tell if a serversupports the method before trying it). It would also be nice if thesolution would let us treat data symmetrically, both getting andputting sub-parts of resources as needed and using the same syntax.

There are three contenders for a general solution pattern:

Expose Parts as Resources. PUT to a sub-resourcerepresents aresources' sub-elements with their own URIs. This is in spirit whatWeb3Sdoes. However, it pushes the complexity elsewhere: Intodiscovering the URIs of sub-elements, and into how ETags work acrosstwo resources that are internally related. Web3S appears to handleonlyhierarchical sub-resources, not slicing or arbitrary selections.

Accept Ranges on PUTs. Ranged PUT leverages andextends theexisting HTTP Content-Range:header to allow a client tospecify a sub-part of a resource, not necessarily just byte ranges buteven things like XPath expressions. Ranges are well understood in thecase of GET but were rejected as problematic for PUT a while back bytheHTTP working group. The biggest concern was that it adds a problematicmust-understand requirement. If a server or intermediary accepts a PUTbut doesn'tunderstand that it's just for a sub-range of the target resource, itcould destroy data. But, thisdoes allow for symmetry in reading andwriting. As an aside, the HTTP spec appears to contradict itselfabout whether range headers are extensible or are restricted to justbyte ranges. This method works fine with ETags; additional methods fordiscovery need to be specified but could be done easily.

Use PATCH. PATCH is a method that's beentalked about for awhilebut is the subject of some controversy. James Snell has revived LisaDusseault's draft PATCH RFC[6] and updated it, and he's looking forcomments on the new version. I think this is a pretty good approachwith a few caveats. The PATCH method may not be supported byintermediaries, but if it fails it does fail safely. It requires a newverb, which is slightly painful. It allows for variety of patchingmethods via MIME types. It's unfortunately asymmetric in that it doesnot address the retrieval ofsub-resources. It works fine with ETags. It's discoverable via HTTPheaders (OPTIONS and Allow: PATCH).

The biggest issue with PATCH is the new verb. It's possible thatintermediaries may fail to support it, or actively block it. This isnot too bad, since PATCH is just an optimization -- if you can't useit, you can fall back to PUT. Or use https, which effectively tunnelsthrough most intermediaries.

On balance, I like PATCH. The controversy over the alternatives seemto justify the new verb. It solves the problem and I'd be happy withit. I would like there to be a couple of default delta formats definedwith the RFC.

The only thing missing is symmetricalretrieval/update. But, there's an interesting coda: PATCH is definedso that Content-Range is must-understand on PATCH[6]:

The server MUST NOT ignore any Content-* (e.g.  Content-Range) 
headers that it does not understand or implement and MUST return 
a 501 (Not Implemented) response in such cases.

So let's say aserver wanted to be symmetric; it could advertise support forXPath-based ranges on bothGET and PATCH. A client would use PATCH with a range to send backexactly the same data structure it retrievedearlier with GET. An example:

GET /abook.xml

Range: xpath=/contacts/contact[name="Joe"]/work_phone

which retrieves the XML:

<contacts><contact><work_phone>650-555-1212</work_phone>
</contact></contacts>

Updating the phone number is very symmetrical with PATCH+Ranges:

PATCH /abook.xml

Content-Range: xpath=/contacts/contact[name="Joe"]/work_phone

<contacts><contact><work_phone>408-555-1591</work_phone>
</contact></contacts>

The nice thing about this is that no new MIME types need to beinvented; the Content-Range header alerts the server that the stuffyou're sending is just a fragment; intermediaries will eitherunderstand this or fail cleanly; and the retrievalsand updates are symmetrical.

[1]http://www.snellspace.com/wp/?p=683
[2]http://www.25hoursaday.com/weblog/2007/06/09/WhyGDataAPPFailsAsAGeneralPurposeEditingProtocolForTheWeb.aspx
[3]http://www.dehora.net/journal/2007/06/app_on_the_web_has_failed_miserably_utterly_and_completely.html
[4]http://tech.groups.yahoo.com/group/rest-discuss/message/8412
[5]http://tech.groups.yahoo.com/group/rest-discuss/message/9118
[6]http://www.ietf.org/internet-drafts/draft-dusseault-http-patch-08.txt

Some thoughts on "Some Thoughts on Open Social Networks"

2007-08-07T14:58:00.000-07:00

Dare Obasanjo:

"Content Hosted on the Site Not Viewable By the General Public and not Indexed by Search Engines: As a user of Facebook, I consider this a feature not a bug."

Dare goes on to make some great points about situations where he's needed to put some access controls in place for some content. I could equally make some points about situations where exposing certain content as globally as possible has opened up new opportunities and been a very positive thing for me. After which, I think we'd both agree that it's important to be able to put users in control.

Dare: "Inability to Export My Content from the Social Network: This is something that geeks complain about ... danah boyd has pointed out in her research that many young users of social networking sites consider their profiles to be ephemeral ... For working professionals, things are a little different since they mayhave created content that has value outside the service (e.g.work-related blog postings related to their field of endeavor) soallowing data export in that context actually does serve a legitimateuser need."

It isn't just a data export problem, it's a reputation preservation problem too. Basically, as soon as you want to keep your reputation (identity), you want to be able to keep your history. It's not a problem for most younger users since they're experimenting with identities anyway. Funny thing, though: Younger users tend to get older. At some point in the not so distant future that legitimate user need is going to be a majority user need.

Dare: "It is clear that a well-thought out API strategy that drives people toyour site while not restricting your users combined with a great userexperience on your website is a winning combination. Unfortunately,it's easier said than done."

+1. Total agreement.

Dare: "Being able to Interact with People from Different Social Networks from Your Preferred Social Network: I'm on Facebook and my fiancée is on MySpace. Wouldn't it be great if we could friend each other and send private messages without both being on the same service? It is likely that there is a lot of unvoiced demand for thisfunctionality but it likely won't happen anytime soon for businessreasons..."

Will there be a viable business model in meeting the demand that Dare identifies, one which is strong enough to disrupt business models dependent on a walled garden? IM is certainly a cautionary tale, but there are some key differences between IM silos and social networking sites. One is that social networking sites are of the Web in a way that IM is not -- specifically they thrive in a cross-dependent ecosystem of widgets, apps, snippets, feeds, and links. It's possible that "cooptition" will be more prevalent than pure competition. And it's quite possible for a social network to do violently antisocial things and drive people away as Friendster did, or simply have a hot competitor steal people away as Facebook is doing. Facebook's very success argues against the idea that there will be a stable detente among competing social network systems.

Tags: socialnetworking, facebook, open api, social software, open social networks, cooptition

Relationship requires identity

2007-08-07T11:47:00.000-07:00

NishantKaushik:

Let's face it, relationship silos are really justextensions of identity silos. The problem of having to create andre-create my relationships as I go from site to site mirrors my problemof having to create and re-create my identity as I go from site tosite. The Facebook Platform might have one of the better IdentityProvider APIs , but all the applications built on it still have to staywithin Facebook itself.

Yup. Which is the primary reason that I've been interested in identity-- it's a fundamental building block for social interactions of allkinds. And think of what could happen if you could use theInternet as your social network as easily as you can use Facebooktoday. As ScottGilbertson at Wired discovered, it's nothard to replicate most of the functionality; it's the people whoare "on" Facebook which makes it compelling.

Tags: identity, silos, openid, facebook, social software, social networks

Google += Joe Gregorio!

2007-08-06T22:25:00.000-07:00

+1!

cat Google Spreadsheets | Venus > my.feed

2007-08-02T10:59:00.000-07:00

Sam Ruby (prompted by Alf Eaton) combines Google Spreadsheets and Venus to let people manage Venus subscription lists (or whatever) using Spreadsheets. The lingua franca is of course CSV-over-HTTP. Like Unix pipes running over Internet, um, pipes.

Note that this requires the data to be publicly readable on the Spreadsheets side, which is fine for this use. A lot more uses would be enabled with a lingua franca for deputizing services to talk securely to each other.

Tags: cool, pipes, mashups, web as platform

Blogger is looking for engineers!

2007-07-27T16:19:00.000-07:00

Interested in working for Google onthe top blogging platform around? We're looking for engineers. Experience or interest in buildingweb-based social applications is a plus but not a requirement. Selfmotivation, ability to get things done, and burning desire to work onnew things are requirements. Want to find out more? Contact me.

AtomPub now a Proposed Standard

2007-07-24T23:38:00.000-07:00

http://www.ietf.org/internet-drafts/draft-ietf-atompub-protocol-17.txt is now an official IETF Proposed Standard. Whee!

Tags: atom, AtomPub, APP

Share your dog's name, lose your identity?

2007-07-24T12:48:00.000-07:00

From the BBS: Web networkers 'at risk of fraud'.

Credit information group Equifax said members of sites such as MySpace, Bebo and Facebook may be putting too many details about themselves online.It said fraudsters could use these details to steal someone's identity and apply for credit and benefits.

So, to protect the credit bureau's business models, we're all supposed to try to hide every mundane details of our lives? The name of my dog is not a secret; if credit bureaus assume it is, they are making a mistake.

Here's the solution: Make the credit bureaus fiscally responsible for identity theft, with penalties for failing to use good security practices.

Tags: identity, fraud, external costs, my dogs name is rover

Open Authorization, Permissions, and Socially Enabled Security

2007-07-19T20:31:00.000-07:00

The session I proposed at Mashup Camp, Open Authentication and Authorization for Mashups, went pretty well (though I should have done more marketing). Unfortunately none of the people on the OAuth group were at Mashup Camp, but perhaps we generated some more interest and use cases for it.

Consider a user navigating web services and granting various levels of permissions to mash-ups; a mash-up might request the right to read someone's location and write to their Twitter stream, for example. The first time this happens, the user would be asked something like this:

The TwiLoc service is asking to do the following on an ongoing basis:
- Read your current location from AIM, and
- Create messages on your behalf in Twitter.
How does this sound?
[ ] No [ ] Yes [ ] Yes, but only for today

The user would also have a way to see what permissions they've granted, how often they've been used (ideally), and be able to revoke them at any time.

Now, of course, users will just click through and say "Yes" most of the time on these. But there's a twist; since you're essentially mapping out a graph of web services, requested operations, granted permissions, usage, and revocations, you start to build up a fairly detailed picture of what services are out there and what precisely they're doing. You also find out what services people trust. Throw out the people who always click "yes" to everything, and you could even start to get some useful data.

You can also combine with social networks. What if you could say, "by default, trust whatever my buddy Pete trusts"? Or, "trust the consensus of my set of friends; only ask me if there's disagreement"? Or more prosaically, "trust what my local IT department says".

Tags: mashup camp, mashupcamp4, openid, oauth, permissions, authorization

At Mashup Camp today and tomorrow

2007-07-18T15:55:00.000-07:00

I'm at Mashup Camp IV today and tomorrow. Ping me if you're around too and want to chat.

Tags: mashup camp, mashupcamp

Every mashup attempts to expand...

2007-07-18T15:43:00.000-07:00

Proposed, half-seriously:

Every mashup attempts to expand until it can do social networking. Those that can't are replaced by those that can.

(With apologies to Zamie Zawinski.)

Tags: Mashups, social networks, social software, Panzers Law

Implications of OpenID, and how it can help with phishing

2007-07-10T09:22:00.000-07:00

:Last month, Simon Willison gave a talk at Google (video, slides) which is a good intro and summary of technical implications of OpenID. He points out a very important point: OpenID does outsource your security to a third party; so does sending a "forgot your password" email to an arbitrary email address. All of the attacks that work against OpenID also work against these emails.

So the implication is that the security policies that you currently have around "forgot your password" are a good starting point for thinking about OpenID security. Specifically phishing vulnerabilities and mitigations are likely to be similar. However, OpenID also changes the ecosystem by introducing a standard that other solutions can build on (such as Verisign's Seat Belt plugin).

OpenID really solves only one small problem -- proving that you own a URL. But by solving this problem in a standard, simple, deployable way, it provides a foundation for other solutions.

It doesn't solve the phishing problem. Some argue that it makes it worse by training users to follow links or forms from untrusted web sites to the form where they enter a password. My take: Relying on user education alone is not a solution. If you can reduce the number of places where a user actually needs to authenticate to something manageable, like say half a dozen per person, then we can leverage technical and social aids much more effectively than we do now. In this sense, OpenID offers opportunities as well as dangers. Of course, this would be true of any phishing solution.

Tags: OpenID, google, talk, security, phishing

Disorder, Delamination, David Weinberger

2007-07-09T10:14:00.000-07:00

David Weinberger's presentation in Disorder: Feature or Bug? at Supernova 2007 was like watching a great rock singer deliver a passionate performance you just know is destined to be a classic. How good was it? The IRC channel went dead. That's the conference equivalent of everybody waving their lighters in the air. Um. Well, you just had to be there. I can't find a video. Anybody have a bootleg?

Anyway. David's now posted a new essay well worth reading, Delamination Now!. Also, well worth acting on. Money quote: "[T]he carriers are playing us like a violin."

Tags: delamination, david weinberger, carriers, net neutrality, net architecture, dumb network, innovation, supernova, disorder

There she blows! (The Moby Dick Theory of Big Companies)

2007-07-08T11:50:00.000-07:00

Having spent some time in the belly of the whale[1], I can testify that the decision making process of a large company is indeed a chaotic system even when seen from the inside. The variables that control decisions are very well hidden.

The Pmarca Guide to Startups, part 5: The Moby Dick theory of big companies

[1] In the same whale as pmarca in fact, though in a somewhat different location along the alimentary tract.

Tags: moby dick, startups, big companies, pmarca,

35 views of social networking

2007-07-07T09:31:00.000-07:00

35 Perspectives on Online Social Networking: One fewer view than Hokusai's Mt. Fuji series. Just as much diversity.

Tags: social networking, mt fuji, hokusai, viewpoints

A FULL INTERACTIVE SHELL!

2007-07-06T23:28:00.000-07:00

iPhone SERIAL HACKED, FULL INTERACTIVE SHELL:

IT GIVES YOU A FULL INTERACTIVE SHELL
I REPEAT, A FULL INTERACTIVE SHELL

Nice.

Tags: iphone, open platform, standards, the street will find its own use for technology

Fireworks, Social Compacts, and Emergent Order

2007-07-05T18:33:00.000-07:00

Yesterday the family went to see the 4th of July fireworks just outside the Google campus, in Charleston park. Great park, lots of friendly helpful people, the kid had a blast running in the water fountain, and he saw his first fireworks show. It was great!

Then, we left (quickly, to avoid the crowds) and immediately got snarled in traffic. Of course everyone was leaving at the same time so we expected it to be slow, but we were literally not moving for a quarter of an hour. After a while we figured out that we couldn't move because other cars kept joining the queue ahead of us from other parking lots. Around this time, other people started figuring this out too and started going through those same parking lots to jump ahead. This solution to the prisoner's dilemma took about 30 minutes to really begin to cascade: Everyone else began to drive through parking lots, under police tape, on the wrong side of the road, cutting ahead wherever they could to avoid being the sucker stuck at the end of the never-moving priority queue. (Full disclosure: I drove across a parking lot to get over to the main road where traffic was moving, but violated no traffic laws.)

I wonder how the results would have been different if the people involved could communicate efficiently instead of being trapped incommunicado in their cars. I bet every single car had at least one cell phone in it, many with GPS. Imagine an ad hoc network based on cell phones and GPS, communicating about traffic flow -- nothing more complicated than speed plus location and direction, and maybe a "don't head this way" alert. It'd be interesting to try.

Theory P or theory D?

2007-07-01T12:49:00.001-07:00

Whichtheory fits the evidence (Raganwald):

Theory P adherents believe that there are lies, damned lies, andsoftware development estimates. ... Theory P adherents believethat the most important element of successful software development is learning.

Maybe I'm an extreme P adherent; I say that learning is everythingin software development. The results of this learning are captured incode where possible, human minds where not. Absolutely everything elseassociated with software development can and will be automated away.

Finally:

To date, Theory P is the clear winner on the evidence, and it’s noteven close. Like any reasonable theory, it explains what we haveobserved to date and makes predictions that are tested empiricallyevery day.

Theory D, on the other hand, is the overwhelming winner in themarketplace, and again it’s not even close. The vast majority ofsoftware development projects are managed according to Theory D, withlarge, heavyweight investments in design and planning in advance, verylittle tolerance for deviation from the plan, and a belief that goodplanning can make up for poor execution by contributors.

Does Theory D reflect reality? From the perspective of effectivesoftware development, I do not believe so. However, from theperspective of organizational culture, theory D is reality, and youignore it at your peril.

So this is a clear contradiction. Why is it that theory D is sosuccessful (at replicating itself if nothing else) while theory Planguishes (at replicating)? Perhaps D offers clear benefits to itsadherents within large organizations -- status, power, large reportingtrees... and thus P can't gain a foothold despite offering clearorganization-level benefits.

But I suspect that it's simpler than that; I think that people simplydon't really evaluate history or data objectively. Also, it may bedifficult for people without the technical background to really howdifficult some problems are; past a certain level of functionality,it's all equally magic. The size of the team that accomplished a taskthen becomes a proxy for its level of difficulty, in the way that highprices become a proxy for the quality of a product in the marketplacefor the majority of consumers. So small teams, by this measure, mustnot be accomplishing much, and if they do, it's a fluke that can beexplained away in hindsight with a bit of work.

Somebody should do a dissertation on this...

Theory P or Theory D?

2007-07-01T12:49:00.000-07:00

Which theory fits the evidence (Raganwald):

Theory P adherents believe that there are lies, damned lies, and software development estimates. ... Theory P adherents believe that the most important element of successful software development is learning.

Maybe I'm an extreme P adherent; I say that learning is everythingin software development. The results of this learning are captured incode where possible, human minds where not. Absolutely everything elseassociated with software development can and will be automated away.

Finally:

To date, Theory P is the clear winner on the evidence, and it’s noteven close. Like any reasonable theory, it explains what we haveobserved to date and makes predictions that are tested empiricallyevery day.

Theory D, on the other hand, is the overwhelming winner in themarketplace, and again it’s not even close. The vast majority ofsoftware development projects are managed according to Theory D, withlarge, heavyweight investments in design and planning in advance, verylittle tolerance for deviation from the plan, and a belief that goodplanning can make up for poor execution by contributors.

Does Theory D reflect reality? From the perspective of effectivesoftware development, I do not believe so. However, from theperspective of organizational culture, theory D is reality, and youignore it at your peril.

So this is a clear contradiction. Why is it that theory D is sosuccessful (at replicating itself if nothing else) while theory Planguishes (at replicating)? Perhaps D offers clear benefits to itsadherents within large organizations -- status, power, large reportingtrees... and thus P can't gain a foothold despite offering clearorganization-level benefits.

But I suspect that it's simpler than that; I think that people simplydon't really evaluate history or data objectively. Also, it may bedifficult for people without the technical background to really howdifficult some problems are; past a certain level of functionality,it's all equally magic. The size of the team that accomplished a taskthen becomes a proxy for its level of difficulty, in the way that highprices become a proxy for the quality of a product in the marketplacefor the majority of consumers. So small teams, by this measure, mustnot be accomplishing much, and if they do, it's a fluke that can beexplained away in hindsight with a bit of work.

Somebody should do a dissertation on this...

Tags: software development, theory p, theory d, statistics, evidence, process

Does social software have fangs? And, can it organize itself?

2007-06-28T08:43:00.000-07:00

SuwCharman just wrapped up a talk at Google (Scary Monsters: DoesSocial Software Have Fangs?) around the adoption and use of socialsoftware such as wikis and blogs within businesses. It was a good talkand the on-the-ground experience around corporate adoption wasparticularly valuable for me.

Suw reported that corporate users tend to impose their existingcorporate hierarchy on the flat namespace of their Wikis, which is finebut may not be exploiting the medium to its full potential. And Wikisearch tends to be at best mediocre. Has anyone looked at leveraginguser edit histories to infer page clusters? I could imagine anautogenerated Wiki page which represented a suggested cluster, with away for people to edit the page and add meaningful titles andannotations to help with search, which could serve as an alternativeindex to at least part of a site.

Tags: social software, suw, wikis, clustering, scary monsters

Identity Panel at Supernova, or How I Learned to Stop Worrying and Love User Centric Identity

2007-06-22T12:28:00.000-07:00

The Identity Panel just wrapped up:

(John Clippinger, Kaliya Hamlin, Reid Hoffman, Marcien Jenckes, Jyri Engestrom)
As our lives increasingly straddle the physical and the virtualworlds, the management of identity becomes increasingly crucial fromboth a business and a social standpoint. The future of e-commerce anddigital life will require identity mechanisms that are scalable,secure, widely-adopted, user-empowering, and at least as richlytextured as their offline equivalents. This session will examine howonline identity can foster relationships and deeper value creation.

It was interesting to see the reactions from the crowd and on the #supernova backchannel. There's a lot of reactions of the form "but I want to be anonymous" though what they really mean is psuedoanonymous. It's not really made clear that OpenID enables all those scenarios. There were objections to calling things like OpenID "identity" and maybe some people think that's something of a meme grab.

OpenID is definitely very simple, very focused on doing just one part of identity. It enables the unbundling of identity, authentication, and services. It lets you say "this X is the same as this other X from last week, or from this other site" in a verifiable way that's under the control of X. Is there a better word for this than "identity"?

Also, every discussion of OpenID should start out with a simple demo: I type =john.panzer at any web site, and it lets me in. Then talk about the underpinnings and the complications after the demo.

Tags: supernova2007, identity, OpenID

Poll: Best Atom Publishing Protocol abbreviation?

2007-06-20T11:21:00.000-07:00

Since there's a debate about the best-practice abbreviation to use, I thought I'd run a survey:

Tags: atom, APP, AtomPP, AtomPub, APub, APubPro, APPro, polls

Will Copyright Kill Social Media? (Supernova)

2007-06-20T11:07:00.000-07:00

(Moderator Denise Howell, Ron Dreben, Fred von Lohmann, Mary Hodder, Mark Morril, Zahavah Levine)

The promise of social networks, video sharing, and online communitiesgoes hand-in-hand with the challenge of unauthorized use. Is socialmedia thriving through misappropriation of the creativity of others? Or are the responses to that concern actually the greater problem?

-- Will Copyright Kill Social Media?

Everyone agreed that copyright won't kill social media, though it will shape it (that which does not kill you makes you stronger?) Unfortunately we ran out of time before I was able to ask the following, so I'll just blog them instead.

Mark Morrill was very reasonable for the most part, but made two outrageous claims: That DRM is pro-consumer, and that we should be able to filter on upload for copyright violations. The first claim is I think simply ridiculous, especially when the architect of the DMCA says that the DRM provisions have failed to achieve their effect and consumers are rejecting DRM wherever they have a choice. You can say it's needed for some business model, or required to make profit, but I don't see how you can say it's pro-consumer with a straight face.

On filtering, Zahavah Levine pointed out that copyright isn't like porn; there's nothing in the content itself that lets you know who the uploader really is and whether they own any necessary rights. But even if you had this, it seems to me that you'd need an artificial lawyer to have a scalable solution. (GLawyer?)

On the technical side, I heard one thing that isn't surprising: That it would be very helpful to have a way for rights holders to be able to assert their rights in a verifiable way. An opt-in copyright registration scheme that provided verifiability might be a step forward here. Alternatively, perhaps a distributed scheme based on verifiable identities and compulsory licenses might be worth looking at.

Tags: supernova2007, copyright, social media, GLawyer

Going Supernova

2007-06-19T18:11:00.000-07:00

I'll be at the Supernova conference on Wednesday (at Wharton West) and Friday. Unfortunately I couldn't make the Open Space even today. Ping me if you're there too and want to sync up.

Tags: supernova, conferences, sync, lazy web

In Which I Refute Web3S's Critiques Thusly

2007-06-18T17:55:00.000-07:00

So the other shoe has dropped, and Yaron Goland has just given some background on Microsoft's draft Web3S protocol, while Dare comments. Which seems at first glance kind of interesting and certainly could expand the field of REST based services in a big way. At the same time, I'm confused by some of the stated rationales for not extending APP the way GData does. I think there are some straightforward answers to each of the gaps he identifies:

Hierarchy

Turtles all the way down:

<entry>
...
    <content type="application/atom+xml">
        <feed> ... it's turtles all the way down! ... </feed>
     </content>
</entry>

Merging

I think this is orthogonal, but there's already a proposed extension to APP: Partial Updates. Which uses (revives?) PATCH rather than inventing a new verb or overloading PUT on the same resource. I'm neutral on the PATCH vs. POST or PUT thing, except to note that it's useful to be able to 'reset' a resource's state, so having the ability to allow this via either two verbs or two URIs is useful too. I'm a little confused though since Yaron says that they're using PUT for merging but they're also defining UPDATE as a general purpose atomic delta -- so why do you need to overload PUT?

I need to think about the implications of extending the semantics of ETags to cover children of containers as well as the container.

I do like Web3S's ability to address sub-elements individually via URIs; APP provides this out of the box for feeds and entries, but not for fields within an entry. It's not difficult to imagine an extension for doing so that would fit seamlessly within APP though.

I think it'd be interesting to look at an APP+3S (APP plus 2-3 extensions) to see how it would compare against Web3S, and whether the advantages of a stable, standard base do or do not outweigh the disadvantages of starting from something not tailored for your solution. Certainly the issues raised by Yaron are fairly generic and do need solutions; they're not new; and the thinking of the APP WG has pretty much been that these sorts of things are best dealt with via extensions.

Tags: Atom, APP, Web3S, Standards, Extensions, Turtles all the way down

Social Network Partition

2007-06-16T19:19:00.000-07:00

At work we're experimenting with social networks. It's amusing to note the non-overlap between the Orkut people and the LinkedIn people -- different purposes and different goals. And the standard wisdom is that Myspace users graduate to Facebook as their social identity evolves. Is this a function primarily of age? Once the generation growing up with social networking hits their mid-20s, will they continue to network-hop or will they settle on one? Or will they, like my office mates, sign up for all the networks any of their friends or colleagues are with?

Tags: social networks, partition, lions

Is the Atom Publishing Protocol the Answer?

2007-06-14T17:28:00.000-07:00

Are Atom and APP the answer to everything? Easy one: No.

Dare Obasanjo raised a few hackles with a provocative post (Why GData/APP Fails as a General Purpose Editing Protocol for the Web). In a followup (GData isn't a Best Practice Implementation of the Atom Publishing Protocol) he notes that GData != APP. DeWitt Clinton of Google follows up with a refinement of this equation to GData > APP_t where t < now in On APP and GData.

I hope this clarifies things for everybody.

There seems to be a complaint that outside of the tiny corner of the Web comprised of web pages, news stories, articles, blog posts, comments, lists of links, podcasts, online photo albums, video albums, directory listings, search results, ... Atom doesn't match some data models. This boils down to two issues, the need to include things you don't need, and the inability of the Atom format to allow physical embedding of hierarchical data.

An atom:entry minimally needs an atom:id, either an atom:link or atom:content, atom:title, and atom:updated. Also, if it's standalone, it needs an atom:author. Let's say we did in fact want to embed hierarchical content and we don't really care about title or author as the data is automatically generated. I might then propose this:

<entry>
    <id>tag:a unique key</id>
    <title/>
    <author><name>MegaCorp LLC</name></author>
    <updated>timestamp of last DB change</updated>
    <content type="application/atom+xml">
        <feed> ... it's turtles all the way down! ... </feed>
     </content>
</entry>

Requestors could specify the desired inline hierarchy depth desired. Subtrees below that node can still be linked to via content@src. And when you get to your leaf nodes, just use whatever content type you desire.

Alternatively, if you need a completely general graph structure, there's always RDF. Which can also be enclosed inside atom:content.

The above is about as minimal as I can think of. It does require a unique ID, so if you can't generate that you're out of luck. I think unique IDs are kind of general. It also requires an author, which can be awfully useful in tracking down copyright and provenance issues. So that's pretty generic too, and small in any case.

Of course this type of content would be fairly useless in a feed reader, but it would get carried through things like proxies, aggregators, libraries, etc. And if you also wanted to be feedreader friendly for some reason, the marginal cost of annotating with title and summary is minimal.

Tags: atom, app, life the universe and everything

Google += Feedburner

2007-06-01T15:41:00.000-07:00

This is a validation of how important feeds are to the Web ecosystem. And of course I'm personally happy they're going to Google. I think they're pretty happy too:

The local weather forecast calls for general euphoria with intermittent periods of off-the-rails delight.

Tags: feeds, feedburner, google, acquisition, cool

My God, it's full of stars!

2007-05-26T12:22:00.000-07:00

(voice over)

Tags: my god its full of stars, universe, 2001, odyssey, google

Goodbye AOL; Hello Google!

2007-05-18T16:29:00.000-07:00

Today is my last day at AOL. I celebrated my binary millennial in February, and it's time to move on... to some exciting new things over at Google. I'm going to continue to work in the community/social networking area and I plan to keep gently evangelizing user centric identity, REST, Atom, and feed technologies, among many other things. And, yes, getting products out the door too. It'll be fun.

I don't know yet if I'll continue using this blog; but regardless, http://abstractioneer.org will always resolve to where I'm blogging (anything can be solved with one level of indirection). And =john.panzer will always reach me.

Tags: john panzer, aol, google, last day, first day, moving on, abstractioneer

Iiw2007 wrap

2007-05-16T21:55:00.000-07:00

New Journals features: Video, pictures, mobile... and Atom!

2007-05-16T09:58:00.000-07:00

The team just added some cool features to Journals last night. There's a new button that lets you easily add pictures from various Flickr, your computer, AOL Pictures, or an arbitrary URL. There's a video button that lets you upload a video to embed in your entry or About Me, or record directly from your webcam. The latter uses the Userplane video recorder widget, which was a breeze to integrate with. We're also highlighting our mobile posting feature at http://journals.aol.com, which lets you post via your cell phone (or email!) including pictures or video. Here's a quick trick: You can use this feature to integrate iPhoto with your blog; just choose to Share via email and put in your blog's email address.

We've also made some changes to our Atom API to bring it more into line with the draft APP standard; it's not 100% there yet but it's close and certainly usable.

Tags: journals, release, pictures, video, userplane, mobile, iphoto, atom, api, app

At iiw2007a: Concordia (Eve Maler)

2007-05-15T10:40:00.000-07:00

Eve draws up a diagram showing how 'bootstrapping' works in SAML/Liberty WS. Discussion ensues with many queries about Condordia. More questions than answers, but I think that people have a lot of related/interlocking problems that need solving.

Starting from OpenID, it sounds to me like all these cases are a subset of the "access a service on behalf of a user" use case; hopefully solving either one will help with the other.

Tags: iiw2007, openid, concordia, saml, liberty

At IIW2007

2007-05-14T15:29:00.000-07:00

I'm at IIW right now and also hacking away on OpenAuth and Blogs. Which does make sense since the people I need to talk to about how it should work are mostly here, with the exception of Praveen, who for some inexplicable reason prefers France.

So far so good; this curl command posts a blog entry on my Atom blog service:

curl -k -v -sS --include --location-trusted --request POST --url'https://monotreme:4279/_atom/collection/blogssmoketester' --data@/tmp/ieRN0zhgh6 --header 'Content-Type: application/atom+xml;charset=utf-8' --header 'Authorization: OpenAuthtoken="%2FwEAAAAABYgbMtk4J7Zwqd8WHKjNF6fgJSYe4RhTuitkNyip%2BEru%2FY43vaGyE2fTlxKPAEkBC%2Bf5lhWg18CE2gaQtTVQy0rpillqtUVOOtrf1%2BLzE%2BNTcBuFJuLssU%2B6sc0%3D"devid="co1dDRMvlgZJXvWK"'

Note that the token, which gives authorization and authentication, is obtained with a separate login call to an internal OpenAuth server. It looks like I need both the token and the devid; the devid essentially identifies the agent representing the user.

I should be able to post this curl command line with impunity because it shouldn't expose any private data, unlike the HTTP Basic Auth equivalent which exposes the user's password in nearly clear text. This also implies that it should be possible to avoid using TLS.

Now, if I had a standard way to get a token for an OpenID session, I could pass that in like so:

Authorization: OpenID token="...."

And my server will advertise that it accepts all three -- Basic, OpenAuth, and OpenID. I hope to talk with like minded people about this at IIW.

Tags: iiw2007, openauth, openid, Atom, REST

Sun += OpenID

2007-05-07T14:05:00.000-07:00

Tim Bray just bloggedabout openid.sun.com, which is an identity provider for Sunemployees only. Interesting! Though technically one would like to beable to do independent assertions about user centric identities ("worksfor Sun" being a reasonable assertion one could make about anyidentity). I suppose though that someone could use OP delegation topoint http://me.example.org to openid.sun.com and achieve part of thesame effect. And in the end you'll need to rely on something likeopenid.sun.com to validate assertions presumably.

AOL OpenAuth and Atom Publishing Protocol

2007-05-04T16:26:00.000-07:00

I'm looking to see how best to implement Atom-compatibleauthentication for AOL's OpenAuth service. The service provides ways for users to authenticate themselves and togrant permissions to services to do things such as read buddy lists onbehalf of a user. These permissions are encapsulated in a portabletoken which can be passed around. The primary use cases for thisinvolve pure web based AJAX applications, so making this something thata generic application can deal with isn't fully specified.

So, here are my thoughts. Let's say the client has a token stringwhich encapsulates authentication and authorization. They need to sendthis along with an Atom Publishing Protocol (APP) request.

Windows Live and GData both implement custom RFC 2617 WWW-Authenticate:headerschemes. Unfortunately they don't follow exactly the same pattern,or I'd just copy it. But using RFC 2617 is clearly the right approachif the server can support it. So here's a proposal:

If a client has an OpenAuth token, it sends an Authorization: header. The format looks like this:

Authorization: OpenAuth token="..."

where ... contains the base64-encoded token data (an opaque string,essentially).

When there is a problem, or the Authorization: header is missing, a401 response is returned with a WWW-Authenticate: header.

401 Need user consent
...
WWW-Authenticate: OpenAuth realm="RealmString",fault="NeedConsent",url="http://my.screenname.aol.com/blah?a=boof&b=zed&...."

where the status code contains a human readable message, and theWWW-Authenticate header contains the precise fault code -- NeedToken,NeedConsent, ExpiredToken. If present, the urlparameter gives the URL of an HTML page which can be presented to theend user to login or grant permissions. For example it can point to alogin page if the fault is NeedToken. A client would then need to dothe following in response:

Append a "&succUrl=..." parameter to the url, telling theOpenAuth service where to go when the user successfully logs in orgrants permission.
Open a web browser or browser widget control with the givencomposed URL, and present to the user.
Be prepared to handle an invocation of the succUrl with anappended token parameter, giving a new token to use for subsequentrequests.

Note that the succUrl can be a javascript: URL, or a custom scheme(e.g., aim:) if your destination client is not a web browser. Or itcould be pointing to a local web server(http://localhost:1080/openauthcallback.cgi) or perhaps other schemesto get the token from point A to point B. Whatever method is chosen,eventually the client will receive an OpenAuth token that it can use totry (or re-try) the Atom request.

As a wrinkle, to enhance autodiscovery, perhaps we should allow an"Authorization: OpenAuth" header with no token on any request(including HEAD). The server could then respond with a 401 andfault="NeedToken", telling the client what URL to use for loginauthentication. The interesting thing about this is that the protocolis then truly not tied to any particular authentication service --anyone who implements this fairly simple protocol with opaque tokenscan then play with anyone else. The whole thing could be built on topof OpenID, for example.

Perhaps this doesn't quite work. I notice that the Flickr APIs don'tdo this, and instead have non-browsers establish a session with aclient-generated unique key ("frob"). But this requires that users endup looking at a browser screen that tells them to return to theirapplication to re-try the request. Which the above scheme couldcertainly do as well, by making the succUrl point at such a page. Sois there a reason why Flickr doesn't redirect? There's likely a gapingsecurity hole here that I'm just missing.

Web 2.0 Expo: Mashing Up With User-Centric Identity

2007-04-18T14:43:00.000-07:00

(Reposted from my dev.aol.com blog):

PraveenAllavilli and I just finished our talk, "Mashing Up With User-CentricIdentity", at Web 2.0 Expo. The final presentation (which differssomewhat from the original version we sent to the conferenceorganizers) is available at http://johnpanzer.com/presos/MashWithIdentity.ppt.

People said it went well; I hope so. We think it's important to dealwith 'deputization' and user permissions and I hope we can get a widelyaccepted OpenID extension to do this as well. In the mean time, ourOpenAuth APIs show one way it can be done, and they enable some prettycool mash-ups.

Tags: web20expo, openid, mashups, praveen, presentations, identity, user centric identity

At Web 2.0 Expo: Recordon and Ellis

2007-04-18T11:25:00.000-07:00

Implementing OpenID. With cat pictures too!

Tags: openid, web20expo, recordon, ellis

I am so proud of my alma mater...

2007-04-17T23:48:00.000-07:00

Donkey Kong re-imagedusing 6,400 Post-It 'pixels' at UCSC. Sweet.

APP Interop Final Score

2007-04-17T15:25:00.000-07:00

The APPInterop event was a lot of fun. Thanks everybody! I saw a bunchof people who I've only talked to via email. And a few I haven't seenin a long time... perhaps since the original Atom kick-off at Googlemany years ago.

The final scorefor AOL Journals is 1-1. If you want to continue testing againstour production endpoint, feel free to update the matrix:

service document: https://journals.aol.com/atomprotocol/service.xml
user: atomprotocol
password: password

I also got a chance to play with EC2 (thanks to M. David Peterson) inan attempt to get our latest server available for testing against. Itwas tremendous fun to play with EC2 and I'd love to try using it for areal scalable application. I did eventually get a server up longenough to verify our current bug fixes, but I didn't have time to fixthe date bug that James Snell found.

I've now found 3 bugs in our date parsing code; it seems to be the mostfragile part of the parsing by far. I'd love to see what test casesother people have for dates. So far I know I need to add both UTC andvarious timezones, and now I know we need to round fractional seconds. (Does anybody but James send fractional seconds?)

At the Atom Interop Event

2007-04-16T15:48:00.000-07:00

In Mountain View.

AOL OpenAuth Launches!

2007-04-16T08:09:00.000-07:00

Praveen just blogged aboutthe launch of AOL's OpenAuthentication service. We'll be talking about this and more atWeb 2.0 Expo. Why launch another authentication service when wealready support OpenID? Because there are lots of cool things thatOpenID doesn't yet support. I think that it really supportsuser-controlled consent and permissions, for example. And Praveen isalready working within the OpenID community to add some of thesecapabilities as well.

Bee Colony Collapse Disorder and Cell Phones

2007-04-15T12:40:00.000-07:00

Thisstory is popping up all over, apparently due to the possiblecorrelation with cell phone use. It's funny that "four years leftbefore we all starve to death" doesn't get big headlines, but "cellphones might cause bee colonies to collapse" does... The graduate optimizationclass that my wife TAs at Stanford just finished their classproject. It was to write code to optimize food production given thatyou need to allocate some land for natural bee colonies. Apparently alot of the commercial bee colonies get trucked around to do their jobsfrom field to field, and bee colony collapse has been causing a lot ofproblems with this system. Whatever the cause, decentralization of beeproduction seems like a good idea.

Talk @ Web 2.0 Expo: Mashing Up with User-Centric Identity

2007-04-13T14:42:00.000-07:00

Praveen and I are going to tag-team in Mashing Up with User-Centric Identity, at Web 2.0 Expo. It's about how to leverage user centric identity to combine services in a seamless way:

In a Web 2.0 world, users combine services from many providers. Havinga common identity across providers eliminates a barrier to entry andadopting a user-centric identity system puts the user in control of howtheir information is combined. This session is about the opportunitiesand issues involvedspecifically with adopting open protocols, the solutions they provide,and open issues that remain to be solved. These include userexperience, permission management, and mashup API authentication.

Unfortunately this conflicts with David Recordon's Implementing OpenID talk at the same time, which is likely to be really good and draw a similar crowd. (Could we simulcast?)

Date: Wednesday, April 18
Time: 1:00pm - 1:50pm
Location: Room 2014, Web 2.0 Expo, San Francisco CA

Tags: user centric identity, talk, openid, mash ups, identity, web 2.0 expo

Animated Bayeux Tapestry by The People Formerly Known as the Audience

2007-04-12T11:52:00.000-07:00

Very cool video created by David Newton of the SCA group on MySpace, highlighted by the moderator, and being emailed around today. I plan to put this up on a loop in our break room.

Tags: bayeaux, animation, people with too much time on their hands

Announcing 10100100101.com

2007-04-01T22:40:00.000-07:00

I'm excited to announce that AOL has decided to leapfrog the competition and come out with the next generation blogging service, even more streamlined than Twitter. It's blogging pared down to its bare essentials. And it works great on mobile devices, RSS, and Atom feeds.

Here's how it works: You register your phone, and every 15 minutes, you get an SMS asking "How's it going? (0/1)". You send back 0 if you're feeling down, 1 if you're feeling good, and nothing if you're asleep. We call each of these a How's It Goin', or HIG. You can subscribe to your friends' HIGstreams and see how they're doing. And we're planning a visualization tool which maps the Buddy List connection matrix to a two dimensional projection showing how the emotional states of each buddy affects their neighbors:

Interestingly, having either too few or too many happy buddies makes a buddy sad. Further research is needed.

Tags: 10100100101, next generation blogging, hey its still 10:43 here on the west coast

Authenticated RSS Feeds: Drosophilia of Delegation?

2007-04-01T16:48:00.000-07:00

Jon Udell has noticed that authenticatedRSS feeds don't work very well. It's a chicken and egg situation: There are few authenticated RSS/Atom feeds because there are few feedreaders that deal with them, and vice versa. But beyond thatbootstrapping problem there's a larger one.

A lot of popular feed reader services such as My Yahoo or Bloglines arehost based. With current feed authentication mechanisms, this meansthat you have to hand your user name(s) and password(s) to your feedreader service and let it impersonate you to do anything useful. Notgreat. Recently, Kim Cameron has been blazing away at theconcept of impersonation, not just the problem of handing your passwordout. I'd like to suggest that authenticated feeds provide an idealplace to experiment with better approaches: They're read only, the baris currently very low, and there's a whole host of immediatepossibilities that would become possible once you can cleanly authorizea feed reader to read feeds on your behalf. I think the right way todo this is through a lightweight assertion mechanism that lets you say"I authorize service X to asynchronously read feed Y on my (Z's)behalf".

I'm still trying to digest all of the twists and turns of the threadbelow. I am pretty sure that whatever solution is adopted, it has tocleanly allow for the "allow a service to read a feed" to be at alluseful.

The Impersonation/Delegation Discussion
Presented in backwards chronological order
Dramatis Personae: Eve Mahler, Kim Cameron, Conor Cahill, Pete Rowley,Phil Windley

PhilWindley: On Impersonation and Delegation
ConorCahill: Delegation, Impersonation, and downright access
PeteRowley: The umpire delegates back
ConorCahill: SAML, Liberty, and user presence
Kim Cameron: Drillingfurther into delegation
Kim Cameron: Wrong-headedimpersonation

Tags: authentication, delegation, openid, cardspace, feeds, impersonation, drosophili

AOL's Overton Window += Ficlets

2007-03-29T12:57:00.000-07:00

Kevin Lawver is giving a talk about the genesis of Ficlets.com. He's the dark shadow next to the slide. It's very interesting to hearhow the team was able to experiment and push this small project out ina matter of weeks.

It strikes me that Ficlets may well increase the Overton Windowof what's reasonable to seriously consider at AOL. The agile process,tools such as Rails, OpenID, and Creative Commons have now beenlaunched, which moves discussions from "will it work?" and "can we getit approved?" to "will it work for this project?". Which I think is amovement in AOL's internal window of acceptable discourse and givesother people air cover.

Tags:ficlets, overton, process, technology, ruby, rails, openid, creative commons, aol

All that is necessary for the triumph of evil... (death threats)

2007-03-26T18:35:00.000-07:00

Just read Kathy Sierra's Deaththreats against bloggers are NOT "protected speech" post. I feelsick, and sad, and angry that people like that have that kind ofpower. They should not. If cyberspace is going to be a realsociety, we can not permit this. If this behavior were goingon in a physical public street, the perpetrators would be ostracizedand then arrested. Sounds like the police is working on the latter;great. In the meantime we should publicize and ostracize the behavior,and the people responsible for it, in all the ways that we can.

Kathy, you have my sympathy and support.

Netscape.com += OpenID; more...

2007-03-25T22:55:00.000-07:00

Note: The folks over at Netscape are doing this on their own and I haveno inside information whatsoever. So I can blog freely.

It's great to see Netscapestart accepting OpenIDs. It's good to see the user experienceissues being worked out -- the best methods will win out over time. Dmitry Shechtman calls for theelimination of registration, though what he actually says is a bitstronger:

The usefulness of OpenID is void if yourservice requires all users to sign upregardless of whether they have an OpenID. This doesn’t mean that youshouldn’t ask them to provide additional details, such as an activee-mail address (although OpenID Simple Registration usually handlesthat for you). You can do it after the user signs in for thefirst time. Just don’t ask her to sign up.

I understand this point of view, but disagree:

Users expect to find "sign up" or "register" links when going toa new site; not providing them breaks their expectations and that's bad.
If a user doesn't have an OpenID already, they do need toregister with your site. You have to make this just as easy as beforeyou added OpenID to your UI.
Sometimes you really do need to gather an e-mail address, agreeto Terms of Service legalese, etc. before using a site.

Whether you call this "providing additional details" or "registration"the flow is much the same. I do agree that in the long run itwould be better if you could simply let OpenID-enabled people sign in,collect whatever registration info you can automatically, and thendecide if and when to bother them with registration administrivia atthat point or when you actually need it. In the short run, it's simplynot feasible to optimize for OpenID experience at the expense of yourother users.

Unfortunately, you don't currently know who has an OpenID and whodoesn't. Maybe there's room here for a service: gotopenid.org. Yousign in there once with your OpenID, and it cookies you as someone whowants to use OpenID wherever possible. Sites that want to offer anOpenID-streamlined experience could make a JSON call tohttp://gotopenid.org/check.js. This would tell the site that the userknows what OpenID is and prefers to use it for signing in.

On a side note, it looks to me by the screen shots that the AOL/AIMscreen name integration is simply using the AOL OpenID identityservice. We'll see on Monday; certainly that's the easy way to do it-- a convenience wrapper around an OpenID sign in.

Tags: openid, launch, openid consumer, aol

Ficlets launched (with OpenID!)

2007-03-10T07:50:00.000-08:00

Congratulations to the Ficlets teamon their launch(escape?). In addition to being a neat site, it's also a greatdemonstration of what's possible with a small, motivated team and awillingness to take risks. It's also AOL's first OpenID consumer;meaning, you don't need to sign up for an AOL / AIM screen name to useit.

Tags: openid, ficlets, launch, openid consumer, aol

Wordpress.com += OpenID

2007-03-06T21:38:00.000-08:00

Simon blogs it. Cool! Wordpress is a provider only for now, not a consumer. Meaning you can't use http://journals.aol.com/screenname to leave comments on Wordpress.com blogs. Which is fair because you can't use a Wordpress URL to leave comments on Journals yet either. The effort to consume OpenID is higher than providing identification on top of an existing authentication system. Hopefully soon (for both).

Tags: openid, wordpress, asymmetry

My millenial milestone (base 2)

2007-03-04T14:14:00.000-08:00

Milestone this February: 1000₍₂₎ years working at Netscape and then AOL. What's next?

Tags: millenium, whats next

Resource Oriented Architecture? Inconceivable!

2007-03-03T19:55:00.000-08:00

I'm looking forward to the new RESTful Web Services book. I recall a 'Web Services' conference session a couple of years back on where the presenter essentially declared that WS-* had 'won' the standards 'wars'. He dismissed REST with a wave and an indulgent chuckle. Inconceivable!

REST's most visible implementation -- the Web -- has clearly been insanely popular as a read-only space. There are many books on how to put documents on the Web. There are very few (no?) books on how to apply the full REST style to distributed programming over HTTP (inconceivable?). It's appropriate that RESTful Web Services starts with a manifesto:

We want to restore the World Wide Web to its rightful place as arespected architecture for distributed programming. We want to shiftthe focus of web service programming from an RPC-style architecturethat just happens to use HTTP as a transfer protocol, to a URI-basedarchitecture that uses the technologies of the web to their fullest.

Does this remind anyone else of the clifftop scene from The Princess Bride?

[Scene: At the top. Fezzik, Vizzini and Inigo are looking down at
the masked man climbing the cliff after Vizzini has cut the rope]

Fezzik:     He's got very good arms.

Vizzini:    HE DIDN'T FALL? INCONCEIVABLE!

Inigo:      You keep using that word. I do not think it means
            what you think it means. [pause] My God! He's
            climbing!

Tags: REST, ROA, inconceivable, manifesto, i don't think it means what you think it means

The Door Whisperer

2007-02-26T12:01:00.000-08:00

I'm playing around a bit more with our Journals mobile upload feature. It not only lets you blog photos, but also short videos. Here's my son learning how to command doors:

I've got it set up to drop the results into a draft blog first, then previewing and copying the results over to a public blog later. It'd be nice to automate that step a bit more. Sometimes you really don't want to liveblog your mobile life.

Tags: video, mobile blogging, door whisperer

OpenID podcast (idcast.org)

2007-02-22T19:18:00.000-08:00

John Mills just put up the podcast for the first idCast discussion from yesterday. I really enjoyed talking with Jon, Gabe, Drummond, Scott, Mike, and Dmitry about OpenID. I think the discussion around XFN, microformats, and OpenID is just getting started. (And I apologize to everyone for my flaky mike or network or whatever the problem was -- Jon did a great job of cleaning things up.)

Tags: OpenID, idCast, xfn, microformats, flaky mike

The value of open identity

2007-02-22T17:10:00.000-08:00

This seems to be turning into OpenID Month, but Fred Stutzman just posted a great explanatory article about OpenID at dev.aol.com: OpenID and the Value of Connected Identity. It does an especially nice job going beyond the single-signon aspect and talking about the value of connecting all your the various relationship networks together, with you in control over how they're used.

Tags: openid, claimid, identity

Your blog is your OpenID

2007-02-21T10:55:00.000-08:00

As of this morning's Journals update, you now have a couple of new choices to use for your OpenID:

http://journals.aol.com/screenname
- or -
http://journals.aol.com/screenname/blogname

When would you want to use one of these instead of http://openid.aol.com/screenname? It depends on what you're doing. If you're leaving a comment over on a LiveJournal, you may want to point back at your blog so they can come and comment on yours. This lets you do that in a verified way. If you have more than one blog, the first URL gives a synopsis of all your blogs, and you may want to hand that out instead. These are for experimentation only at this point and may change, so please don't use one of these for a permanent identity quite yet. They both delegate to http://openid.aol.com/screenname so all three are effectively transparent aliases of each other. Which one you use is up to you.

Another subtle change: When someone comments on your blog, their screen name now links to a search for their AOL Journals. So when you click on their signature, you can get to their list of blogs if they have any. This makes it slightly easier to go back and comment on their blog.

We're also experimenting with hCard support. The signatures on entries are marked up so that hCard processors will pick up on the fact that they name a person that has a URL -- and that URL is an OpenID. We don't do this for comments yet. I think we'd want to give commenters control over how much information gets published about them. Combining this with OpenID opens some very interesting possibilities. People who opt in would be able to verifiably link their comments across OpenID-enabled sites, automatically track responses to their comments, or find other people who write similar comments. And a lot more things I haven't thought of.

Tags: OpenID, AOL Journals, blogs, hCard, distributed reputation

Digg += OpenID!

2007-02-20T17:13:00.000-08:00

I'm about to run out the door, but just had to blog (flog?) this: Digg will support OpenID! This is obviously huge, and particularly cool for us considering the features we're about to put into production for AOL Journals tonight.

(There are so many OpenID announcements that I feel the need for shorthand. Thus, "X += OpenID", plus ! for a particularly good move.)

Tags: OpenID, Digg, synergy

AOL and 63 Million OpenIDs (from dev.aol.com)

2007-02-16T15:58:00.000-08:00

I'm following up on the OpenID discussion over at dev.aol.com/blog:

Yesterday, I blogged about AOL's work-in-progress on OpenID. It generated a lot of positive commentary.I realized after reading the reactions that I buried the lead: Thereare now 63 million AOL/AIM OpenIDs. Anyone can get one by signing upfor a free AIM account. This is cool.

(full article)

Tags: OpenID, identity, social software, user centric identity

AOL and OpenID: Where we are

2007-02-14T23:22:00.000-08:00

It's not really a secret that AOL has been experimenting with OpenID. As I've said, I think that user-centric, interoperable identity is hugely important to enable the social experiences we're trying to provide. This is a work in progress, but things are coming along thanks to our authentication team's diligent effort. Here's where we are today:

Every AOL/AIM user now has at least one OpenID URI, http://openid.aol.com/<sn>.
This experimental OpenID 1.1 Provider service is available now and we are conducting compatibility tests.
We're working with OpenID relying parties to resolve compatibility issues.
Our blogging platform has enabled basic OpenID 1.1 in beta, so every beta blog URI is also a basic OpenID identifier. (No Yadis yet.)
We don't yet accept OpenID identities within our products as a relying party, but we're actively working on it. That roll-out is likely to be gradual.
We are tracking the OpenID 2.0 standardization effort and plan to support it after it becomes final.

Update: Thanks for all the responses; I've posted a followup over on dev.aol.com.

Tags: OpenID, identity, social software, user centric identity

How many different identities can one person sanely manage?

2007-02-11T22:05:00.000-08:00

...asks John Udell, in Critical mass and social network fatigue. I'd argue that the answer is about one and a half, in the long run. (The Shockwave Rider pushed it a lot further, but then he was founding cyberpunk. Also, he was fictional.)

When "identity" was just basically a system-local nickname used between you and a service, having multiple names for yourself was a minor inconvenience at worst, and sometimes mildly useful for privacy. Now that the services are more and more intermediaries between people, nicknames are a problem. Jon notes:

Years ago at BYTE Magazine my friend Ben Smith, who was a Unixgreybeard even then (now he’s a Unix whitebeard), made a memorablecomment that’s always stuck with me. We were in the midst of evaluatinga batch of LAN email products. “One of these days,” Ben said in, Ithink, 1991, “everyone’s going to look up from their little islands ofLAN email and see this giant mothership hovering overhead called theInternet.”

Yep. Email is still the killer app of the Internet, and it wouldn't be if it were still stuck in LAN mail silos. Though spam is now so bad that I literally can't send email to a few people who can't manage their spam filters. That's a real issue and represents a real opportunity as well.

While OpenID isn't a total solution for these issues, it's a critical piece of infrastructure. As an example of the kinds of things it enables, take a look at Jyte.

Tags: identity, social networks, openid, jyte

Jangl, community, Internets

2007-02-10T14:05:00.000-08:00

I'm hanging out at the Community Next conference at Stanford this afternoon. A couple of quick takes:

Jangl provides control over who can call you; put a widget on your web page and people can call you but not get your real phone number. And Jangl lets you control who actually gets through. Really a feature that the cell phone companies should provide but don't -- so if Jangl is successful, will the telecoms just copy it?

Also, from skinnyCorp:

(Don't you just ask people to roll coins through the tubes?)

Tags: skinnycorp, threadless, community next

OpenID + CardSpace = Ubiquity

2007-02-07T22:24:00.000-08:00

It's great to see Bill Gates announce that CardSpace will integrate with OpenID. Convergence, and momentum, is building.

It's also great to see people like Kevin blogging about OpenID and AOL (OpenID: Should AOL Care?) (Yes.)

Tags: openid, convergence, cardspace

The Essential Hardness of Programming

2007-02-05T13:59:00.000-08:00

Software engineering's preoccupation is the arrangement of bits, as opposed to atoms. One of the properties of bit arrangements is that their marginalmanufacturing cost is zero; once you have an arrangement of bits, youcan make as many exact copies of that arrangement as necessary,whenever and wherever they're needed. By contrast, an arrangement ofatoms such as a bridge has a large marginal manufacturing cost, even ifyou just want an exact copy. Further, there are few physical limits tobits, while there are sharp physical limits to atoms. The only reallimit to bit arrangement is the human brain, and economics (how badlypeople want bits arranged in particular ways).

These are the fundamental reasons why nearly every software engineering project is attempting new design, and is thus hard. This is because, in the world of software, design equals bitarrangements and copying a prior bit arrangement has zero cost. Finding an appropriate bit arrangement used to have substantial cost,but that cost is falling towards zero too. So for a given project, youcan assume that competent software engineers have mostly found andcopied the relevant patterns of bits where possible, and the remainingwork is design.

Think about what the statement above means. This isn't like a civilengineer dealing with slightly differing terrain or traffic loads whenadapting an existing design for a new bridge; it's more like a civilengineer being asked to build a bridge out of Jello on Pluto. And thenext time, to build atop a moving lava flow on Mercury. In otherwords, with the easy, mechanical adaptations being taken care of bythose ubiquitous bit patterns, the problems that are left for people towork out are the hard, surprising, novel ones. Usually with nonlinearAnd in software, design really is everything; once you've taken designto a detailed enough level that the implementation is mechanical... welet the machines do it.

Which is why I winced when I read Scott Rosenberg's interview in Salon. He gets it exactly right when he notes that there's always somethingnew in every software project, otherwise there'd be no point in doingit. But he goes off the rails when he says, "...programmers areprogrammers because they like to code -- given a choicebetween learning someone else's code and just sitting down and writingtheir own, they will always do the latter." Jonathan Rentzsch hasalready skeweredthis statement better than I could. It is of course true that thereare some people who just aren't good at finding prior solutions, or atunderstanding them once found, and they may contribute to unnecessaryre-creation of software, increasing both cost and risk to largerprojects. But they're not the norm, and aren't a major cause of the"always something new" phenomenon. The essence of software developmentis new design.

This is also why attempts to map manufacturing based activities tosoftware development are at best rough approximations and at worstdangerous distractions. Software development is a knowledgeacquisition activity, not a manufacturing activity.

Tags: programming, software engineering, design, bits vs atoms

Buddy Stalker

2007-01-18T13:11:00.000-08:00

Kevin Lawver demoed a cool Buddy Stalker mashup yesterday at Mashup Camp 3 that combines AOL's feeds infrastructure, AIM , and Open Authentication to let you stalk your buddies online. All of it tied together with Rails in about 8 hours (and the last 2 hours were probably just tweaking the CSS).

Personally, I want one more level of abstraction on top of this. Something like a personal buddy zeitgeist tag cloud, updated dynamically, showing what my buddies are up to.

Tags: mashup, buddy stalker, aol, feeds, aim, open authentication

application/atom+json? and more on our converter

2007-01-16T17:07:00.000-08:00

Sam Ruby's recent post (Application/Atom+JSON) has sparked some interesting comments. Is there value in a standard way to represent Atom data in JSON data structures? I think so. I do think that we need to have a way for transparent pass-through of generic elements (which means some scheme for dealing with Atom extensions, which means dealing with at least some subset of XML namespaces).

In other news, we're working on fixing the issues that Sam noted in AOL's Atom-to-JSON converter. The good news is that most of these issues have to do with the underlying ROME library we're using, and so the fixes that Joseph van Valen is making will be contributed back to the community.

It turns out that the same issues are present even if you specify Atom as the output format; while Atom-to-Atom isn't a very useful converter, it's awfully useful as a test case. For example, the Feed Validator tells us that something is munging hreflang. It's also dropping rel="self" and other links for some reason... all of these things look like simple bugs, easily fixed. The converter actually does seem to pass through elements it doesn't understand for entries, but not for feeds, which looks like a simple oversight.

Tags: json, atom, ROME, conversion, validation

Yes, Steve, I do want my phone to be an open platform

2007-01-14T00:01:00.000-08:00

I was captivated by the iPhone announcement. It's a great looking design; I suspect that Apple, being Apple, has probably nailed the actual interaction too, including keyboard. I was ready to buy one immediately (darn that pesky FCC!). But then:

"You don't want your phone to be an open platform," says Steve Jobs.

What? Yes, I do. I want to carrry just one personal device that serves me the way I want. I do not want to be locked in to a "garden of pure ideology" (warning: ironic link) defined by any one company. I want a free market.

And: "Cingular doesn't want to see their West Coast network go down because some application messed up."

That's simply laughable. If Cingular's network is so fragile that a single poorly coded application can destroy it, they have much bigger problems than needing to lock down the iPhone. You deal with those issues at the protocol and network levels, not at the clients.

I won't even try to predict whether this is a fatal flaw for the iPhone -- it has a lot of other things going for it, and of course it is a decision which can easily be changed. But it will determine whether the iPhone is a game-changer or just a cool looking phone.

Doc Searls' take: "Well, it's good either way. Because a closed iPhone is a market opening for Nokia, Motorola and the rest of them."

Yup.

Tags: iPhone, walled garden, 1984, open platform

Generic Atom-to-JSON Conversion

2007-01-11T22:49:00.000-08:00

Yesterday, our feeds infrastructure team released a bunch of new code. There's actually a lot that this service can do; one of the cooler things it now does is to convert arbitrary Atom or RSS feeds into cross-domain-retrievable JSON data structures:

curl -v 'http://headlines.favorites.aol.com/hlserver/api/GetFeed.do?
url=http://journals.aol.com/panzerjohn/abstractioneer/atom.xml&format=atom-json&callback=cb'

which returns, in part, the data structure below. The big win is the ability for any web page to retrieve feed data from any feed source without needing to set up a custom proxy. Obviously, there's still a proxy involved here, and it's one we're running; it does both feed format normalization and caching and is highly scalable. I hope we can turn this into a supported, documented API on dev.aol.com soon.

The output looks like this:

cb(
{
"feed" :
{
"aj:accessType":
      {"xmlns:aj":"http://journals.aol.com/_atom/aj#","content":"public"},
"xmlns:sy" : "http://purl.org/rss/1.0/modules/syndication/",
"aj:blogShortName":
      {"xmlns:aj":"http://journals.aol.com/_atom/aj#","content":"abstractioneer"},
"title":"Abstractioneer",
...
"entry": [
    "title":"Why AOL Should Go OpenID",
    "published":"2006-12-15T23:15:48Z",
...

});

There are a few oddities in the output -- atom:author gets mapped to dc:creator, for example -- which I'll find out about tomorrow.

Tags: json, atom, rss, feeds, mashups, apis, web services, infrastructure

Why AOL Should Go OpenID

2006-12-15T15:15:00.000-08:00

I've argued beforethat identity is a building block -- an essential amino acid, if youwill -- for social networks. It's far from the only thing you need,but without stable, persistent, verifiable identity, it's very hard tobuild relationships. It's so important that there are specialized subnets in the human brain that recognize voices and human faces to help you remember people.

The digital world doesn't work like that. Identifying someone onlineis hard. Even solving the more limited problem of verifying that this person is the sameperson who you were socializing with yesterday online is not trivial. All social software has some mechanism for letting people verify someonline identity -- usually a user name and password. Of course thatjust means that you have different user names for different services. In the new "Web 2.0" world, though, a primary rule is for services to be open and interoperate and play together. That's difficult if people have to remember that you're leetjedi67 onservice A and urtha52 on service B. It's fine if you want to do that,but most people want to be themselves most of the time. And ourinfrastructures don't allow for that.

Well, at least they didn't. There's a remarkable convergence of usercentric identity systems happening right now. At the lightweight end,basically everyone has converged on the OpenID standard. This lets you be leetjedi.net everywhereif you want. Or at least everywhere that supports OpenID. The first,most practical benefit is that you won't need to fill out anotherregistration screen on most new services. The more long term benefitis that you get to keep your identity and your reputation with you asyou move between services.

Of course none of this matters if companies don't adopt it, so what'sthe benefit for them? Well, if their service involves a socialnetwork, it gains immediate access to both a network and an ecosystemof services which work with it. The value of a social network grows quadraticallywith the number of users; the value increases linearly as thedifficulty in connecting two users drops. Connecting two OpenID userswith is a lot easier than if you have to convince one or both to acquire a new identity.

This is the big value in promoting and leveraging a common standard. Even Microsoft is adopting open standards for their CardSpace identitysystem (and CardSpace and OpenID are talking cordially to each other,by the way). So embracing the open network, leveraging the quadraticmultiplier in network value, and competing on value added services isreally the way to go. Of course this means that you are opening upyour own services to more competition as well as cooperation). SinceAOL has already committed to open web services, this is a logical nextstep. Just playing around with ideas: What would happen if every AIMuser name were OpenID enabled? What if you didn't need to evenregister to use UnCut Video, AIM Pages, or AOL Journals?

Tags: identity, OpenID, networks, social networks, web2, user centric identity

Atom API for AOL Journals

2006-12-12T10:25:00.000-08:00

Journals exposes a very complete API for creating and managing blogs, entries, and comments. I'm working on getting the API documentation up on dev.aol.com sometime soon. But it's very easy to get started with basic blog posts. Here's an example using curl, that would post to this blog, if my password were MYPASSWORD:

curl -k -sS --include --location-trusted --request POST --url 'https://journals.aol.com/_atom/journal/panzerjohn/abstractioneer' --data @entry.xml --header 'Content-Type: application/atom+xml; charset=utf-8' --user panzerjohn:MYPASSWORD

where entry.xml is the Atom entry to be created, like this:

<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:aj="http://journals.aol.com/_atom/aj#">
<title>Blog entry title</title>
<published></published>
<content type="html">
   Hello World!
</content>
</entry>

On success, you'll see something like this in response:

HTTP/1.1 201 Created
Date: Tue, 12 Dec 2006 18:21:57 GMT
Server: Apache/2.0.54 (Unix) mod_ssl/2.0.54 OpenSSL/0.9.7e mod_jk/1.2.14 mod_rsp20/RSP_Apache2_v6_2.05-08-11:mod_rsp20.so.rhe_x86-3.v8_r1.44
Set-Cookie: RSP_DAEMON=1ceaffc0a8b18da03cfaaea9b70f236f; path=/; domain=journals.aol.com; HttpOnly
Set-Cookie: MC_UNAUTH=1; path=/; domain=journals.aol.com
Location: http://journals.aol.com/_atom/journal/panzerjohn/abstractioneer/entryid=168
Transfer-Encoding: chunked
Content-Type: application/atom+xml;charset=UTF-8

<?xml version="1.0" encoding="utf-8" ?>
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:aj="http://journals.aol.com/_atom/aj#">
<link rel="alternate" type="text/html" href="http://journals.aol.com/panzerjohn/abstractioneer/entries/2006/12/12/blog-entry-title/168" />
<link rel="http://journals.aol.com/service.edit" type="application/atom+xml" 
  href="http://journals.aol.com/_atom/journal/panzerjohn/abstractioneer/entryid=168" />
<link rel="http://journals.aol.com/comments" type="application/atom+xml" title="Comments feed for this entry" 
  href="http://journals.aol.com/panzerjohn/abstractioneer/entries/2006/12/12/blog-entry-title/168/atom.xml" />
<id>tag:journals.aol.com,2003:/panzerjohn/abstractioneer/168</id>
<title type="text"><![CDATA[Blog entry title]]></title>
<updated>2006-12-12T18:21:00Z</updated>
<published>2006-12-12T18:21:00Z</published>
<author>
<name>panzerjohn</name>
</author>
<aj:entrySource>AtomAPI</aj:entrySource>
<aj:mood>0</aj:mood>
<aj:commentCount>0</aj:commentCount>
<content type="html"><![CDATA[   Hello World!]]></content>
</entry>

There are a lot of other parts of the API, but they're best left for a full document rather than a blog post. There's also at least one known bug, where our servers don't accept the 'xhtml' content type. That should be fixed on beta.journals.aol.com this Wednesday.

At IIW2006b

2006-12-04T14:41:00.000-08:00

I'm at the Internet Identity Workshop (part B), listening to a bunch of smart people like Dick Hardt, Johannes Ernst, Kim Cameron, and of course Kaliya. Looking forward to hearing a lot of exciting developments. Already people are announcing open source libraries supporting OpenID.

Dec 5, 11:45am: There's a good article just put up at ZDNet: "The case for Openid" It's been Slashdotted already. At IIW, I've been sitting in on the basic OpenID discussions, finding out what's new with 2.0, and listening in on the user experience/microformats discussion. The latter is potentially interesting; at least there are specific short-term obvious next steps, like supporting XFN, that would help enable potential applications down the road. This is a very difficult thing to sell to business people, though. Maybe there's a session on that -- evangelizing to the business?

Tags: iiw2006b

Caching for AOL Journals

2006-11-20T12:11:00.000-08:00

We're continuing to work on improving the scalability of the AOL Journals servers. Our major problem recently has been traffic spikes to particular blog pages or entries caused by links on the AOL Welcome page. Most of this peak traffic is from AOL clients using AOL connections, meaning they're using AOL caches, known as Traffic Servers. Unfortunately, there was a small compatibility issue that prevented the Traffic Servers from caching pages served with only ETags. That was resolved last week in a staged Traffic Server rollout. Everything we've tested looks good so far; the Traffic Servers are correctly caching pages that can be cached, and not the ones that can't. We're continuing to monitor things, and we'll see what happens during the next real traffic spike.

The good thing about this type of caching is that our servers are still notified about every request through validation requests, so we'll be able to track things fairly closely, and we're able to add caching without major changes to the pages.

The down side is of course that our servers can still get hammered if these validation requests themselves go up enough. This is actually the case even for time-based caching if you get enough traffic; you still have to scale up your origin server farm to accommodate peak traffic, because caches don't really guarantee they won't pass through traffic peaks.

We're continuing to carefully add more caching to the system while monitoring things. We're currently evaluating using a Squid reverse caching proxy. It's open source and has been used in other situations -- and it would give us a lot of flexibility in our caching strategy. We're also making modifications to our databases to both distribute load and take advantage of in-memory caching, of course. But we can scale much higher and more cheaply by pushing caching out to the front end and avoiding any work at all in the 80% case where it's not really needed.

Tags: caching, etags, squid, scalability, traffic, performance, aol journals, blogs

Jonathan Miller: 破釜沉舟

2006-11-17T23:12:00.000-08:00

Jonathan Miller wasn't a charismatic leader. But he recognized theneed to fundamentally change AOL's strategy, mappedout a new direction, and got people moving the way that he waspointing. Since last year, especially, he had been executing well andreally seemed to "get it". Giving AOL's services away for free thissummer forced the organization to focus on the new world rather thanthe old; a Rubicon crossing, or as Jon's Sifu might say, "Break the woks and sink the boats(破釜沉舟)". And as Ted notes, the strategy that Jon architected is starting to show good results.

Given all of this, Jon's departure wasa shock. Neither the communications to the rank and file nor to Jonhimself were handled well. There are plenty of rumors and speculationflying around. I hope that the Time Warner leadership team handles thesituation going forward with the openness and honesty that are due tothe people who have worked so hard to turn AOL around.

Jon, you'll be missed.

The AIM Network: AIM6, AIMPages, Buddy Feeds

2006-11-15T10:25:00.000-08:00

I love launches. This week, we're launching AIM 6 along with a major upgrade to the AIMPages beta. And they work together!

The new AIM is a big improvement; I've been running the various betas over the past many months and they've been both rising in quality and slimming down in footprint. And the UI is finally reasonable: I can now once again edit my buddy list right there in the main window. And buddies now have a little (i) icon that tells you when your buddy has published something -- anywhere. Like blog entries! Profile updates! Or, if they've set things up, new Flickr photos, Diggs, Myspace or Blogger or Xanga updates, or any custom Atom or RSS feeds you care to add.

There are some problems: It keeps telling me about their away message status, and I don't really care that Kevin Lawver is away at lunch. And I see that Kevin posted a photo but there's no thumbnail in the feed... but this is a first release, we can fix these nits. (The feed data is available at http://buddyfeed.oscar.aol.com/rss-push/aol:buddy_feed?request=user&sn=<screenname>, and as an AIMPage module named "What's New" which I demoed at Widgets Live last week.)

There's a good review of all of this on GigaOM by Liz Gannes.

The only downside of all this is that I'm mostly on a Mac these days, and there's no Mac AIM 6 client right now. On the other hand, Adium plus a feed reader works pretty well too.

AIMPages has also added AIM Pages Buddies. I don't think this is the best name, but the concept is good. It's two way, meaning that both people have to opt in to it. And by default your AIM Pages Buddies are shown on your profile (so you really don't want your whole Buddy List showing up there). The invitation mechanism is easy: When you start to add, the system sends an IM to the buddy asking if they agree. A lot better than email, if they're online. If not, they'll get reminded with a little status link at the top of their AIM Page: "Buddy Requests (42)".

Aside: By default, for newly created profiles, only your AIM Pages Buddies can post comments on your profile. You can change this in the settings to open it up if you want to. On the other hand, it's also a way to get more buddy requests.

Tags: aim, aim6, aimpages, buddy feeds, social networks, launch

Gold stars for good feed readers!

2006-11-14T15:16:00.000-08:00

Over the past few weeks, we've been rolling out support for proper cache control on the Journals blogging platform, which will help reduce both bandwidth and database load, and also make things faster in general. Last week we added ETag based cache control for both Atom and RSS feeds. Today I took a quick peek at our logs to see how things are going...

The good news is that a lot of feed readers are being great citizens. Around 33% of our feed requests get satisfied with 304 responses, meaning that clients only need to do a quick validation that they have the latest content, rather than fetching everything all over again. Here's a quick list of feed readers, in no particular order, which are doing the right thing with our servers. Gold stars for everybody!

Bloglines (http://www.bloglines.com)
FeedDemon/2.0
NewsGatorOnline/2.0
NetNewsWire/2.0
LiteFeeds/2.0
http://www.squeet.com
Firefox/1.5.0.7
Thunderbird/1.5.0.8
FacebookFeedParser/1.0 (UniversalFeedParser/4.1;) +http://facebook.com/
Windows-RSS-Platform/1.0
LiveJournal.com
Planet GBT +http://planetgbt.priyadi.net Planet/1.0~pre1-terasi+http://www.planetplanet.org UniversalFeedParser/3.3+http://feedparser.org/
Google Desktop

Pretty clearly, the UniversalFeedParser library should get a lot of the credit. It's great that the Windows RSS Platform is doing the right thing, considering the likely amount of traffic it's going to generate.

Surprisingly, we actually have a lower cache hit ratio for Atom feeds than RSS ones... mostly due to one major crawler that seems to prefer Atom feeds and never gets a 304, presumably because it's never sending an If-None-Match header:

Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)

Hey Google: It would really help the stats if you guys could support ETags and If-None-Match.

Tags: feeds, caching, etags, gold star, google-why-oh-why

Web 2.0: 2 Quick Takes

2006-11-09T14:27:00.000-08:00

Digg Labs' Swarm demo is hypnotic.
Marissa Mayer is all about the data. Also, she has hypnotic sparkly things hanging from her belt.

Launching AOL Developer Network

2006-11-09T12:01:00.000-08:00

We've been talking about the launch of our new dev.aol.com site all week. The goal is to provide a place to talk about our open services in one standard place. (We've tried the strategy of hiding our APIs in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying 'Beware of the leopard', but it didn't work out too well.)

We really want to get feedback on both the site itself and the APIs that we're exposing. Both are evolving rapidly and I anticipate that we'll be adding some new APIs there in the very near future -- there are a couple I'm going to be pushing for. So please, give us feedback, or just link to dev.aol.com and we'll pick it up.

Web 2.0, Sudoku, and EC2

2006-11-08T16:35:00.000-08:00

I'm at Web 2.0 now -- it took secret ninja moves to actually get a pass, even though AOL is a triple iridium sponsor. The place is so thick with VPs you can barely reach the free food.

In breaking news, our own Michael Chowla just won HCL's Sudoku contest and will be taking home a very nice trophy. I'll add the picture... as soon as the network actually lets my camera phone upload it... oh darn. Apparently the Ning demo suffered from some network issues earlier today too. Infrastructure!

Speaking of infrastructure: Amazon's Elastic Compute Cloud looks very cool. You could run a startup now with nothing more than a laptop and a table at a WiFi-enabled cafe.

Tags: web20, sudoku, ec2, amazon, infrastructure