I wrote my first computer program at age 10, which means I’ve been programming for 32 years. For most of that time, I’ve done what used to be called ‘systems programming.’ This term is hard to explain, but I’ve always visualized it as being somewhere in the middle: not down at the kernel or hardware level, nor up at the pure, user-oriented application level. Rather, when I program I am navigating the swirly paths of APIs and protocols, moving amidst the foggy realm of interprocess communications, banging up against all sorts of scary and friendly daemons and scripts and ports and sockets and god knows what else. Systems programming is not about the edges, but rather about the center: it is the glue that binds the components, thus making a bag of parts into a coherent whole.

And this is what I like. Even in those primordial days of 8-bit microcomputers running CP/M, I was hacking on code (admittedly, written by someone else) that injected itself into the operating system, installing itself in the nether reaches of what seemed like a vast landscape of 64 kilobytes of memory, modifying the swamp of system call jump tables to intercept the BIOS calls. Why? So I could run what was then called a ‘computerized bulletin board,’ and what we’d now call a forum. (For a window into that time, see Remote CP/M Software Exchange Systems, List # 26; search for ‘Bethesda’ to find my face in the crowd of 1982.)

Eventually I discovered Unix, whose existential core not only admitted the relevance of systems programming, but reveled in it with its pipes, sockets, and processes. Programs on the scale that we’d call ‘apps’ today were rare; even those large-scale programs like Emacs were themselves an ecosystem, replete with hooks and modules and plugins.

My Mac today is a tiny ecosystem that I have let evolve out of 25 years of Macintosh technology, merged with another 25 years of Unix. Some of the utilities I use are descended from programs written a decade ago; every day I type commands that I’ve typed for so many years that I almost feel the patina on their letters.

Most of my research today revolves around heavily networked web-based ecologies of data, code, and the links between them. I’m exploring ways of distributing information even further apart, splitting up heavy applications into lightweight, gossamer threads of code out in the clouds. The work is intentionally airy, abstract, independent.

None of this richness is possible on the current iPhone platform. This restriction is akin to the earliest computers that only ran one program at a time; once the stack of punchcards was loaded, that program was the sole identity of that computer. In 1950, that was reasonable and expected; in 2009, it seems pointless and limited.

There is, of course, a world out there of the jailbroken iPhone or iPod touch. Talented folks have figured out the cracks and crevices, raised their flags, reskinned the official UI, even invoked the old Unix daemons to run silently in the background. But it’s not an attractive, welcoming world to me. I understand the challenge, appreciate the incredible effort in cracking open the back door to the ivory tower, and find a jailbroken iPhone to be useful, even essential. But it’s a risky world that can be entirely undone with a single software update from Apple. That’s not a healthy ecology, either.

So for now I will sit back, content in my comfortable Mac environment, happy with the symbiotic flora and fauna of code, and observe. Perhaps those ivory towers will turn to follies, half-ruined structures that still remain upright amongst the barren fields of the iPhone. And perhaps even later the follies will be remade into small but lively abodes, both large and small, all cooperating to build a more inclusive, verdant, and interesting landscape. And perhaps then, I shall start to program for the iPhone.

Chipchase gives an example of wanting to spend time at a small, local bar, a place where no one knows him, no one knows where he is, and where he is certain never to bump into someone he doesn’t really want to see. (The article implies a long-over relationship.)

This expectation of anonymity is peculiar to dense, urban places. It should not be mistaken for the isolationism of the outback or countryside, nor the genericness of the suburban wastelands; urban anonymity is a protective shield against the pressure of a lot of people in a little space. In a dense city like Tokyo, a part of survival is being able to walk a block or two away and be in a different social universe.

But the boundaries of these separate universes have been slowly dissolving, due to the two heralds of this century: the mobile phone and the digital camera. The massive popularity and rapid evolution of those devices, along with the synthesis with the modern computer in the guise of the iPhone, as well as satellite GPS and cell tower triangulation, is to add another sense to the panopticon: the sense of place.

Now we can all see the nearly flicker-free stream of documentation of our lives, mediated through the Internet cloud of texting, blogging, and photostreaming.

In the early 1990s, I lived in San Francisco and participated in the early rave scene. One reason the scene held my interest was a sense of mystery and play. Obscure flyers advertised the dates and featured DJs, but often the printed directions led one only to the ‘map point’; once you found yourself there, you’d receive directions to the actual location of the rave. Sometimes that was around the corner, in a dark warehouse in the old industrial district; sometimes it was all the way across the bay in a complex of connected houses in Oakland.

Today, a single geocoded Twitter message could give the whole game away.

This issue has been covered in different degrees in various places around the net. One of the most cogent observers has been Jonathan Bailey, in his articles ‘Is Flickr Letting Down its Users?’ and ‘Why Flickr Licensing Fails’.

The basic problem is this: some third-party websites and applications have been using the Flickr API or feeds to publish photographer’s images in ways those photographers did not intend or allow. Photographers familiar with the relatively clear licensing interface of the Flickr.com website have been surprised to learn there is a ‘back door’ into their carefully-licensed images.

Some photographers reacted by stating that they did not intend their photo to appear in any form other than the original: in the HTML pages of the Flickr.com website. A few feel their only choice is to quit Flickr, or even quit the public Internet, moving their images to private domains they believe to be safer for their property. These drastic, reactionary moves are unfortunate, but understandable. The technical situation is complex and facts are few.

As a photographer, developer, and Flickr user, I decided to investigate the problem from a technical point of view. (Jonathan Bailey and others have already put forth good arguments for more ‘soft’ improvements in Flickr’s licensing model.)

I have worked with the Flickr API and syndication feeds for several years, and am familiar with Flickr’s architecture, as well as their technical documentation. (I do not work for Flickr, or have any other sort of professional affliation with them.) By reading the API documentation, testing the API using simple programs, and examining the syndication feeds’ data, I was able to determine the actual licensing information provided by the Flickr service, as well as the extent, if any, of Flickr’s enforcement methods or policy.

What I found was that the problems are serious, but not fatal. By making a few technical changes and by improving their documentation, Flickr could mitigate the worst of the problems.

I realize that many people interested in this problem may not be familiar with the technology behind APIs and feeds. I’ll do my best to explain how the system currently works, hopefully in a way that explains the necessary details, but is still understandable to nonprogrammers.

Flickr is a platform

Flickr isn’t really a website. Most people use the website to upload and browse images, but the website can be thought of as separate program which interacts with the core Flickr platform. The platform contains all the images and their metadata — tags, descriptions, license type, camera-generated EXIF data — stored in a large distributed database and managed by several servers.

Besides the website itself, many other programs interact with the Flickr platform: Aperture and Lightroom plugins, screensavers that display Flickr images, desktop uploaders, iPhone browsers, etc. The syndication feeds can be seen as just another program that interacts with the platform, although one that’s linked more tightly with the website.

All of these programs interact with the Flickr platform using Flickr’s application programming interface (API). You can think of the API a bit like searching on Google: You type in a query (a request), and you get back a list of results (a response). However, the API isn’t intended for use by humans, but rather by computer programs. It is the utilitarian face of the Flickr platform, a special-purpose website that is designed to take requests from programs, and give back responses that the programs can use.

(I’m not certain that Flickr’s own website and feeds use their API. However, I’ve heard rumors of this, and in my experience in designing systems that had APIs, it’s a good design and so a reasonable assumption.)

For example, on the Flickr website, every user has a photostream — a list of the most recently uploaded photos by that user. In the API, the photostream can be accessed by requesting a resource called flickr.photos.getRecent. Doing that has same result as looking at the photostream on the website, but the photostream is intended for a human and the other for a computer program. That flickr.photos.getRecent name is called a method; a programmer uses particular methods to do particular things in the API. As you can see, the name of the method is indicative of what the method does.

Flickr’s API has dozens of methods, each one doing something different with the Flickr platform. The API is extensive: pretty much anything you can do on the Flickr website, you can do using the API. The methods are clearly documented (at least compared to many APIs I’ve worked with; if you’re interested, check out the full list of the methods supported by the Flickr API.)

When a person interacts with the Flickr website, they get back HTML pages that look good in your web browser; when a program interacts with the API, it gets back a carefully coded outline of the information being asked for. (Technically, the API returns XML or JSON.) Unlike the HTML page, the API responses are designed to be examined and interpreted by the program doing the request.

A subtle but important fact to know is that the Flickr API itself does not return the actual image data — that is, the JPEG file uploaded by the photographer. The API (as well as the syndication feeds) is really just a way of searching and accessing (and sometimes modifying) the metadata that Flickr stores about each image. Image metadata includes items like its title or description, as well as links to the image data. But once a program knows those image data links, it accesses the images outside the API, in the same way a regular web browser can (by URL, usually specified in a img HTML element).

I point this out because I think it’s important to understand that all Flickr images are available as individual web resources, accessible to anyone, independent of the API. While some photographers argue whether this should be the case, it’s part of the reality of the current technology of the web. Once someone knows the link to an image stored on Flickr, it’s just as accessible (and copyable) as any other image on the web.

When a program requests a Flickr image’s metadata, that program may display some of the metadata directly to the user — like the title or description. It may use the metadata to determine how to display an image — like using the size of the image to leave enough space for it in a window. It may use the metadata to determine whether to display the image — like the license.

And here we arrive at the gist of the problem.

Licensing potentials

Flickr has a reasonable set of available licenses, and requires that every photograph on their system have a license. The current licenses (as obtained by the flickr.photos.licenses.getInfo API method) are:

(The ‘code’ is important, but I’ll talk about it later.)

Flickr does a good thing here, and defaults to ‘All rights reserved’ (code 0); a photographer is free to change this, either on a particular image or for all new uploads.

‘All rights reserved’ is such a common part of today’s fine print that most of us gloss over the phrase. But to a working photographer, this means something specific: rights to this image have been granted for use in a particular instance, in a particular medium, sometimes in a particular location; any other uses have to be requested (and negotiated) separately and explicitly.

But who looks at the license?

The Flickr website does a reasonable job at describing photographers’ licensing terms. It’s right there on every image page, under ‘Additional information’; most images say ‘All rights reserved.’ If you’re the owner of the image, you’ll see an ‘edit’ link, where you can select the license you desire. Although Flickr doesn’t explicitly enforce the license, they arguably make the information clear enough that only the most blatant image thief would assume the image was freely available.

However, the license data returned by the syndication feeds and the API aren’t as obvious.

Looking through the API documentation, I found around a dozen different methods that obtain information about a photo; these API methods relate to various Flickr features like sets, pools, geo-tagging, favorites, and so on.

The Flickr API documentation is quite good at describing the parameters needed to make a request to an API method. However, it’s quite bad at describing the parts of the response being returned. Flickr basically leaves it to an example, showing the XML output of an arbitrary request to the method.

For example, flickr.photos.getInfo is one of the main methods in the Flickr API, and returns all the metadata for a given image, as well as information on how to retrieve the actual image data. Here is an excerpt from the response example in the method’s documentation:

<photo id="2733" secret="123456" server="12"
    isfavorite="0" license="3" rotation="90"
    originalsecret="1bc09ce34a" originalformat="png">

See the license? No? Look again. Second line, second attribute. It’s the part that says license="3". If we look up ‘3’ in the table of license terms earlier in this post, we get ‘Attribution-NonCommercial-NoDerivs License.’

I could find no documentation that suggested how the license code should be interpreted. Nothing I found even said to use the flickr.photos.licenses.getInfo method to look up more information about the license code. Most likely, programmers who familiarized themselves with the API would eventually find the method to look up the license information. But the fact that there is no guidance in the documentation is worrying.

Further, it appears that flickr.photos.getInfo is the only method that returns the license code by default. Here are the other methods that return information about specific Flickr images:

• flickr.favorites.getList
• flickr.groups.pools.getPhotos
• flickr.interestingness.getList
• flickr.people.getPublicPhotos
• flickr.photos.getContactsPhotos
• flickr.photos.getContactsPublicPhotos
• flickr.photos.getNotInSet
• flickr.photos.getRecent
• flickr.photos.getUntagged
• flickr.photos.getWithGeoData
• flickr.photos.recentlyUpdated
• flickr.photosets.getPhotos

None of these methods return the license information by default. A programmer must ask for the image’s license explicitly (by setting the extras parameter to include the keyword ‘license’); once this is done, the methods will return the license code just like the flickr.photos.getInfo method does by default. There is no explicit documentation about this parameter, either — a programmer must infer that requesting this ‘extra’ will actually return the license code in the same way that flickr.photos.getInfo does.

The API itself does no actual enforcement. If I write a program that uses the API to show images, there’s nothing stopping me from pulling over any image that might be licensed ‘All rights reserved.’ That license’s code (0) will be returned — if I remember to set the optional parameter in the request. It’s up to me as a programmer to inspect the license status and determine whether it’s one that allows for republication.

While the various licenses are described on your Flickr account’s Set a default license page, the only place where the license code is described is in the response from the flickr.photos.licenses.getInfo API method. And that response only lists the code, the license name, and the website where the license terms themselves can be obtained.

It’s unclear how and how often Flickr might change those licenses, so to play it safe, a developer might decide to call the flickr.photos.licenses.getInfo method every time a group of photos is fetched, or at least each time the application connects to the Flickr platform.

So we have ourselves here a situation:

• The API itself does not enforce the licensing specified by the owner.
• The API does not return licensing information unless explicitly requested (except for one method).
• The license information returned for an image is coded as an arbitrary number, whose meaning may change over time.
• The license code must be looked up using a second API method to determine the actual license.
• The license information is not sufficient to determine whether a given use is allowed.

Do all programs use the API?

Although the Flickr API is well-designed, it’s still complicated. Because the API works over the network, it depends on an infrastructure and knowledge that many desktop or web programmers lack.

In response to this, modules of code have been developed to act as ‘middleware’ between the actual Flickr API and the application. Programmers call these ‘libraries’ or ‘frameworks’; Flickr calls them ‘API kits.’ These kits (most of which have been developed by independent developers, not Flickr) are available in all major programming languages and systems, and many applications do use them instead of programming directly to the Flickr API. Flickr publishes a list of most of the API kits available on their Flickr Service page.

Because the Flickr API is considered to be conceptually well designed, most API kits simply mirror the native API calls, adapting to the peculiarities of a language or operating system where necessary. So most programmers end up consulting the Flickr API documentation anyway, even when they aren’t directly using the API. Some API kits provide a simpler view of the Flickr API by abstracting or generalizing the methods.

Any third-party application, whether it uses the Flickr API natively or one of these API kits, has to apply to obtain a ‘key’ that allows access to the API; this key is included in every request for an API method. As part of the application process, Flickr requires an email address. Flickr therefore knows in theory who to contact about the usage of any particular application that uses the API, either directly or indirectly.

The problem raised by API kits is that another level of (mis)understanding and (mis)interpretation has been invented. If Flickr itself has not provided sufficient tools or documentation to interpret the license of a particular image, the developer of an API kit can hardly be expected do better. And that developer could do worse, if a poorly designed API kit obscured or removed the licensing information.

What about the feeds?

Flickr offers several different kinds of syndication feeds. They’re quite useful; I monitor many photographers’ photos using these feeds, and then browse their newest images in my news reader.

While Flickr publishes in all the major feed formats — Atom, RSS, RSS2, and RDF — the only one that contains any licensing information at all is Atom. And it is only this sad, broken line found in each image’s feed entry:

< link rel="license" type="text/html" href="deed.en"/ >

While this is a valid Atom idiom (see Specifying a License at the Creative Commons’ wiki pages), it’s not specified correctly: the ‘deed.en’ link should either point to a full address (perhaps http://creativecommons.org/licenses/by/2.0/deed.en_US), or be a valid path relative the feed’s base address. It’s neither of those, which make the license information completely invalid. Not to mention that it’s even more difficult than in the API example to determine what rights granted actually are.

So, effectively, Flickr is offering no licensing information with their syndication feeds.

Why are they doing this?

The choice of Flickr’s founders to open up so much of the platform in a relatively public way has made the Flickr community stronger and far more interesting than a more static, centrally-designed website. However, their decision to treat licensing terms as just more (optional) metadata shows a disconnect between data accessibility and respect for property.

I don’t believe Flickr is being malicious. I think they care about copyright; their support of the Creative Commons licenses was groundbreaking, and they do make it relatively easy for their users to select a license. And the fact that they default to an ‘All rights reserved’ license says a great deal about their attitude.

Flickr also requires that anyone who uses the API comply with each image’s assigned license. Here is section (1)(a)(ii) of the Flickr APIs Terms of Use:

Comply with any requirements or restrictions imposed on usage of the photos by their respective owners. Remember, Flickr doesn’t own the images — Flickr users do. Although the Flickr APIs can be used to provide you with access to Flickr user photos, neither Flickr’s provision of the Flickr APIs to you nor your use of the Flickr APIs override the photo owners’ requirements and restrictions, which may include “all rights reserved” notices (attached to each photo by default when uploaded to Flickr), Creative Commons licenses or other terms and conditions that may be agreed upon between you and the owners. In ALL cases, you are solely responsible for making use of Flickr photos in compliance with the photo owners’ requirements or restrictions. If you use Flickr photos for a commercial purpose, the photos must be marked with a Creative Commons license that allows for such use, unless otherwise agreed upon between you and the owner. You can read more about this here: www.creativecommons.org or www.flickr.com/creativecommons.

But this legalese, however well it’s written, coupled with the associated lack of technical advice about applying the license on a given image, leads to the situation like Arthur Dent faced in The Hitchhiker’s Guide to the Universe: licensing terms for images are ‘on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying “Beware of the Leopard.” ’

Most programmers I know develop software by experience, not by specification. We play, experiment, poke at a system and see what comes back. If a piece of that data looks interesting, we’ll look at it closer. If it seems unimportant or optional, we may never examine it closely.

To tell you the truth, I never noticed that license="3" sitting there in the response, either. But I would have noticed if Flickr’s documentation had a big box saying: ‘You must always examine the license value, and only transfer or display the image if you can comply with its assigned license.’

By making the request for licensing terms optional, or at best obscure, Flickr is effectively saying that following the terms is also optional.

How Flickr could improve the situation

Flickr could rectify this situation, and reduce the confusion and misinformed reaction, by making a few simple technical changes to its API, and minimally reaching out to its community of photographers, viewers, and developers.

Return licensing status by default in the API, and include additional attributes for license name and link, avoiding a second method call.

Document and emphasize this change in the documentation.

Publicize this change to developers of API kits, as well as developers of applications that use Flickr APIs.

Correct the broken syndication feeds and include licensing information and links wherever possible.

Clarify the permissions settings page so photographers understand the rights they are granting or releasing.

Flickr may understandably want to avoid this. After all, their default license is ‘All rights reserved.’ If most of their users (whether naive or advanced) keep that license, and they advocate for third-party applications only fetching Creative Commons-licensed images, the result may be little of Flickr’s content being legally available to the applications. This would diminish Flickr’s goal of ‘Share your photos. / Watch the world.’

Some have posited that Flickr’s passivity about enforcement comes from an attitude of being a something like a common carrier: Flickr may feel they would be liable if they actually examined and judged the usage of Flickr content in terms of copyright and license. However, by making licensing status obscure and unobvious, Flickr is certainly not helping, and arguably could be accused of contributing to copyright violations through negligence.

Flickr has an opportunity here to educate its community about the benefits of copyright. Like its pioneering effort in offering the Creative Commons licensing model, Flickr could make copyright understandable to photographers, viewers, and developers of applications that use those APIs.

How does one resolve the Absolute Good of semantic HTML against the need for a CSS layout framework?

But I must ask: Semantic to whom? The framework? The browser? The web developer? The user? The user who peers into the HTML page to see how well/poorly it’s constructed?

Semantic HTML has always seemed like a misguided idea. I have no problem with the idea of expressing content semantically, but I’m not sure HTML is the right place to take the fullest advantage of semantic markup. It’s language is too limited, too concrete, and was designed with the analog of printed, mostly textual publications.

And CSS doesn’t help: its generally a one-to-one styling mechanism. It can bridge generic presentation with specific presentation (‘make this header Helvetica’), but it can’t provide more abstract functions: eg, creating a navigation system. This abstraction, I think, is what’s missing in the HTML/CSS universe.

At HTML’s inception, there were two worlds: one of pixel-level object interaction (CD-ROM/multimedia) and one of hypertext (SGML/running text). Tim Berners-Lee chose the latter. Interestingly, early HTML was all semantic: you marked up headings, paragraphs, citations, lists, etc. Here is the original ‘spec’ for HTML.

Most of those elements still function in modern HTML. And in fact this is the only way I’ve been able to keep myself sane and still do web development: use old-style HTML elements (h[1-6], p, em, ul/li, dd/dl/dt) for marking up the text structure, use logical elements (div, span) for defining areas that need special treatment, and very occasionally change the style of elements directly (eg, <p class=”foo”>). I will even check sites using the lynx/links text-mode browsers just to make sure I haven’t made a mess of it all.

The most pure method seemed to be pushing XML down to the browser, and using something like XSLT to convert truly semantic markup into presentation markup. I think it was an idea ahead of its time, though; we were still knee-deep in the browser wars, decent web frameworks were rare, and most users/developers were still getting their heads around CSS. I suppose nowadays the answer is XMLHttpRequest, JSON responses, and buckets of DOM-modifying Javascript code.

Of course, technology has moved on and we’ve solved this in a few ways:

• Domain/content-type/browser detection: Instead of trying to make one perfect HTML/CSS page that properly renders on all browsers, give it up and create pages that are more specific for the user’s purpose. Just go full throttle and be unsemantic and concrete.

• High-level widgets: Express the content semantically on the back-end, and implement either server-side widgets that build HTML that is then sent down the pipe, or client-side Javascript widgets that are rendered by the browser.

• Hold your nose and use a framework: I know this has been my attitude in my recent usage of YUI. Yeah, it looks sucky and is unintuitive, but it works and I know I will never again vent my frustration for ‘hacks modes.’ Of course, I still need a site-specific CSS file, but I find that its content is far simpler and less browser-dependent.

Bill Burcham continued:

Compare doing layout in semantic HTML and CSS to doing layout in Adobe InDesign for instance. I assert that the requirements addressed by the two approaches are roughly equivalent and yet you hardly have to be a high priest to do layouts in InDesign. You just lay things out. Sheesh.

But this is apples and oranges: you’re comparing semantic HTML/CSS with (presumably) non-semantic page layout.

Doing semantic layout in InDesign is actually a total pain. I do a lot of InDesign layouts (these days, probably more than websites), and am often irritated that I have to use “hacks” to, for example, get the first paragraph in a chapter flush left but remaining paragraphs indented. There’s no apparent way to do it other than a special substyle of a the paragraph with no indenting, and manually setting the first paragraph to that substyle. Worse, there’s no way to have the chapter title appear a few picas down on a page without either moving the text box handles, or adding a dummy pre-title line that has the appropriate amount of leading.

InDesign layout seems easier because the end result is just ink on paper; its semantic meaning is implicit in the relationships of the page elements. Your choice of paper size, spacing, line measures, type size, font, etc., all create a meaning (if the design is successful) that the reader understands. The effect is the same regardless of whether you (the designer) create the elements by directly specifying text blocks, fonts, etc., or whether you use InDesign’s XML markup, stylesheets, anchors, etc.

And this gets back to my original point: who is the semantics intended for? All that really matters, in the end, is that the user/reader sees the printed/web page and understands its meaning.

Akin to the houses built into the castle walls, the early personal homepages were homely and homemade. The architecture was vernacular, the materials simple: hand-coded HTML and simple GIF images. We gained access through the twisted doorways of the tilde characters in the URLs; an address like http://meer.net/~johnl/writings/icecream.html was a common sight. It was a time before domain names became brand names, and before URLs became inscrutable.

Those days before the browser wars were primitive but invigorating. Web publishing technology was simple and relatively accessible. If you could run a text editor, you could learn enough HTML to design your first page in just a few hours. Browser bugs and incompatibilities existed, but the markup syntax and layout model was so simple that bugs could be worked around. No one considered pixel-level positioning, custom fonts, scripting, or stylesheets. There was simply no way to do it, short of large graphic files with image-maps. A page changed only when the creator changed it; the web was expansive, but static. The worldview of the early web was about description; we described the way we wanted the page laid out, and so it was.

Many years have passed. The kingdom of the hobbyists and hackers has scattered; the castle walls have come down, replaced with carefully designed monuments expertly built with exotic materials. No one hand-hews raw HTML tags, or lifts their files into place using FTP. Most of the old personal homepages were left behind; some still exist in dark corners of the Internet, their simplicity holding them together like baling wire and tin cans.

The movement towards WYSIWYG authoring tools (the words themselves should have been a warning) began the industrialization of the web. Like a buyer of a Sears Craftsman house, a homepage author could begin by searching through a library of designs, templates into which their personal vision would be slotted. The raw materials of HTML would be hidden from him: he would need only to design with the abstract GUI tools of DreamWeaver, FrontPage, and other desktop and web-based applications too many to mention. These are still with us, still uploading horrendous, hacky code to homepages.

But even templates could not satisfy. We needed the machines to not just blindly copy, but create. The age of informational robotics, of dynamic content, was upon us. Calendars! Menus! Forms! Widgets! Plugins! Action! A webpage was not just a description, but directions in many languages to many players: the backend code creating brand-new and perfectly adapted content for each viewer, the web server constantly shuffling data, the stylesheet converter prettying up the elements, the layout engine placing boxes to and fro, the JavaScript perversely modifying its own document, and way back in the rear of the hall, tired and overworked, the decrepit HTML attempting to glue it all together.

In the sprawl, we lost track of the open space, the squats where the personal homepages had largely existed. Gone was the open access in the free space of a borrowed server; schools and companies no longer permitted personal use. Evolution through generations of ISPs left us with the shiny but controlled and corporate expanse of the AOLs, Earthlinks, Yahoos, and Googles. Creating a ‘profile page’ in those super-franchised spaces offered only the illusion of a web-room of one’s own.

If we found our way to the real estate being hawked by the commercial webhosting services, we pressured ourselves to make a big splash. Our own domain, custom code, a database, mailing lists, RSS feeds, perhaps even a saleable product or two… We became celebrities of our own making, abandoning the simplicity of the homepage signed with a tilde at the back of a borrowed server, and lit out for the bright lights of the owned and paid-for personal site. But even that was ultimately unsatisfying, even frustrating: the technology had grown so complex, and the interface expectations so high, that one spent more time debugging browser bugs than creating a personal space.

Yet through all this the homepage survived. Industrialization, automation, centralized hosting, and the desire for an online personal space finally created that awkwardly and anti-climactically-named prime construct of the early 21st century: the blog.

The modern blog is as rigid and formal as the pioneering homepages were fluid and diverse. Almost no one, save the geeks, creates a blog from scratch, purpoase-built to fit their purpose; only a few people customize their blogs beyond selecting from a tiny catalog of ‘designer’ themes and choosing a few widgets. Most of the millions of blogs are hosted on the same half-dozen server farms. Creating a blog is easier than finding parking in the mall.

We could almost just stop here. The architecture of the blog is simple, but useable; it’s flexible, but not overwhelming. I’ve seen amazing websites built on top of blogging platforms, and the simple abstractions of posts and pages (plus decent widgets for WYSIWYG editing) finally rises above the tedious construction of HTML files.

But a blog is, in the end, just a notebook. It’s perfect for some people, a reasonable compromise for others. But often a blog ends up where it started: a personal journal, in reverse chronological order. Its constraints can be freeing, or frustrating.

However, such criticism ignores two important elements of blogs that point away from the self-centered act of journaling, and towards the new personal homepage. Those two elements are feeds and comments.

Comments (and trackbacks, to a lesser extent) enable the blog to not only exist as an individual online space (which may have ‘active’ widgets, as per the second-generation web), but as an interactive space. No longer is a visitor a passive viewer in a homepage; they can record their reactions, and far beyond the old-school ‘guestbooks’ that gave a brief nod towards interaction, the comments become a part of the space itself. Granted, comments are a limited tool in a shared space, but they’re important nonetheless.

Feeds allow a long-term, long-range view into the space of the blog. Feeds effectively expand the space of a site, and let me see a part of what happens in your space. The seeing is mediated by the limitation of the data that goes into the feeds, which makes certain that reading a feed does not (in most cases) replace the real experience of actually visiting your space.

And so, starting from the early static homepages, moving through the dynamic but still independent spaces, then adding a little local participation and remote observation, we now arrive in what seems to be a nomadic online world. This is not pioneering; we’re not hewing our own log walls out of the cybernetic wilderness. This is not homesteading; our MySpace or Facebook page is rarely the only place we hang our online hats. And this is not suburban consumerism; we are spending much more time in places other than Amazon and porn sites.

It’s far more effective and interesting for me to share my photos on Flickr or SmugMug than it is to install a gallery on my own site. And posting my travel tales on a shared travel-blog site like TravellersPoint is ultimately more effective than perfecting my own travel-blogging engine. The effort in getting my photos seen or my travel tales read is so much less when I chose a context that encourages that specific kind of content, and more importantly, encourages interaction between the members. I am participating not just in the general online world, but in shared contextual spaces. This is a new dimension in participating online: not only do I decide what and how to express myself, I must consider where to express myself. If I want to start creating songs, it’s probably most effective for me to find a network of other people who are fellow musicians, and who want to share and participate.

As we participate in all these blogs, networking sites, forums, and other communities, we each create and build this network that reflects ourselves. If all this content and interaction was gathered together, it might be similar to the stories told on those early static HTML homepages.

Of course, this isn’t entirely new. Since the earliest use of online forums (bulletin boards, mailing lists, and so on) in the 1970s and 1980s, anyone’s collected participation could be looked at as a sort of dispersed personal space. But that space could not be readily identified, described, or maintained, much less managed.

And I don’t mean managed by an IT staff. The breakthrough of social networks, and what the hullaballoo is largely about, is that these networks represent (and mediate) the relationships between me and all the people I interact with, and I can make choices about which connections to encourage, which to reject, and which to simply ignore. In the past, connecting all my postings, chats, and other online interactions together would have been done by inaccurate webscraping and spidering, postdated notes of memberships, and heuristic searching of archives for common email addresses or nicknames; now, by importing and exporting the definitive (for each site) information through published APIs, many sites allow me to connect together several sub-identities into a super-identity I can call the ‘online John.’

This is, in a sense, ‘John’s homepage.’ The network as a whole contains the strange objects and stranger stories that I have collected, along with the people who have witnessed them. But it is dispersed, and although more navigable and connected than in the past, still without a good starting point.

And that leads me to the original inspiration for this essay: what a personal homepage looks like to me now, and how that differs from the homepages I created in the past.

I believe there is still a purpose for a personal homepage. I still want to hand out a business card to a person I’ve just met, and say, “This is where you’ll find me.” I still want to shape the impression, show the network strands of my online personality. Currently, I can think of no way to do this except for setting down a particular place on the net where I can gather the aspects of the ‘online John,’ perhaps give a preview of where I’ve been, what I’m doing now, and where I’m going.

But while my personal homepage is a starting point — a portal,
dare I say — it is not the center. The homepage is not the content of my online identity; it points to aspects of that identity.

And so, technically speaking, my homepage is a thin skin on top of a few of my online identities and spaces: email, IM/Skype, Twitter (new for me), my photography work, my travels, and my technical work. The entire homepage is dynamic and based on external feeds and APIs: there is no local data (save for some of the overall descriptions, where I don’t have access to the appropriate APIs on the related sites; for example, Flickr doesn’t allow me to access the text of my profile via an API).

As we used to say in the old days: Please check out my homepage, at http://johnlabovitz.com.

Kent’s LA2 compressor

But the gleaners eventually come. The scrapheaps are discovered, evaluated, stripped, sorted. Dead-ends and really cheap crap is tossed. Elders are consulted in how this stuff used to work. And the technology is brought back to life in new ways, with new eyes, and with vision more often backed by art than commerce.

Deconstruct consumerism. Question the traditional modes.

Rob’s typographical sequencer controller

Recontextualize the devices.

Brian’s GainClone amplifier

Today’s technology tends toward the invisible. How small and compact can we make it? How can we remove the spiderweb of cabling? We want transparency; the device is only a player of content. But there is texture we have lost, and that is being rediscovered.

Jörgen’s homebuilt synthesizer modules

Something magical happens when technology is appropriated in such a way as to become art in itself.

Peter’s Minimal Sound Sculpture

I’m not sure of the context here: are these guys just using these soundcards to digitize vinyl to play back later, or are they using their computer system as some sort of ultra-fancy hifi receiver?

Either way, they’re describing the effects of the various sound cards, if they are good, as ‘warm,’ ‘musical,’ or with a ‘wider soundfield.’ These are attributes usually used when talking about analog gear, often in contrast to the analog domain. The unsatisfying cards are labeled ‘uninteresting’ and ‘analytical.’

We’re observing the audiophile world, which has a peculiar, extreme consumer fetishism. In this case, the sound card has become a component in an audiophile system, and so is peered at and dissected in the same way an amplifier, turntable, or speakers might be examined. There is nothing wrong with this, really — a listener should evaluate all points in the signal path.

Like the pro audio world, equipment is not entirely sacred: it can be modified, and several posters in this thread have replaced certain components (usually amplifier chips; the threads reference ‘opa637′). However, the pro audio world replaces components to reduce noise or expand the frequency response. I’m fascinated that the audiophile folks are modifying the circuits to tweak the sound itself.

I’m not saying any of this is invalid. It’s just interesting to me that the pro audio world looks at digital primarily for accuracy and transparency (primarily for recording), and uses analog to shape and warm sound, while the audiophile world either disdains digital entirely, or imposes the conventional desires onto every segment of the signal chain.

How far can this go? Will Windows XP SP1 be looked back at as having a ‘sweeter’ sound than later versions? Will there be a particularly listenable computer system, highly sought after, obscure, manufactured by a tiny firm in the back alleys of Akihabara, that is rumored to sound even better than pure analog?

(It is especially ironic that the ‘possibly related posts’ that WordPress automatically generates just below this posting are exactly the sort of thing I’m talking about.)

I didn’t get very far with the AM radio trick, it being too difficult to program, and not all that interesting, sonically.

Later in the 1980s, I used my Mac Plus to generate drum machine tracks for some songs recorded by my friend Paul & I. This was wonderful, in a lo-fi sort of way.

Over the years, as digital recording became The Next Big Thing, I would occasionally experiment with new audio converters, plugins, editors, and the like. However, I nearly always found the tools unintuitive and complex, requiring too many mouse clicks for too little return (in comparison to the beautiful simplicity of pressing a “record” button). Simulated dials and buttons just made for a messy and fussy interface, rather than somehow bringing back the intuition of a traditional studio.

And the more I exploited the plugins and other software gizmos, the less I felt like I was in a studio, and the more I felt like I was in the computer. To be clear: there’s nothing wrong with being in the computer; as a programmer, I completely enjoy the feeling of “playing” the computer. However, it seems, to me, incompatible with creating the kind of music I like to create.

(I will admit that MIDI, while a crappy protocol itself, is a wonderful idea, and very powerful in a studio. Being able to remotely control processing, including mixdown, has real potential. It’s a cool way to use the benefits of digital even with an analog signal path.)

I also felt the recorded sound quality was far less than I had been used to in the old analog world. Granted, most of these experiments were on cheap hardware. Better preamps help, as do better analog-to-digital converters, and that’s exactly what’s being advertised these days. But that’s part of the current problem with digital: cheap hardware is what most people buy, and it’s easy possible to hear the lack of depth in many contemporary recordings. 4-track analog cassette tape sounded better than some of these digital tracks.

Finally, the largest problem I’ve had with digital recording is the latency. This is a huge issue, and one that is obvious if you read the marketing materials of any of the companies producing audio interfaces. It may be fine for a MIDI-based keyboardist, looper, or primarily in-the-box sound creator, but as someone who primarily overdubs real/live instruments, the delays I encounter while recording are completely maddening. Again, this could be somewhat improved by better, more expensive hardware — eg, Firewire or PCI interfaces instead of USB. However, it seems impossible on regular Mac/PC architectures to reduce the latency to a level that is totally inaudible, especially if any sort of digital processing (compression, reverb, etc.) is being performed. I have always found it interesting that most folks selling digital audio tools are lost without an answer when I really dig into the latency problems.

The usual “fix” is to turn off any processing while recording, reducing the latency to a minimum. However, this goes against my own recording methods of using the studio as an instrument. By delaying any sort of creative texturing until mixdown, I feel like an electric guitarist forced to play without an amp, being told that I can always add that Marshall stack later.