Let us Branch about this! I liked it very much, and found that it raised a few key problems I'd like to try and work out here.

So the Technorati example is a problem, which is to say the web is dramatically bigger than it was at the time of Technorati's peak, and whereas once upon a time it was an achievable goal for a website to catalog all the recent blog activity on the Web it's now the case that specific blog providers can't even meaningfully index their hosted content in a meaningfully searchable way (looking at you, Tumblr).

Another problem: while we can say the better part of the last decade wasn't spent as directly on the specific needs of users or the interoperability of web services or the portability of data or what have you, we must say that the activity of large, well-capitalized consumer web companies has produced a suite of tools that make developing for the web orders of magnitude easier and faster and more capable than we ever could have dreamed of it being a decade ago. Web frameworks like Django and Rails, IAAS platforms like AWS and Heroku, workflow tools like Github—it could be argued these are servicing users just as much as open data formats did a decade ago. So is it just a matter of balance? Let's hope these aren't mutually exclusive.

I thought this was really interesting: twitter.com

That some of the problems are partly issues of scale.

There seems to be also issues of philosophy: when's the last time you used a new web service that suggested you leave your pictures or words or video on the web under a Creative Commons license? There's not the built-in instinct that your content is designed to travel outside of its home platform.

No if anything, Joel, the instinct has reversed — it's not about pushing content outward so much as pulling people in to view the content, and then never leaving. This is why Social Reader apps on Facebook were always untenable for publications, because it transformed them into content farms for Facebook.

I feel like much of this The Web As It Was moment is mere nostalgia — the core sentiment feels uncannily like "when men were men, The Greatest Generation worked the land and It Was Good."

Oh I think nostalgia is very much at work, but I'm also inclined to agree that SOMETHING has been lost (and obviously that things have also been gained—see: push button blogs, even if that comes at the price, as Anil put it, of having your handle just be a subdomain on someone else's service). Atop that list I am both nostalgic for and also deeply regret the loss of the distributed web: the day-to-day use of the web gained tremendously in terms of utility but suffered in a significant loss of charm. It is more efficient to search for something now with Google than it ever was with AltaVista, yes, but that also means so much more of the web is left unseen (we can also blame filter bubbles for this).

Also, a hunch: we are where we are because RSS was never really simple.

2006.

I loved Anil's piece but I think it missed the heart of the problem, which is that over the last decade it has become increasingly difficult for individuals to interact outside of a mediated, "monetized" environment.

Example: one of the Lost Things that personally hurt me most. Facebook used to have a place for you to type in the things you liked--before "fan pages" or any of that. By this means you could find the other twenty people in the whole Facebook universe who would have thought to type in, say, Sei Shonagon. I would go around friending people based on these random points of contact.

Now Twitter serves the same function, but Twitter like FB before it makes no money on such transactions. How long will this last?

Furthermore: search is truly horrible now. If I'd seen a page of today's Google results in 1998 I think I would probably have burst into tears.

Most of Dash's points of reference can be deflated by "the technological problems were so different" and "it wasn't such a great period anyway" and "by the way, now isn't so bad".

To give another example that I haven't already tweeted about: the brief flowering of open APIs and mashups circa 2007 (flickr, google maps, twitter, etc.) became an existential threat to Twitter, at least.

Firstly, the open API was by far the major cost driver both in hardware and development effort, and the singular source of outages. Before the rate limits, Twitter was DOS'd by its "friends".

Secondly, the open API allowed Twitter clients from rivals (e.g., UberMedia) to threaten to create a parallel network.

Both lead to the today's restrictive policies.

I say this in the entire consciousness of the value of date-ordered results, etc.

That said, there is no question that the Austin Spring of 2007 was very different culturally than today. People really believed in open standards (SOAP, RSS) were a way to stimulate innovation. And people weren't cynical about the Lessig-style intellectual property reforms (people actually thought that the technology made this a foregone conclusion).

But the culture had deep unresolved contradictions at its root, especially the tension between the "information wants to be free" longhairs and the "do not track" neckbeards.

That naivete of that world is gone for obvious reasons: it's now a big business. 2007 had no shortage of SEO douchebags. Now we have social media consultants.

There is no question that the culture of The Well, and BBSs, and MIT media labs, and hacker collectives, and the left-wing/libertarian spirit -- which all used to dominate the Internets 20 years ago -- have been in steady decline. There are very few idealists left, or at least we're drowned out by the chattering masses.

But today we have Arduino and 3D printers and Github and Maker culture and Etsy. The New Left!

So turn that frown upside-down Anil Dash.

I like the trichinological breakdown, there.

But can't agree that problems of scale are mostly to blame. It's the feast of the VCs and their eyeball-monetizing that is to blame.

Maria, your point of the loss of non-commercial third places is well taken; I addressed this a bit in my piece on Captive Atria, which basically posits that privately-owned public spaces (like Zucotti Park) will intrinsically fail the public in the same way online as they do offline.

Interestingly, a lot of people are reading my piece as saying "everything was better back then", when I'm explicit that it was not. It just had some *aspects* that were better.

The "but there was so much less data!" thing is a red herring. Sure, that's true, but we also had to have Dell drop off a costly truckload of servers and then pay Oracle a truckload of money for a database license if we wanted to try to make something cool. Some datasets are bigger, but infrastructure to serve users has advanced at a far faster rate than the data have grown.

EXACTLY wow. $40K as I recall. Just for the license.

The "so much less data" argument may or may not be a red herring. Certainly Flickr paid a shit-ton of money to scale out their photo storage.

But take Technorati. I don't know how they actually solved their search problem, (was it Oracle or MySql or custom?). But a simple custom solution for a 350k graph: each node needs only 5 bytes (2**20 > 350k), each edge 10 bytes (assuming no compression), so 40m edges is 400 megabytes. That's not many megabytes.

If you're asking, "why can't I search all old Tweets" or "why can't I paginate back past 3600 tweets (or whatever it is)" the answer is for reasons like it costs a lot of money store the posting lists for 100b tweets in RAM, not a lack of openness.

I don't think it's fair to say the scale argument is a red herring, at least not if shortcomings like Twitter's 3,200 searchable tweet limit are going to be held up as reasonable arguments against them—I think it's a perfectly valid point to make at least until someone figures out how to deploy today's cheap and accessible infrastructure in a way that allows for massively vast data sets to remain hot, indexed, and searchable in their entirety without the need to offload any of it to cold storage. Twitter hasn't done it, Facebook hasn't done it, Google hasn't indexed it, and the Library of Congress more or less gave up entirely.

This is a little bit off-piste, but I really have no idea why the "heat" issue needs to be topmost. Would anyone really mind having to wait minutes or even hours for complicated results? Why this notion that everything must be instantaneous. Just imagine the amount of carbon we could save if people were willing to wait even one second.

@mariabustillos if something takes a long time, it's either because there's lots of congestion or because it's expensive to compute.

You could support slow data export by reading from disk. However, reading from disk is extremely costly to support at reasonable throughput. That's why there isn't async data export at Twitter at the moment.

What I mean is we've created this expectation of instantaneous response *for no reason*. People should be okay with waiting if in exchange (for instance) we save emissions, we can have complicated searches of huge datasets and so on.

Just explain, maybe offer a "slow lane" or a "green lane" or something?? Does that sound crazy?

Reminds me of how one of the 37Signals guys—Jason I think?—once mentioned he wanted to see a website that didn't offer 100% uptime and instead kept hours like a brick & mortar would. Twee, but intriguing.

B&H shuts down for Sabbath. Seems to work for them.

I think one thing worth thinking about is all of these companies get to a choice point that is "Do you want to be some sort of a platform for data of whatever kind to be managed and blah blah blah?" or "Do you want to essentially be a media property, where this particular type of content is the media?"

No one has really figured out how to make money with Option 1, when the users are not profitable businesses themselves. Once you go Option 2, like Twitter & Facebook, controlling the edges of the screen, rather than the API, is the whole business, so stuff like letting you take your content with you or creative commons licensing is just not on the path.

Everyone is afraid of ending up like OG social network king AOL Instant Messenger.

Seems trivial to get lost in the weeds considering the tragedy that happened today.

Clearly @anildash raised an important question, even if the particulars of some of his points don't work.

"We'll fix these things... The pendulum is swinging back" he says. I really doubt it. OpenID and RSS and anything like it aren't coming back. Facebook is more different than AOL than it is similar. The world has moved on: the scale of the data, the means of finance, the network effects, the expectations of users -- it's all very different.

He's right in that we ought to pay more attention to the moral and aesthetic character of the Internet, and sometimes reflecting on the past is useful insofar as it makes us see the present more clearly…

If we focus on companies that are victims of their own success (like Twitter, and perhaps Facebook), yeah we'll get depressed about how much we've lost. But I think those things are gone irrevocably -- not because we stopped caring about it, but because the world (of technology, of finance) makes it impossible to exist even if we do care. The pendulum won't swing back. We should embrace what has a moral character in the present.

If you consider Kickstarter, Arduino, Github, Etsy, Maker culture, people like Bret Victor and Jonathan Blow... Many of these things were unimaginable 5-10 years ago. It's not hard to argue that we're actually living in a golden age, one of much more democratic "artisan" creativity; and thanks to Github, we are producing so much free software, without the pollution of commerce, that it would make a Marxist blush.