The Broker’s Brokerage: Info Broking = Search + Find Local Information

I was, and we started this discussion (read from the bottom up)

OK, got it? So this was my takeaway: Traditional search has been about formulating queries using 1 website (such as at Google or Bing). To do this right, you had to have at least some basic knowledge of how the engine works (understanding the so-called "algorithm", the syntax for searches, etc.).

Today, people don't believe in that crap all that much anymore -- instead, they would rather know what a person whom they trust thinks. And usually that isn't really friends -- friends may help with recommending experts they trust, but to answer the questions themselves, people trust experts more than anything else ... especially trusted experts (those who have a good reputation; but note that I would probably trust a random expert [or enthusiast] more than a friend who has little or no knowledge of the subject matter). Note that I'm not talking about college degrees, but rather actual expertise -- meaning that it is a topic that someone cares about, has studied a lot and also has some degree of experience with. Hackers, for example, are such persons, as are fans, hobbysists, etc. I'm not saying that people who attend colleges or universities can't have expertise, but rather that these days such a piece of paper alone doesn't actually prove very much (most of all it proves that the person who has "earned" the degree is good at jumping through hoops).

I will get to the point -- really, I will. Ultimately, it leads (yet again) to the Wisdom of the Language, but there are some intermediate steps that I did not describe in that article which are crucial -- and these intermediate steps are what I see happening right now (and the discussion with Alex above is but one example of this).

People don't like to type. Fumbling around with a keyboard, pressing buttons, clicking links -- it's all a major PITA. Instead people prefer to communicate "directly" with other people. Of course there is usually some medium between people who are communicating: a computer, a telephone, or simply air, etc. Even language itself is a technology, a medium: 2 people cannot be in the same head -- except maybe in science fiction.

Let me expand on this notion of language a little more. A smile is a message, as is a frown. These are very basic and very universal memes. Then there are languages such as English, Spanish, German, French, Italian, Chinese, Arabic, etc. These are not universal -- they create communities... communities of people who understand and speak -- who use -- those languages. Then there are what I would call meta languages -- such as the languages of graphics and drafting, or even comics and illustration, film and movies, story telling, etc. Although each of these languages have some common threads, usually the grammars for these languages are not well-defined. Then there are formalized / abstract languages -- such as mathematics or "computer" languages such as C, SQL, HTML and so on. The language and syntax requirements of search engines -- insofar as they are "algorithmic" -- fall at this end of the spectrum, but insofar as they also have to do with so-called "natural language" there is also an element of the other end of the spectrum involved (but only to the extent that people actually use those so-called natural languages in written form -- and one should really not be so naive to overestimate the extent to which written literacy is spread throughout the population).

Now there are currently 2 significant developments happening on the web which impact information retrieval (that's the more formal name for "search") in a fundamental way. First, technological advances are making it easier for what I would call "non-literate" people (i.e., those who simply don't read / write all that much text [and by "text" I mean this kind of stuff I'm creating here -- which is "machine readable", has a high information density, a high signal-to-noise ratio, and so on]) -- for people who simply do not use written language / text writing all that much -- to create messages in other formats (such as pictures, video, audio, etc.)... but most of these formats (since they're not codified in a formalized language) are not machine readable. The result is that there is a rising flood of meaningless data spreading across the internet. Luckily, this flood is presently mostly confined to web sites that contain this mass of meaningless data, and here the authors of such audios, videos, etc. are allowed to add text (so-called "tags") to it... but the signal-to-noise ratio of this data is overall quite poor (or even nonexistent).

The other development happening is the previously mentioned trend to build communities of experts -- and this is first of all happening via what are called "social networks". For example: twitter.com is to some degree a social network who produce commercial twitter (i.e., this community utilizes the wisdom of the language) or facebook is a community of people who want to be listed (with their "face" in a "book") as being a member of the global online community...

~ Intermission ~

What I see happening is not so much about society, but rather it's about the way we're transitioning from a highly formalized, generalized language (the geeks here may think of something a "low level" language such as "assembler" -- in which programs are long lists of instructions, with each individual formulated as an individual, detailed and very specific instruction) to a more specific languages which are used for very specific purposes. This makes it easier on the user, because users are thereby not required to formulate complex expressions for the specific tasks / problems they are trying to find answers for (e.g. "Steve Martin Jerk phonebook video") but rather they can utilize a more specific (e.g. "video") search engine to make the search query less complex.

This development has been going on for years already, but it is still only at the beginning. My last tweet to Alex (see the top of this blog entry), in which I say "foursquare has found Alex" basically means that for people who have registered their mobile device with foursquare, the machine is actually monitoring their present location at all times -- and if the location matches a location stored in the database, foursquare will document and return that result. In other words: simply carrying a mobile device is now becoming a kind of non-textual literacy... all you have to know is when to turn the device on or off, and then the device publishes your whereabouts and others could use the foursquare search functionality (if they wanted to) to find out who has signed the virtual guestbook simply by carrying along the mobile device and making sure it's turned on to "automatically" send out a signal when at a favorite location.

I have often referred to this trend as a movement away from "one-size fits-all" search engines to more specific (sometimes called "vertical") search engines. Presently, many computer / device manufacturers are very keen on developing applications ("apps") that will only work on these devices. Many traditional publishers have high hopes for using such "walled gardens" in order to continue to offer their "exclusive" information for a fee. My guess is that this will be the "last hurrah" of traditional publishing before being consumed in a valley of ashes littered with specialized gizmos.

Instead, more and more vertical engines -- such as twitter, facebook, etc. -- will be developed and each of these community sites will have become more and more specific about what type of information is being transmitted -- such as short messages (twitter) or a guestbook / contact listing service (facebook). As more and more such services are developed -- a service for tracking mobile devices, a service for documenting credit card transactions, a service for recommending news articles, a service for sharing photos, etc. -- the search query will increasingly move away from generalized databases that store "all the world's information" (such as the "one-size fits-all" approach at Google or Bing) to a plethora of specific databases for specific purposes. In many cases, the search query will be nothing more than the domain name itself -- so if users want to twitter and/or find out what other people are twittering about, they simply enter twitter.com (rather than using a search engine to search across the entire web for "news").

I predict that this increasing plethora of web sites will lead to a world in which those sites using the wisdom of the language will prevail: sites called "weather" will offer weather information, sites called "shopping" will offer shopping information, sites called "twitter" will offer twitter information, and so on -- in my opinion, that is simply natural.