To follow what's happening in the real world, you need real-time search. Google doesn't have it (yet). Neither do Bing nor Yahoo. But a number--a large, growing number --of new search services scan Twitter and other sites in nearly real time, and will find for you the absolute latest update from the real-time social Web. I looked at more than two dozen products that search Twitter (and other sources) to find the best tools for uncovering the beating pulse of whatever topic you may be interested in.
To set the stage, let's look at Twitter's own search service. It's not bad. It's simple to use, presents results in a very clear way, and does a good job of balancing users' needs to see results in real-time with their inability to actually read the stream of tweets flying by: It updates a counter at the top of the page as new tweets that match your search come in, but you have to hit a link to actually see the updates.
Twitter Search also has an extremely good advanced query builder, and you can subscribe to search results via RSS.
The downside to Twitter search is that it's dumb, at least for now. Results are only sorted chronologically. There's no algorithm to give you the most read, most authoritative, most linked-to, or most re-tweeted items. And it only searches Twitter.
There are three services that do a much better job than Twitter Search, and several others are also worth looking at for special cases. Here are the top three:
Best three real-time search services
Twazzup: As I've said before, Twazzup is a very useful and flexible Twitter search tool that provides almost all of what Twitter Search does itself, plus a few columns of algorithmically generated results such as most influential tweeters on a topic, related photos, and a great list of keywords based on your search results that you can use to refine your query.
Unfortunately, the Twazzup results page can be overwhelming to read, and it only searches Twitter (excusable for Twitter Search, not for other services).
Collecta (story) is a new real-time search service that scans Twitter and many other sources, including mainstream media sites (CNN, etc.) and blogs. Its best features are its super-clean and simple interface, and the fact that it lets you run multiple searches at once, so it's great for monitoring a bunch of topics during the day.
Collecta lacks an advanced query builder and I've found it's sometimes slow to get cooking on a query, but it is shaping up to be an extremely useful tool.
Scoopler is another deceptively simple real-time search product that, like Twazzup, combines raw chronological results with a column of results ranked by popularity. This service doesn't have access to the full Twitter "fire hose" of data, but it does use additional methods to try to pick up more content than the Twitter Search API provides--especially items that contain links. Scoopler also lets you keep three search queries pegged to your page for quick access (although it doesn't run them in the background like Collecta does).
Scoopler also scans Digg, Delicious, Flickr, and Indentica. It earns a spot on my top-three list due to its clear combination of raw and ranked results. (This paragraph has been updated from the original version of the story)
Other good real-time search tools
There are several other real-time search products. Some are highly specialized, or have weaknesses that offset strengths, but many are still very useful for specific needs. For example:
OneRiot only searches items that have links in them. This provides a good built-in filter for finding a lot of good info on the Web, but OneRiot won't find text-only tweets. Save that limitation, it's a very strong service, with options to switch between ranked and raw results, a special tab for shared video links, and a nice clean design. OneRiot searches Twitter, Digg, YouTube, and other services.
Tweetmeme is a stellar tool for discovering what's hot on Twitter, since it's all about the act of re-tweeting. It's not a great search engine, though. See also DailyRT, which has a better search function, including the capability to search only within the network of people you follow.
Monitter gives you a multi-column view of real-time Twitter search results (with no ranking), which makes it useful for monitoring several queries at once. Its special trick is that you can easily filter results by the location that Twitter users report in their profiles.
Topsy does a very good job of helping you find the most influential people on the topic you're searching for. It ranks results by links to the items in question, which is useful if not exactly real-time. There's no option for "live" results.
CrowdEye gives you stats on your query, and popular links related to it, but the results page is static. It's useful if you want to see how a search query is trending over time.
Keep an eye on Itpints, Twitority, Twitalyzer Search, Twitmatic, and Yauba, which gave me uneven experiences but which could improve quickly. Some other engines search Twitter, but aren't designed to provide real-time results; see Tweefind, for example.
As far as the big old-school search companies like Google, Yahoo, and Microsoft go: Don't count them out forever. I am 100 percent sure that they are all looking for ways to set their search offerings apart and have their eyes on some of these companies as acquisition targets. Why do you think there are so many start-ups chasing this problem, after all?