OStatic

Last summer, we reported on Microsoft's acquisition (reportedly for $100 million) of Powerset, which specializes in semantic search based on the open source, cluster-based software framework Hadoop. This acquisition of an open source-centric search company was more strategic than many people realize. Hadoop also underlies Yahoo!'s search engine with its ability to search large data sets quickly, and the acquisition of Powerset may have played a key part in how Microsoft decided to give up its effort to acquire Yahoo!

Of course, Microsoft's big search engine news of the week is Bing, which I've found to have both strengths and weaknesses. Surprisingly, as The Register reports, ?Powerset's technology plays only a small part in how Bing works, but what it does in Bing is open source-driven, and interesting.

eWeek reported this morning that Jimmy Wales, founder of Wikipedia, has closed up shop for Wikia, his community-driven search engine. Wikia Search launched in January 2008, and was reported to be the fifth-fastest growing community destination by Nielsen Online in February 2009. The Nielsen statistics seem to have incorporated data from other sites in the Wiki line, however, and reports say that Wikia Search was only drawing 10,000 unique visitors per month.

Wikia Search was a very different animal than traditional search engines, as it substituted search results provided by algorithms for those chosen and ranked by community members. It's certainly an interesting idea -- it's an idea that might actually work exceedingly well under the right circumstances. For now, however, Wales has opted to put work on community-based search on hold and focus his team's efforts elsewhere. He also holds out hope that community search is workable, and vows that when it takes hold, he'll be there in some capacity, actively contributing or simply cheering on the effort.

Last month's Technology Review featured a piece on semantic computing. Semantic technology -- whether it's applied on the web or the desktop -- seems almost impossibly complex, as it tries to bring some very human traits of relating and connecting information to a machine environment. The artificial intelligence field, relatively speaking, is in its infancy, and since the human brain is largely an indistinguishable mix of biology and culture, it would seem semantic technology would be confined to psychology departments and computer science labs.

That isn't the case, of course. And when you consider that semantic technology deals with computers and people, and that any technology or study ultimately benefits from larger participant pools, it's little wonder that the Nepomuk project is open source and now even comes integrated with the KDE desktop.

If you want to search for a piece of text on the Web, you probably turn to Google or a similar search engine. But how can you integrate search into your Web site?

Lucene is a high-quality Apache-sponsored engine for indexing and searching documents. While that would be enough for most people, a host of add-ons and complementary open source products make Lucene an even better choice when looking for a search engine.

We've recently written about potentially far-reaching moves by the U.S. government to switch significant parts of its internal software infrastructure to open source. First, the news came out that Whitehouse.gov is now based on the open source Drupal content management system (which OStatic runs on too), then the U.S. Department of Defense announced its plans to move to open source software components and platforms.

When the news broke about Whitehouse.gov and Drupal (and the Obama administration has indicated intent to run other government sites with Drupal), my first thought was that Acquia, which provides commercial support for Drupal, might see some valuable support contracts from the government. InfoWorld's Savio Rodrigues takes the idea one step further, though, and I'm inclined to agree with him.