LinkedIn and MySpace Bet On Lucene Open Source Search
by Sam Dean - Nov. 25, 2008Comments (1)- Related Blog Posts
- ApacheCon Series: Sourcesense's CEO Weighs In On Open Development
- Google Provides Closure: Releases Open Source JavaScript Optimizer
- Live from ApacheCon: Subversion Joins ASF
- A Personal Account On Getting Deeply Involved With Apache
- The Apache Software Foundation's President Dissects the "Apache Way"
As Matt Asay reports, the LinkedIn social network has just upgraded its people search features, based on the open source search project Lucene. And, as Ars Technica discussed in June, MySpace recently updated its search features using Lucene as well --including making contributions to Lucene. OStatic's own search features are powered by Lucene, and many leaders of web sites, blogs and web applications should look into this extremely powerful open source information retrieval engine.
MySpace uses Lucene to allow people to search within separate categories, such as Music and MySpaceTV. OStatic also uses it to to allow category-based searches. You can, for example, search only within this blog, or toggle to "Software" and search OStatic's database of open source projects.
Lucene is not focused on crawling and HTML parsing features, as are often found in search engines aimed at web-wide searches. Its specialty is searching content within a site, and it is very fast at indexing and searching large bodies of site content. It also nimbly handles searches within content of various file formats, including PDFs, HTML, Microsoft Word documents, and more.
Lucene has also given rise to many spin-off search projects, as well as a healthy system of extensions. It is sponsored by Apache. The Apache project Nutch is based on Lucene and provides many types of web-wide search capabilities. Apache's Solr does server-based search based on Lucene.
The single-site search that Lucene specializes in is an Achilles heel for many sites on the web, including many recognized blogs and commercial sites. Clearly, LinkedIn and MySpace feel confident in Lucene for searching the giant volumes of content that it will index for them. Many web properties should look into this open source search powerhouse as well. For much more on Lucene, see our detailed discussion here.
