Advanced Metasearch Engine Technology

Among the search tools currently on the Web, search engines are the most well known thanks to the popularity of major search engines such as Google and Yahoo!. While extremely successful, these major search engines do have serious limitations. This book introduces large-scale metasearch engine technology, which has the potential to overcome the limitations of the major search engines. Essentially, a metasearch engine is a search system that supports unified access to multiple existing search engines by passing the queries it receives to its component search engines and aggregating the returned results into a single ranked list. A large-scale metasearch engine has thousands or more component search engines. While metasearch engines were initially motivated by their ability to combine the search coverage of multiple search engines, there are also other benefits such as the potential to obtain better and fresher results and to reach the Deep Web. The following major components of large-scale metasearch engines will be discussed in detail in this book: search engine selection, search engine incorporation, and result merging. Highly scalable and automated solutions for these components are emphasized. The authors make a strong case for the viability of the large-scale metasearch engine technology as a competitive technology for Web search. Table of Contents: Introduction / Metasearch Engine Architecture / Search Engine Selection / Search Engine Incorporation / Result Merging / Summary and Future Research

Introduction.- Metasearch Engine Architecture.- Search Engine Selection.- Search Engine Incorporation.- Result Merging.- Summary and Future Research.

Weiyi Meng is currently a professor in the Department of Computer Science of the State University of New York at Binghamton. He received his Ph.D. in Computer Science from University of Illinois at Chicago in 1992. At the same year, he joined his current department as a faculty member. He is a co-author of the book Principles of Database Query Processing for Advanced Applications. He has published over 100 papers. He has served as general chair and program chair of several international conferences and as program committee members of over 50 international conferences. He is on the editorial board of the World Wide Web Journal and a member of the Steering Committee of the WAIM conference series. In recent years, his research has focused on metasearch engine, Web data integration, Internet-based information retrieval, information extraction and sentiment analysis. He has done pioneering work in large-scale metasearch engines. He is a co-founder of an Internet company (Webscalers) and serves as its president. His company has developed the world’s largest news metasearch engine AllInOneNews. Clement T. Yu is a professor of computer science at the University of Illinois at Chicago. His research interests include multimedia information retrieval, metasearch engine, database management, and applications to healthcare. He has published more than 200 papers in these areas, and he is a coauthor of the book Principles of Database Query Processing for Advanced Applications. Dr. Yu served as chairman of the ACM SIGIR and has extensive experience as a consultant in the fields of query processing in distributed and heterogeneous environments, including document retrieval. He was an advisory committee member for the National Science Foundation and was on the editorial boards of IEEE Transactions on Knowledge and Data Engineering, the Journal of Distributed and Parallel Databases, the International Journal of Software Engineering and Knowledge Engineering, and WWW: Internet and WebInformation Systems. He also served as the General Chair of the ACM SIGMOD Conference and Program Committee Chair of the ACM SIGIR Conference. He is a co-founder of two Internet companies, Webscalers and PharmIR.