Seminar report on search engines

Search engines
According to Wikipedia, a web search engine is a tool designed to search for information on the World Wide Web. We believe most of us are quite familiar with it. Actually, once we connect to the Internet, mostly we will use a search engine (like Google) at first. As for most of us, we have already added Google, Baidu or other search engines to our home pages. However, most of us just know little about how Search engine works so that the lecture by Aaron Tan really makes us get rid of confusion.
1. Web phenomenon
During pre-2000, the number of websites was growing at an exponential rate which resulted in the high value E-commerce systems and the development of technology. The period can be divided into 3 growth stages: 1991-1997: Explosive growth, at a rate of 850% per year; 1998-2001: Rapid growth, at a rate of 150% per year; 2002-2006: Maturing growth, at a rate of 25% per year. During post-2000, the rate of growth is slowing down and the trend towards other media like Youtube, Myspace and Social media like blogs, wikis is becoming the main stream.
2. Web Statistics
At the same time, the amount of information on the web is growing at an exponential rate. Millions of pages are added per month. Billions of unique pages available will double in a year. Moreover, the freshness of updates also becomes the competing goal among search engines: the fresher, the hotter. The most frequently linked sites are changing a lot and we can never guess who will be the next winner.
3. Search Engines
Now, we come to the main point: Search Engines. 59m American adults use search engines on a day, up from 38m a year before – from 30% to 41%, (Pew Internet Project, 2005) a big rising. From iProspec Survey, 2004, we can get it that more than 70% of responders use search engines on the Internet almost every day, and even 20% of them will use it many times a day. We can conclude that search engines are likely to have become part of our lives. Therefore, we should learn more about how the important part of the Internet works. To understand the anatomy of a Search Engine, we have to know Web Crawler. A Web crawler is a computer program that browses the World Wide Web in a methodical, automated manner. (Wikipedia, Web crawler) Next, we learn about problems with Older Search Engines: too much false hits, the old approach which makes it harder to find what we need and always together with a list of rubbish. To make the older ones improve, we need to find out the way to sort the pages by its content more effectively and select what are more important. The importance of each page should be decided by what other pages’ comments on this page and the number of pages pointing to each page but the problem is that others can generate many dummy pages that point to a page. To rank pages, we need apply a more practical method: for example, if there are just three websites and the inlinks among the three, to rank one page, calculate the other two’s inlinks. Therefore, a page becomes important if many important pages link to it. (The method is so practical that we cannot write it properly. If you are interested in it, you can go to the link:
http://www.comp.nus.edu.sg/~fmc1202)


Reflections:
Even though the search engines have already improved a lot, there are still some problems. Instead of the technique, anthropogenic factors have become one of the most serious problems which hamper the healthy development of the Internet. We had better to find the solution to improve the psychological quality of Netizens, rather than just develop the technology.

On 9 October, everyone in the seminar group presented a topic related to Search Engine. Topics such as cost per impression/cost per click, click spam, web crawlers and Search Engine Optimization (SEO) are covered. We find that the SEO presentation is the most interesting and fulfilling as Chunsiong explained clearly how SEO comes about and the workings of SEO. After his presentation, we have learned more about how we can publicize our web sites and hopefully, increase our page rank. In fact, we have never known that link farming is not allowed by Google as a form of SEO until Chunsiong enlighten us during his presentation. However, the weirdest thing is that my web sites have never been blacklisted by Google, although weI use a similar technique as link farming to publicize them. Maybe there are other types of link farming techniques which Google allows? Hmm… What do you think?
In addition to the SEO presentation, we found the presentation about the different types of search engines useful. Thanks to the group’s presentation, we now knew that the name “Baidu” has its origins from the Song Dynasty. It is fascinating to see how the name of a famous search engine comes from a poem back in China’s history.
During the presentation, Zhu Li, Byron, Geck Keat and Li Hua have presented on the topic, Click Spam. We never knew that click spam originated from games until we have started working on this topic.
Overall, we felt that we have benefited from the 2 seminar sessions on Search Engine. Many thanks to my seminar group mates and Dr Aaron Tan for sharing with us what they know about Search Engines.

Posted by Zhu Li and Li Hua