Searching the deep web

Search engines and directories are only the first steps for a good media professional searching for information on the Internet. The fact is that if you rely only on search engines, you are not going beyond what your audiences can do on their own. Most children today know how to search for information using Google.

What you get as information on web pages may not be very reliable or accurate. It is better to see what scientists themselves are saying. As we noted earlier, scientists report their findings in journals. The details of publications in journals are available in databases, which are not accessed by search engines.

Search engines do not really search the World Wide Web directly. They search a database of web pages that it has harvested periodically. Search engine databases are selected and built by computer robot programs called spiders. These “crawl” the web, finding pages for inclusion in their database by following the links in the pages they already have in their database. If a web page is never linked from any other page, search engine spiders cannot find it. The only way a brand new page can get into a search engine is for other pages to link to it, or for a human to submit its URL for inclusion.

After spiders find pages, they pass them on to another computer program for “indexing”. This program identifies the text, links and other content in the page and stores it in the search engine database’s files so that the database can be searched by keywords. The page will be found if your search matches its content.

Many web pages are excluded from most search engines by policy. The contents of most of the searchable databases mounted on the web, such as library catalogues and article databases, are excluded because search engine spiders cannot access them. All this material is referred to as the “invisible web” or “deep web”.

Databases

There are different types of databases: bibliographic databases from where you can get references to reading materials, abstracts databases from where you can get abstracts of scientific papers and full text databases from where you can get the full text of the paper you want to read.

Google Scholar (http://scholar.google.com/) leads you to more academic materials – books, abstracts and papers from academic journals – and provides more in-depth information than Google.



http://www.scirus.com is another website where you can search authors, journals and scientific topics. Advanced search options for Boolean combinations of terms and preferences settings that allow only journals in specific subject areas or to include webpages, patents and so on are also provided on this website. Again, you may get only a few lines of the abstract. The search could lead you to Science Direct or other publishers and, if you are lucky, you may even get the full text of the paper that you need. The site also provides the latest science news from New Scientist, with relevant links.Abstracts of papers from nearly 7,000 journals can be found on http://www. sciencedirect.com/. Some of these journals allow access to the full text of the papers. But to access most, payments may have to be made. Let this not deter you. Reading only the abstracts of the most recent papers will also giveyou an idea about the status of the scientific research in a particular field and prepare you to interview scientists.

http://www.go3r.org/ is organized in a different manner. It provides the beginnings of semantic search. It is more oriented to medical literature and its ideal seems to be related to reducing animal experimentation. The possibilities of search include criteria like who, when and where besides what. As you search, you will find that the structure of information available changes to help you narrow down (or expand) what you want/need.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The Online Journal Search Engine (http://www.ojose.com/) is a website that helps you to search many databases and scientific publishers. You can search databases such as Scopus, Inist and Medline from here.

Quite often, you may find that you end up seeing only the abstract and need to pay up before you get to the research article. For producers from the developing world, this can be very frustrating. Fortunately, there are many journals that are more open. For a list of journals that you can access easily, see the directory of open-access journals at http://www.doaj.org/ doaj?func=byTitle&alpha=J More journals are expected to open up access to their contents. Some have already made their archives (all but the more recent issues) available for search.

http://www.scienceresearch.com/search/ searches databases and organizes the output in the same manner as go3r. The data available is wider than offered by go3r.

The Online Journal Search Engine (http://www.ojose.com/) is a website that helps you to search many databases and scientific publishers. You can search databases such as Scopus, Inist and Medline from here.

Quite often, you may find that you end up seeing only the abstract and need to pay up before you get to the research article. For producers from the developing world, this can be very frustrating. Fortunately, there are many journals that are more open. For a list of journals that you can access easily, see the directory of open-access journals at http://www.doaj.org/ doaj?func=byTitle&alpha=J More journals are expected to open up access to their contents. Some have already made their archives (all but the more recent issues) available for search.