Scraper site
From Wikipedia, the free encyclopedia
A scraper site is a website that copies all of its content from other websites using web scraping.[1] No part of a scraper site is original.[citation needed] A search engine is not a scraper site:[citation needed] sites such as Yahoo and Google gather content from other websites and index it so that the index can be searched with keywords. Search engines then display snippets of the original site content in response to a user's search.
In the last few years, and due to the advent of the Google Adsense web advertising program, scraper sites have proliferated at an amazing rate for spamming search engines.[1] Open content sites such as Wikipedia are a common source of material for scraper sites.
Contents |
[edit] Made for AdSense
Some scraper sites are created for monetizing the site using advertising programs such as Google AdSense.[1] In such case, they are called Made for AdSense sites or MFA. This is also a derogatory term used to refer to websites that have no redeeming value except to get web visitors to the website for the sole purpose of clicking on advertisements.
Made for AdSense sites are considered sites that are spamming search engines and diluting the search results by providing surfers with less-than-satisfactory search results. The scraped content is considered redundant to that which would be shown by the search engine under normal circumstances had no MFA website been found in the listings.
These types of websites are being eliminated in various search engines and sometimes show up as supplemental results instead of being displayed in the initial search results.
Some sites engage in "Adsense Arbitrage"--they will buy Adwords spots for lower cost search terms and bring the visitor to a page that is mostly Adsense. The arbitrager then makes the difference between the low value clicks he bought from AdWords and the higher value clicks generated by this traffic on his MFA sites. In 2007, Google cracked down on this business model by closing the accounts of many arbitragers[2]. Another way Google and Yahoo are combating the proliferation of arbitrage are through quality scoring systems. For example, in Google's case, Adwords penalizes "low quality" advertiser pages by placing a higher per click value to its campaigns[3]. This effectively evaporates the arbitrager's profit margin.
[edit] Legality
Scraper sites may violate copyright law. Even taking content from an open content site can be a copyright violation, if done in a way which does not respect the license. For instance, the GNU Free Documentation License (GFDL) and Creative Commons ShareAlike (CC-BY-SA) licenses require that a republisher inform readers of the license conditions, and give credit to the original author.
[edit] Techniques
 |
This section does not cite any references or sources. Please help improve this article by adding citations to reliable sources (ideally, using inline citations). Unsourced material may be challenged and removed. (November 2007) |
 |
This section may contain original research or unverified claims. Please improve the article by adding references. See the talk page for details. (September 2007) |
Many scrapers will pull snippets and text from websites that rank high for keywords they have targeted. This way they hope to rank highly in the SERPs (Search Engine Results Pages). RSS feeds are vulnerable to scrapers.
Some scraper sites consist of advertisements and paragraphs of words randomly selected from a dictionary. Often a visitor will click on a pay-per-click advertisement because it is the only comprehensible text on the page. Operators of these scraper sites gain financially from these clicks. Ad networks such as Google AdSense claim to be constantly working to remove these sites from their programs, although there is an active polemic about this since these networks benefit directly from the clicks generated at these kind of sites. From the advertiser's point of view, the networks don't seem to be making enough effort to stop this problem.
Scrapers tend to be associated with link farms and are sometimes perceived as the same thing, when multiple scrapers link to the same target site.[1] A frequent target victim site might be accused of link-farm participation, due to the artificial pattern of incoming links to a victim website, linked from multiple scraper sites.[1]
[edit] Wikipedia scrapers
The sites that follow offer Wikipedia content some of them adding advertising. Compare a sample Wikipedia page about Haskell with the following sites and follow their links:
[edit] References
- ^ a b c d e "Scraper sites, spam and Google (tactics/motives)". Googlerankings.com diagnostics. 2007. http://diagnostics.googlerankings.com/scraper-sites.html. Retrieved on 2008-01-02.
- ^ See: http://www.jensense.com/2007/05/18/google-adsense-disabling-arbitrage-publisher-accounts-as-of-june-1st/
- ^ See for example: http://www.seobook.com/yahoo-kills-ppc-arbitrage
[edit] See also
|
|||||||||||||||||||
