News Corp. (Murdoch) comienza a bloquear agregadores que utilizan sus contenidos

Me entero a través de 233grados que Murdoch comenzó a probar suerte para mejorar el negocio de sus medios con la política que había anunciado hace unas semanas: bloquear el acceso a terceros que utilizan los contenidos de sus medios para atraer tráfico.

La metodología que utilizó es bastante simple: agregar en el archivo robots.txt, que indica a los sistemas automáticos qué contenido quiere dejar fuera de indexación cada sitio, una prohibición global. Por el momento, solo alcanza a algunos agregadores de noticias: NewsNow, Omgili, WebVac, WebZip, psbot y ia_archiver.

Este es el archivo ROBOTS.TXT que tiene Times Online hoy, 8 de enero de 2010 a las 17:45 GMT

##ACAP version=1.0

#Robots.txt File
#Version: 0.8
#Last updated: 04/01/2010
#Site contents Copyright Times Newspapers Ltd
#Please note our terms and conditions http://www.timesonline.co.uk/section/0,,497,00.html
#Spidering is not allowed by our terms and conditions
#Authorised spidering is subject to permission
#For authorisation please contact us - see http://www.nisyndication.com/about_us.html

# Globally Disallowed Sections: All Robots
User-agent: *
Disallow: /_gatools/
Disallow: /_nipd/
Disallow: /images/
Disallow: /img/
Disallow: /tol/Print.do
Disallow: /tol/sitesearch.do
Disallow: /tol/incomingFeeds
Disallow: /tol/arts_and_entertainment/commercial/vip_lounge
Disallow: /tol/viewArticle.arc
Disallow: /tol/tools_and_services/rss/times_mobile_content/
Disallow: /tol/tools_and_services/mobile_times/mobile_2/
Disallow: /tol/tools_and_services/mobile_times/mobile_3/
Disallow: /tol/tlskeywordsearch.tls

#Agent Specific Disallowed Sections
User-agent: NewsNow
Disallow: /

User-agent: Omgili
Disallow: /

User-agent: WebVac
Disallow: /

User-agent: WebZip
Disallow: /

User-agent: psbot
Disallow: /

User-agent: ia_archiver
Disallow: /

ACAP-crawler: *
ACAP-disallow-crawl: /_gatools/
ACAP-disallow-crawl: /_nipd/
ACAP-disallow-crawl: /images/
ACAP-disallow-crawl: /img/
ACAP-disallow-crawl: /tol/Print.do
ACAP-disallow-crawl: /tol/arts_and_entertainment/commercial/vip_lounge
ACAP-disallow-crawl: /tol/viewArticle.arc
ACAP-disallow-crawl: /tol/tools_and_services/rss/times_mobile_content/
ACAP-disallow-crawl: /tol/tools_and_services/mobile_times/mobile_2/
ACAP-disallow-crawl: /tol/tools_and_services/mobile_times/mobile_3/

ACAP-crawler: NewsNow
ACAP-crawler: Omgili
ACAP-crawler: WebVac
ACAP-crawler: WebZip
ACAP-crawler: psbot
ACAP-crawler: ia_archiver
ACAP-disallow-crawl: /
##ACAP version=1.0

#Robots.txt File
#Version: 0.8
#Last updated: 04/01/2010
#Site contents Copyright Times Newspapers Ltd
#Please note our terms and conditions http://www.timesonline.co.uk/section/0,,497,00.html
#Spidering is not allowed by our terms and conditions
#Authorised spidering is subject to permission
#For authorisation please contact us - see http://www.nisyndication.com/about_us.html

# Globally Disallowed Sections: All Robots
User-agent: *
Disallow: /_gatools/
Disallow: /_nipd/
Disallow: /images/
Disallow: /img/
Disallow: /tol/Print.do
Disallow: /tol/sitesearch.do
Disallow: /tol/incomingFeeds
Disallow: /tol/arts_and_entertainment/commercial/vip_lounge
Disallow: /tol/viewArticle.arc
Disallow: /tol/tools_and_services/rss/times_mobile_content/
Disallow: /tol/tools_and_services/mobile_times/mobile_2/
Disallow: /tol/tools_and_services/mobile_times/mobile_3/
Disallow: /tol/tlskeywordsearch.tls

#Agent Specific Disallowed Sections
User-agent: NewsNow
Disallow: /

User-agent: Omgili
Disallow: /

User-agent: WebVac
Disallow: /

User-agent: WebZip
Disallow: /

User-agent: psbot
Disallow: /

User-agent: ia_archiver
Disallow: /

ACAP-crawler: *
ACAP-disallow-crawl: /_gatools/
ACAP-disallow-crawl: /_nipd/
ACAP-disallow-crawl: /images/
ACAP-disallow-crawl: /img/
ACAP-disallow-crawl: /tol/Print.do
ACAP-disallow-crawl: /tol/arts_and_entertainment/commercial/vip_lounge
ACAP-disallow-crawl: /tol/viewArticle.arc
ACAP-disallow-crawl: /tol/tools_and_services/rss/times_mobile_content/
ACAP-disallow-crawl: /tol/tools_and_services/mobile_times/mobile_2/
ACAP-disallow-crawl: /tol/tools_and_services/mobile_times/mobile_3/

ACAP-crawler: NewsNow
ACAP-crawler: Omgili
ACAP-crawler: WebVac
ACAP-crawler: WebZip
ACAP-crawler: psbot
ACAP-crawler: ia_archiver
ACAP-disallow-crawl: /
(Visited 10 times, 1 visits today)