Me entero a través de 233grados que Murdoch comenzó a probar suerte para mejorar el negocio de sus medios con la política que había anunciado hace unas semanas: bloquear el acceso a terceros que utilizan los contenidos de sus medios para atraer tráfico.
La metodología que utilizó es bastante simple: agregar en el archivo robots.txt, que indica a los sistemas automáticos qué contenido quiere dejar fuera de indexación cada sitio, una prohibición global. Por el momento, solo alcanza a algunos agregadores de noticias: NewsNow, Omgili, WebVac, WebZip, psbot y ia_archiver.
Este es el archivo ROBOTS.TXT que tiene Times Online hoy, 8 de enero de 2010 a las 17:45 GMT
##ACAP version=1.0 #Robots.txt File #Version: 0.8 #Last updated: 04/01/2010 #Site contents Copyright Times Newspapers Ltd #Please note our terms and conditions http://www.timesonline.co.uk/section/0,,497,00.html #Spidering is not allowed by our terms and conditions #Authorised spidering is subject to permission #For authorisation please contact us - see http://www.nisyndication.com/about_us.html # Globally Disallowed Sections: All Robots User-agent: * Disallow: /_gatools/ Disallow: /_nipd/ Disallow: /images/ Disallow: /img/ Disallow: /tol/Print.do Disallow: /tol/sitesearch.do Disallow: /tol/incomingFeeds Disallow: /tol/arts_and_entertainment/commercial/vip_lounge Disallow: /tol/viewArticle.arc Disallow: /tol/tools_and_services/rss/times_mobile_content/ Disallow: /tol/tools_and_services/mobile_times/mobile_2/ Disallow: /tol/tools_and_services/mobile_times/mobile_3/ Disallow: /tol/tlskeywordsearch.tls #Agent Specific Disallowed Sections User-agent: NewsNow Disallow: / User-agent: Omgili Disallow: / User-agent: WebVac Disallow: / User-agent: WebZip Disallow: / User-agent: psbot Disallow: / User-agent: ia_archiver Disallow: / ACAP-crawler: * ACAP-disallow-crawl: /_gatools/ ACAP-disallow-crawl: /_nipd/ ACAP-disallow-crawl: /images/ ACAP-disallow-crawl: /img/ ACAP-disallow-crawl: /tol/Print.do ACAP-disallow-crawl: /tol/arts_and_entertainment/commercial/vip_lounge ACAP-disallow-crawl: /tol/viewArticle.arc ACAP-disallow-crawl: /tol/tools_and_services/rss/times_mobile_content/ ACAP-disallow-crawl: /tol/tools_and_services/mobile_times/mobile_2/ ACAP-disallow-crawl: /tol/tools_and_services/mobile_times/mobile_3/ ACAP-crawler: NewsNow ACAP-crawler: Omgili ACAP-crawler: WebVac ACAP-crawler: WebZip ACAP-crawler: psbot ACAP-crawler: ia_archiver ACAP-disallow-crawl: /
##ACAP version=1.0 #Robots.txt File #Version: 0.8 #Last updated: 04/01/2010 #Site contents Copyright Times Newspapers Ltd #Please note our terms and conditions http://www.timesonline.co.uk/section/0,,497,00.html #Spidering is not allowed by our terms and conditions #Authorised spidering is subject to permission #For authorisation please contact us - see http://www.nisyndication.com/about_us.html # Globally Disallowed Sections: All Robots User-agent: * Disallow: /_gatools/ Disallow: /_nipd/ Disallow: /images/ Disallow: /img/ Disallow: /tol/Print.do Disallow: /tol/sitesearch.do Disallow: /tol/incomingFeeds Disallow: /tol/arts_and_entertainment/commercial/vip_lounge Disallow: /tol/viewArticle.arc Disallow: /tol/tools_and_services/rss/times_mobile_content/ Disallow: /tol/tools_and_services/mobile_times/mobile_2/ Disallow: /tol/tools_and_services/mobile_times/mobile_3/ Disallow: /tol/tlskeywordsearch.tls #Agent Specific Disallowed Sections User-agent: NewsNow Disallow: / User-agent: Omgili Disallow: / User-agent: WebVac Disallow: / User-agent: WebZip Disallow: / User-agent: psbot Disallow: / User-agent: ia_archiver Disallow: / ACAP-crawler: * ACAP-disallow-crawl: /_gatools/ ACAP-disallow-crawl: /_nipd/ ACAP-disallow-crawl: /images/ ACAP-disallow-crawl: /img/ ACAP-disallow-crawl: /tol/Print.do ACAP-disallow-crawl: /tol/arts_and_entertainment/commercial/vip_lounge ACAP-disallow-crawl: /tol/viewArticle.arc ACAP-disallow-crawl: /tol/tools_and_services/rss/times_mobile_content/ ACAP-disallow-crawl: /tol/tools_and_services/mobile_times/mobile_2/ ACAP-disallow-crawl: /tol/tools_and_services/mobile_times/mobile_3/ ACAP-crawler: NewsNow ACAP-crawler: Omgili ACAP-crawler: WebVac ACAP-crawler: WebZip ACAP-crawler: psbot ACAP-crawler: ia_archiver ACAP-disallow-crawl: /
(Visited 15 times, 5 visits today)