ScraperWiki

From Data-gov Wiki

Jump to: navigation, search
Infobox (Application) edit with form
  • name: ScraperWiki

  • description: ScraperWiki is a web platform for collecting and publishing public data ...
  • homepage: http://blog.scraperwiki.com/
  • modified: 2010-8-11


Description

ScraperWiki is a web platform that allows users to author and edit python code to scrape web pages.

As an example, check out https://www.og.decc.gov.uk/pls/wons/wdep0100.qryWell. It offers a hideous query interface to select a set of "Quadrants" and "Blocks". Clicking submit will show a table of oil wells. Clicking on a well number will show its information.

The existing web page is only (marginally) useful to a human, and inaccessible to a machine. Why can't we just get all of the data?

Enter ScraperWiki. It grabbed the data about oil wells near the UK, and provides it at http://scraperwiki.com/scrapers/show/uk-offshore-oil-wells/data/.

How did the data get there? http://scraperwiki.com/scrapers/show/uk-offshore-oil-wells/history/ shows when ScraperWiki executed the scraper code that a ScraperWiki user created. ScraperWiki provides a code editing interface, so users can author and edit scraper code directly on the site.

The tutorial page lists a wide range of parsing utilities difficult formats such as Excel and PDF.

Facts about ScraperWikiRDF feed
Dcterms:descriptionScraperWiki is a web platform for collecting and publishing public data ...
Dcterms:modified2010-8-11
Foaf:homepagehttp://blog.scraperwiki.com/  +
Foaf:nameScraperWiki
Skos:altLabelScraperWiki  +, scraperwiki  +, and SCRAPERWIKI  +
Personal tools
internal pages