Introduction to Text Analysis  

• Reporters' Lab Reviews: Scraping Tools
• wget
• Nvivo (with NCapture add-in)
• School of Data: Scraping Resources
• School of Data: Course on Web Scraping
• Table of screen scrapers (Google spreadsheet)
• On Scrapers
• ScraperWiki
• Web Scraping Using PHP and jQuery
• dapper
• Pipes

"Text analysis" is a broad term covering various processes by which text and natural language documents can be modified so that they can be organized and described.

This guide collects resources for several phases of the text analysis process, including text collection, text parsing and cleaning, text summary and analysis methods, and text visualization.

Overviews/summaries

• Ted Underwood – Where to start with text mining
• Tooling Up for Digital Humanities – Text Analysis
• Ryan Shaw – Text Mining

• Native digital text
• Email
• (Thunderbird extension, MUSE*)
• HTML
• RSS feeds
• Sample specific services:
• Twitter
• Wikipedia
• Data Liberation Front
• New York Times API
• Tutorials for data collection from various services
• Digitized
• Internet Archive
• Project Gutenberg
• Google Books
• Hathi Trust
• JSTOR Data for Research* (with Early Journal Content bundle)
• PubMed Open Access Subset
• Monk Workbench*
• Document Cloud*
• Open American National Corpus (collection of American English from various sources)
• WordHoard* (tagged literary texts)

* - also has some processing/analysis capabilities