On the development of a suite of tools for the analysis of Twitter discourse

Abstract

Social media has become a ubiquitous avenue of natural language use that represents an enormous amount of data, which has in the last decade been used in an increasing amount of linguistic research in such topics as regional variation, stylistic variation, and identity and community construction. For a number of technical reasons, however, the collection and analysis of Twitter data for research purposes is currently prohibitively difficult, especially for studies with a particular interest in Twitter as a site of discourse creation. Existing collection tools require almost all require programming skills, and existing analytical tools are not equipped to handle the metadata that Twitter data includes, much less to leverage that metadata to highlight linguistic trends or to make large datasets navigable. In an attempt to address this problem, I have developed a web-based set of tools, called TWIG (TWItter Getter), for collecting and analyzing Twitter data. This document begins with a discussion of the basic mechanics of Twitter and its nature as a discourse context and then details the need for specialized Twitter data collection and analysis tools that focus on the needs of researchers interested in Twitter generally, and those interested in discourse specifically. The remainder of the document describes TWIG and demonstrates its use in two sample studies—one an analysis of a community’s construction of identity and values on twitter, and the second a usage-based analysis of semantic change in the construction really do be.