Social media platforms like Twitter offer an important window into contemporary experience and social movements, but archiving tweets for preservation poses challenges that are both practical and ethical. Practically, how are archivists to sort through and preserve social media content? Ethically, how can archivists navigate issues relating to the content owners’ privacy, consent, and control? Documenting the Now addresses these complex challenges. The community-driven project is creating ethical standards and tools for the collection and preservation of significant social media content.
“Documenting the Now is aimed at accomplishing two different, but deeply interrelated goals,” Co-investigator Ed Summers explains in an article introducing the project. “The first is to develop an open source Web app called DocNow that will allow researchers and archivists to easily collect, analyze and preserve Twitter messages and the Web resources they reference. The second is to cultivate a much needed conversation between scholars, archivists, journalists and human rights activists around the effective and ethical use of social media content.”
The project began in the aftermath of the police killing of 18-year-old Michael Brown in Ferguson, Missouri on August 9, 2014. Twitter played a key role in raising awareness and mobilizing response. It was clear that this material needed to be preserved, but how?
The Practical Challenge
It has proven difficult to apply traditional archival processes to the vast amount of social media content available, co-investigator Bergis Jules explains.
“One thing we’re finding is that archivists are having a hard time making sense volume of content [on social media],” Jules says. “I think people have just resorted to collecting massive amounts of data because they don’t have tools to help them make sense of the content. The normal process of archiving [begins with] appraisal and selection, making decisions about what is taken to the archive. But it’s hard to transfer that over to social media content.”
Ethical Issues
Though social media content is potentially available to all online, content creators do not necessarily intend their posts to be preserved indefinitely in an archive. This is particularly the case for media produced by activists and others in times of crisis, when content owners may be especially interested in issues of privacy and access.
When collecting material, archivists typically require deeds of gift to receive material, Jules explains. “We have practices that we put around consent with our deeds of gifts that we could easily transfer to this type of material, we just need to rethink how we do it,” Jules continues: “What does a deed of gift look like for tweets or Facebook posts? How do we get that out to 500 people? Is there an automated process you could create to make contact with a large number of people? We’re rethinking some of the processes we already have.”
Twitter enables users to delete posts at any time. For Jules and the Documenting the Now team, any archiving solution needs to accommodate this and to respect the rights of content owners not to be archived.
DocNow is a Community and a Tool
With these practical and ethical challenges in mind, Documenting the Now has convened diverse activist, journalist, and academic stakeholders in person and online via a Slack channel and Twitter to discuss possible ways forward. These ongoing conversations are helping to clarify ethical standards and are producing a number of tools for archivists, including DocNow.
Currently in development, DocNow is a cloud-based open-source app that will enable archivists to assess social media materials for collection. Through the app, users are able to search Twitter along a range of axes. The app retrieves a manageable sample of 1,000 tweets and breaks them down by their users, hashtags, links, and media, offering a number of entry points for analyzing Twitter conversations.
With DocNow, archivists can save searches and download their data in a way that honors content owners’ rights to opt out and to delete their tweets. The app saves only unique tweet identification numbers, not the text of the tweets themselves. The text of tweets is saved in a central database that is updated regularly. If a content owner deletes a tweet, it is deleted from the central database and the tweet will not be accessible in the archive.
The project is ongoing. To learn more and to join the conversation, visit the project website.