How The COVID Tracking Project uses Contentful, Netlify and GatsbyJS to publish testing data

In the first weeks of March, Alex Madrigal, a staff writer at The Atlantic, realized that no single reliable source of data on the COVID-19 pandemic existed in the US. After dealing with limited PowerPoint slides and attending press conferences with unreliable data, he decided to work on a solution. The COVID Tracking Project, with the help of hundreds of volunteers collecting and verifying data, became a single site where anyone can access testing data from every US state and territory.

The team built a static site with Contentful, Netlify and Gatsby to publish the data. We spoke with Kevin Miller, the web design team lead, to learn more about the project and the complexities of running a critical service at such a fast pace.

Challenges of collecting and standardizing data from various sources

Data related to the COVID-19 pandemic isn’t clean, and it varies in form and quality from region to region. Although knowing the number of tests done, the number of positive and negative results, and the number of deaths and recoveries is critical — only some state health organizations provide full sets of data. Rather than scraping data, The COVID Tracking Project volunteers collect and verify data manually. 

The project attracted hundreds of volunteers from across the US. Web designers, developers, journalists and others are contributing to the project. They watch conferences to see what local news sources say, and follow county, city and hospital sources to verify their numbers.  

This process results in volunteers updating the site about 300 times per week and rebuilding pages dozens of times a day. Even established organizations don’t involve such a high number of contributors nor rate of change. Running such an operation requires more than an average static page.

Building a static site that lasts: GatsbyJS, Netlify and Contentful  

The creators of The COVID Tracking Project thought of it as a temporary solution at first. They believed a government agency would eventually catch up, or the need would disappear. Neither has happened. 
The team now thinks long term, and Miller used GatsbyJS, Netlify, and Contentful to build a reliable, user-friendly static site that scales. It’s efficient, retains historical data and lets people without tech experience publish updates.

The team uses Contentful to manage and update their site, although they don’t store the hard COVID-19 data in Contentful. The GatsbyJS static site generator handles large sets of data changes without formatting issues — a clear choice for the data-heavy project. The Netlify integration with Contentful enables the team to preview and develop pages together.

Static sites and Contentful save time and distribute agency

Miller had already worked with Contentful and appreciated its flexibility, ease of use and efficiency. Creating a blog content model, putting it in an environment and creating a static page preview through Netlify and Gatsby’s cloud functionality only took a few minutes. 

It proved an easy tool for his team, too — he didn’t even need to train people on how to use the web app. They just published their first blog post, and they worked collaboratively (remotely, of course) to create and improve the new content type. When someone suggested the addition of a lead to the blog article, it only took Miller a minute to create an author content type and link it to the blog’s.

The team comprises only volunteers, many of whom work full time. They generously fill their evenings processing data and updating pages. The Contentful web app alleviates some of that pain, especially for contributors writing, editing and publishing editorials. 

The team plans to eventually integrate their database with Contentful, which will allow the project to produce editorials with complex data relationships. This development will allow them to create interactive content and automate some page updates — saving even more time. 

You can help 

The COVID Tracking project provides information to organizations such as the New York Times and The Atlantic on the pandemic, and it continues to grow. It provides a critical service in a time of great need. You can contribute to the project, too. Read up on what they need and how you can help.

Subscribe for updates

Build better digital experiences with Contentful updates direct to your inbox.