Imagine if policymaking could be more proactive and less reactive. What if we had a system that could make it easier for people to use data to drive decisionmaking and hold decisionmakers accountable?
For instance, what if community groups could easily view the latest information on loan application approvals in their neighborhood to check for bias? What if journalists and researchers could instantly access and visualize historical data on jobs in a specific area? And what if local policymakers could quickly visualize housing trends when deciding how to address affordability in their city?
Thanks to the Amazon Web Services (AWS) Imagine Grant Program, our technology and data science team has the opportunity to build a system to provide this information.
With AWS funding, we plan to create a central source of preprocessed big, local datasets spanning various sectors and issues. Using an application programming interface (API)-first approach and the latest in cloud technology, we hope our system will unlock the potential of these data for the folks we work with every day — local policymakers, researchers, journalists, and community groups.
The issue
Data are critical for government decisionmaking, and they’re at the heart of Urban’s work. Cleaning, analyzing, and sourcing data for policy analysis takes time and resources, both of which policymakers may lack when crafting important policies that affect their constituents’ lives. Although the Urban Institute strives to produce timely evidence for many different audiences, we, along with other organizations in our field, simply don’t have the time to serve even a fraction of those wishing to leverage data to craft more informed policy.
While larger governments, states, and cities may have the budget and staff to process data themselves, smaller jurisdictions, researchers, community groups, and journalists may not, and they are often left to make decisions and hold officials accountable with inadequate data.
Even in larger jurisdictions and among research groups with the capacity, days are often spent collecting, cleaning, and combining large data sources for each project and analysis, limiting the amount of information policymakers receive in a timely manner. As a result, even where data-driven decisionmaking seems necessary, it is perceived as costly and time-intensive and is therefore used more often to evaluate past or current efforts than to inform future policymaking, where it’s potentially valuable.
We know we can do better.
How APIs can make policymaking more proactive
APIs allow us to create a central, clean data source that can be used as a foundation for many applications serving different audiences. Researchers can access and use the raw data, journalists can quickly build data visualizations, and policymakers can easily find summary answers. You can see this vision in action with Urban’s recently released Education Data Portal.
Traditionally, analyses of multiple datasets were exclusively the domain of social science researchers. By using the latest in big data technology, we can build an API foundation that allows us to provide rapid, customized analytics on big datasets for new audiences. We plan to leverage AWS to make millions of raw records accessible to researchers while allowing policymakers, journalists, and community groups to access aggregated statistics in seconds. Recent advances in technology allow the data to be stored in efficient parallelized formats and queried by performant data warehouse services capable of rapidly processing big data.
Cutting-edge technology makes data accessible to more stakeholders than our reports alone can reach, empowering different groups to use these data for proactive policymaking, journalism, and advocacy.
We need your help
We plan to build the API over the coming year, and we need your help. We will reach out to journalists, policymakers, community groups, and researchers over the next few months to get feedback on the concept and solicit suggestions for which initial datasets should be available.
And this API framework is just a start. We plan to use this work as a foundation for applications on our APIs that target specific audiences with specific needs. We are in the first phase of our journey to make policymaking more proactive and less reactive, and we’re excited to get started.
