Using Administrative Data for Statistical Purposes

The Obama Administration has championed “open data” by encouraging agencies to make a wider range of agency statistical information available to the public.  Its philosophy is “information is a valuable national resource and strategic asset.” To that end, it has issued directives and created a centralized website for data open to the public.

But what about those administrative data that cannot be publicly shared for privacy or business confidentiality reasons?  “Administrative data” are collected and used by agencies to carry out the basic administration and processing of different government programs. For good reasons, government records of individuals’ tax, social security earnings, health insurance, and other benefit information cannot and should not be shared publicly.  In fact, there are strict laws in place to safeguard these kinds of data.

Sharing Non-Public Data.  Can these non-public protected data at least be shared between federal agencies -- if stripped of individual identifiers -- so they can be used for statistical purposes?  To date, that has been difficult, and each agency has been left to interpret whether they can share their data or not, and under what circumstances.

Existing laws allow certain kinds of sharing, but in many cases, existing constraints (either real or perceived) on sharing data has meant that one agency will have to conduct a survey of people or businesses for information that has already been collected by another agency. This has resulted in costly and duplicative data collection efforts for government, businesses, and individuals. 

For the past five years, the Office of Management and Budget (OMB) has been advocating the sharing of statistical information between agencies in order to conduct evidence-based analyses supporting better program decisions, without new and costly data collection efforts.  For example, the Department of Housing and Urban Development has worked with the Department of Veterans Affairs to share statistical data about homeless veterans in order to better target services to them.  As a result of their joint efforts, they have reduced veteran homelessness by a quarter since 2010.

OMB’s New Guidance.  Last week, OMB quietly released a significant piece of guidance to agencies that provides practical advice on how to share non-public administrative data for statistical purposes in responsible ways that protect confidentiality.   

• Foster collaboration across program and statistical agencies.  An earlier OMB memo required agencies to develop an inventory of the various data sets they collect and maintain. Agency heads need to “communicate the importance of identifying those administrative datasets with great potential for statistical use” and highlight their existence to relevant statistical agencies. 
• Adhere to data stewardship practices that safeguard program data used for statistical uses.  OMB outlines a set of principles for data stewardship by agencies “in order to maintain public trust in their ability to appropriately handle identifiable information.”  For example, certifying procedural safeguards to be used, and eliminating identifiable information when the data are no longer needed.
• Documenting clear quality control measures for statistical agencies. OMB says “program agencies should provide the technical documentation or other assistance that statistical agencies or components require to adequately assess the quality of a particular data set.”  Sometimes, OMB notes, program agencies might be able to make their data more useful for statistical purposes by making changes to how they collect or report it.
• Foster the use of clearly written interagency agreements for data sharing.  OMB says agencies and statistical agencies should use interagency agreements to document “terms and conditions” of when and how non-public data should be shared for statistical purposes.  OMB’s guidance even includes a helpful checklist of what should be included in such an agreement – but encourages agencies to “seek the dvice of their counsel before signing.”

Next Steps.  Of course, no OMB memo would be complete without a requirement for agencies to report back on their progress.  So agencies are required to report to OMB no later than June 30, 2014 on what processes they have put in place to implement the guidance.  Again, OMB provides a helpful outline of what that report should contain, including:

• A copy of whatever “communication” the department head sent out to staff to tell them this is important stuff.
• Identify at least three data sets they have identified as having “highest potential statistical value” – including data sets in other agencies! . . . and the status of any requests to actually use these data.
• A list of any barriers they see as preventing them from sharing data with other agencies for statistical purposes.

So, if you are in an agency and think another agency has data you think would be useful to your meeting your mission – you now have the permission slip to ask for it (for statistical purposes, that is)!