Should We Assess How Well Agencies Manage?

They began several months ago to catalog similar efforts in the private sector and other countries, as well as previous efforts in the U.S. federal government, such as the five-part scorecard developed as part of President George W. Bush’s Management Agenda and the Baldrige Excellence Framework.

What is the value of measuring the quality of management of federal agencies? Thompson and Medina found that in the private sector, measuring the quality of management in organization helps to draw attention to the importance of the management function in the context of the performance focus.  In initiatives such as the World Management Survey, the discipline of conducting a review helps promote management excellence and a learning dynamic.  Organizations that score well become exemplars to their peers.

An analogous federal-level effort sponsored by the non-profit Partnership for Public Service is called the “Best Places to Work” and it ranks by agency the results of an annual employee survey conducted governmentwide. It found that the mere act of measuring and ranking can provoke a learning dynamic, and it has brought top agency leadership attention to improving employee engagement.

How should a quality measurement framework be designed?  Last week, Thompson and Medina summarized their findings for an expert panel of current and former government executives convened by the IBM Center and the Senior Executives Association which discussed the potential design elements of such an initiative. The academic researchers solicited the opinions of multiple experts on key issues including;

• Should the initiative be top-down vs. bottom-up? In several countries surveyed by the researchers found the measurement initiatives were central staff-driven. An advantage of this approach is that agencies can be required to participate. A disadvantage is that the initiatives are seen by agencies as compliance exercises. An example of a voluntary, bottom-up approach is the Baldrige Excellence Framework.
• Should the survey be qualitative or quantitative in approach? Issues include: (1) is there a set of metrics that can best capture agency management quality? (2) Can a valid assessment of agency management quality be made exclusively on the basis of available metrics? And (3) If the answer to the preceding question is, “no,” what form of qualitative data would be needed as a supplement? In the U.K., for example, a team of five individuals (two executives from other agencies, three from outside the government) engaged in a series of interviews with senior officials at the subject agency.
• How should assessment results be characterized? Should there be a single score and ranking, like the “Best Places to Work” scorecard, or should the results be more qualitatively presented, such as the quarterly assessments of the Cross-Agency Priority Goals? The choice likely depends on the intent of the designers – is the framework’s focus to be internally, to increase an agency’s ability to learn how to improve, or to be externally focused as an accountability mechanism?
• Should there be a common assessment framework or a customized framework? Those interviewed by the researchers observed that it is very different managing in a mid- or small-size, semi- autonomous agency such as USAID or NASA than in a large, federated department with diverse functions like Interior or Agriculture. A management quality protocol would need to take size and perhaps other contextual elements (e.g., service delivery, regulatory, policymaking) into account.
• What practices characterize well-managed organizations? The academic researchers compiled items from various management assessment surveys. There were some commonalities, but a caveat was to avoid creating laundry lists and instead focus on a handful of key indicators so managers would take the effort seriously instead of as a compliance exercise.
• Should the assessment be a fixed or a “living” protocol? It is unlikely that a definitive protocol can be devised to assess the management quality of federal agencies. Consistent with a learning framework, the protocol/instrument should therefore be subject to ongoing refinement with broad participation on the part of the management community.

Next Steps.  The expert panel provided a rich discussion on these design topics based on their experiences of having worked in numerous federal agencies and in different roles.  In the coming months, the academic researchers will synthesize their insights and propose a framework that could potentially be an initiative in the next presidential administration. Stay tuned!

Graphic Credit:  Courtesy of Stuart Miles via FreeDigitalPhotos.net