The crisis surrounding the National Weather Service’s (NWS’s) handling of its assessment of the agency’s performance before and during Hurricane Sandy – including but not limited to the controversial decision not to issue hurricane warnings for the northeastern United States coastline – continues to deepen. A new assessment effort was announced on Thursday, but in an apparent reversal of course, the new panel will not include any experts from outside of the federal government.
A Broadening Tent
Since 2005, inclusive, the NWS (including its regional headquarters) has conducted 18 service assessments of everything from tornado outbreaks to tsunami warnings, an average of just under three per year. This is somewhat dependent on the number and impact of the events: 2011 was a very busy year in terms of events and impact and resulted in six assessments. 2010 and 2006 were relatively quiet with only one assessment each.
In 2005 and 2006, nearly every service assessment member – 37 out of 39, or 95% – was an employee of the NWS; however, that percentage has dropped as the agency as warmed to input from other federal agencies as well as non-federal employees. The proportion of non-NWS representation on these service assessment teams has increased from near zero in 2005 to hover around 25% in the last four years. One reason for the increase in non-federal representation is the incorporation of social science researchers – sociologists, communication scholars, and others – charged with understanding what happened with the NWS products once they were completed and made available through various means to the public.
The original, now aborted, assessment team for Sandy had more than a dozen people on it, including the first non-federal co-chair of such an effort. It also included a number of social science researchers: One of the central questions facing the team was the consequences of the choice to post high wind and coastal flood warnings in lieu of hurricane warnings for the northeastern US coast, a question external experts from the academic and research community are better equipped to answer.
According to the NWS, the new effort will not include any such outside experts. NWS spokesperson Susan Buchanan told USA Today this panel will consist of only federal employees, a marked reversal from previous practice. As Rob White pointed out this decision “appears to show a lack of concern with point 6 of U.S. Rep. Paul Broun’s letter to the Administrator of the agency, issued on November 20th”. I would add it shows similar lack of concern for point 8 of the aforementioned letter, which asks whether the team would consist of “a broad range of federal as well as private sector individuals”.
National Academy of Sciences: Include Outside Representatives
Rep. Broun’s concerns for inclusion of outside representatives are not without scientific merit. In the 2012 report, Weather Services for the Nation: Becoming Second to None, the National Academies of Sciences recommended the NWS “broaden the scope of the system for evaluation its forecasts and warnings.” The report goes on to recommend (II.a: Post-Event Evaluations):
[The NWS] should consider whether having an independent entity conduct all post-event evaluations of performance after false alarms and significant events would be more effective. These evaluations should address the full scope of response issues, from forecasts and warnings to communication and public response, and be conducted by an appropriate mix of individuals from within and outside the NWS.
In omitting non-NWS personnel and independent participants from the new Sandy assessment team, the NWS is thumbing its nose at the congressionally-chartered National Academy of Sciences as well as reversing its own trend of including more such experts.
A concern has been raised about whether assessment teams consisting of non-federal employees ran afoul of the Federal Advisory Committee Act (FACA), which is designed to ensure a formality in how groups consisting of non-federal employees can advise the Executive Branch (under which the NWS falls) operate and to require additional reporting toward the end of better transparency of such efforts. You may remember Vice President Dick Cheney sued regarding whether his National Energy Policy Development Group (NEPDG) should be forced to comply with the law. The original verdict would have forced the Vice President to reveal the names of people on this group and make public how the committee did its work and the results of their deliberations. After an appeal and a reconsideration, the US Court of Appeals for the District of Columbia rules the NEPDG did not fall under the purview of FACA and the Group’s operations could remain non-public.
Clearly, the NWS has operated a number of service assessment teams that included non-federal employees going back at least six years. While the NWS service assessment process, and its inclusion of non-federal participants, may not be in total compliance with the letter of FACA, its track record of the process suggests a desire to comply with the spirit of the law. The teams were chartered for a specific task, operated for a limited amount of time, produced a final report made available to the public, and documented the constitution and affiliations of its membership. To what extent additional work product – field notes, interview recordings and transcriptions, draft reports, minutes from deliberative sessions, and interactions with the NWS and National Oceanic and Atmospheric Administration (NOAA) hierarchy (including any requested modifications to the team’s otherwise final report) – and other elements would be subject to public review is a legal question and, if the NWS had not considered it, a valid concern before this team moved forward. However, neither public NOAA/NWS statement regarding the scuttling of the original team has referred to FACA being a concern.
There have been two statements by the NWS regarding the suspension of the original Sandy team. The first was by Acting NWS Director Laura Furgione, as reported by Climate Central:
NWS Acting Director Laura Furgione told Climate Central that the formation of the service assessment had been “premature” and that she had not seen or approved a charter governing the scope of the team’s work. Furgione did not say who approved the initial decision. She said the NWS is committed to having a review, and is trying to speed up the review process.
In that article, the NWS also refers to the possibility of a “a larger, multi-agency review of this event” but does not offer any details.
Later, the NWS told The Washington Post’s Capital Weather Gang, “work on the NWS service assessment was stopped by the acting director of the National Weather Service after initial discussions with DHS [Department of Homeland Security] about a potential broader federal assessment”.
There are two distinct narratives here:
- The original assessment team’s charter and budget had not been approved by the NWS director’s office. This is refuted by Mike Smith, the co-chair of that team, who says he can “provide the email approving the team, the team’s charter, and even the accounting codes we were given for expense reimbursement”.
- The original assessment was stopped because DHS expressed interest in a broader, multi-agency review. This is problematic for a number of reasons because DHS has no oversight authority over the NWS or NOAA, which exist under the Department of Commerce (DOC), nor was there any reason why the original NWS team could not move forward with the NWS-centric portions of their work. The NWS should have continued the team’s work and invited other agencies to join their effort, not suspended an in-progress review based on “initial discussions” of the “potential” of a broader review.
These two narratives are not mutually exclusive, of course, but we have yet to receive an official story from the NWS that addresses both. In the meantime, how one reads the tea leaves – and the observation that we have not yet gotten a single, cohesive answer about this from the NWS – can lead to conclusions ranging from bureaucratic clumsiness to ineptitude and poor leadership. In any event, as Bryan Norcross notes, the NWS has a “PR problem” with the very group who controls its budget. It has already been subject to a cabinet-level review, called out by another US congresswoman, and had its previous director forced into retirement all in the last year as a result of an investigation into discrepancies in its handling of contracts and funding. While no malfeasance was found, the less Congress feels your agency needs additional oversight, the better. Congress may be inclined to dig further into the management of the NWS if it does not rectify the heretofore poor handling of the Sandy assessment process – and that’s before the assessment team is allowed to complete its report. Given the magnitude and extent of Sandy’s impact, the report itself may lead to additional questions the NWS must answer.
NWS leadership risks a full-blown crisis, a growing swell of mistrust and suspicion of the agency within the weather enterprise, congressional hearings, and potentially worse if it does not act quickly and decisively to address the numerous concerns have been raised regarding its handling of the Sandy assessment. Good crisis management dictates a full and forthright provision of information, including that which may not reflect well on the organization or its administrators, in an attempt to answer existing questions beyond reproach. To that end, it should (immediately):
- Answer thoroughly and publicly each of the questions raised in Rep. Broun’s letter from November 20th, including a detailed explanation of who approved the original Sandy assessment team, who drove the decision to scuttle that effort, and the reasoning behind that decision. In addition to the reasons given above, some in the weather community believe, correctly or not, that the controversial choice of Mike Smith – a senior vice president with the large private-sector forecasting firm, AccuWeather – contributed to this team’s having been terminated. An explanation of who approved Smith’s inclusion as co-lead in the first place and whether his participation played a role in the termination of the original team is warranted.
- Form and commission (not just announce) a service assessment team charged with evaluating the agency’s performance before and during Hurricane Sandy. That team should be granted a wide latitude and independence to complete its work, including travel to any relevant national centers and local forecast offices and access to any personnel involved in any stage of the forecasting, warning, and preparation for Sandy. It should include appropriate participants from outside the federal government, per the NAS recommendation and the intent behind the first Sandy assessment team. If outside experts are not included, a thorough explanation of this should also be provided and should specifically address the NAS report.
- Consider reaching out to the other segments of the weather enterprise to begin reversing the damage it has done to relationships with other members of the community.
Each delay, each partial release of information, each inconsistency, and each act seen as thumbing the agency’s nose at Congress (among others) only serves to drive the NWS further into crisis. The agency’s meteorologists, hydrologists, and technicians who work shifts 24 hours a day, seven days a week to collect data, generate forecasts, and issue warnings the nation and its commerce and safety depend upon – not to mention the broader weather enterprise that relies on the NWS as a partner – deserve better than what is exemplified by the poor handling of this mess.