SPEAKERS CONTENTS INSERTS
Page 1 TOP OF DOC
75–811PS
2002
NATIONAL SCIENCE FOUNDATION'S
MAJOR RESEARCH FACILITIES:
PLANNING AND MANAGEMENT ISSUES
HEARING
BEFORE THE
SUBCOMMITTEE ON RESEARCH
COMMITTEE ON SCIENCE
HOUSE OF REPRESENTATIVES
ONE HUNDRED SEVENTH CONGRESS
FIRST SESSION
SEPTEMBER 6, 2001
Serial No. 107–48
Printed for the use of the Committee on Science
Available via the World Wide Web: http://www.house.gov/science
Page 2 PREV PAGE TOP OF DOC
COMMITTEE ON SCIENCE
HON. SHERWOOD L. BOEHLERT, New York, Chairman
LAMAR S. SMITH, Texas
CONSTANCE A. MORELLA, Maryland
CHRISTOPHER SHAYS, Connecticut
CURT WELDON, Pennsylvania
DANA ROHRABACHER, California
JOE BARTON, Texas
KEN CALVERT, California
NICK SMITH, Michigan
ROSCOE G. BARTLETT, Maryland
VERNON J. EHLERS, Michigan
DAVE WELDON, Florida
GIL GUTKNECHT, Minnesota
CHRIS CANNON, Utah
GEORGE R. NETHERCUTT, JR., Washington
FRANK D. LUCAS, Oklahoma
GARY G. MILLER, California
JUDY BIGGERT, Illinois
WAYNE T. GILCHREST, Maryland
W. TODD AKIN, Missouri
TIMOTHY V. JOHNSON, Illinois
Page 3 PREV PAGE TOP OF DOC
MIKE PENCE, Indiana
FELIX J. GRUCCI, JR., New York
MELISSA A. HART, Pennsylvania
J. RANDY FORBES, Virginia
RALPH M. HALL, Texas
BART GORDON, Tennessee
JERRY F. COSTELLO, Illinois
JAMES A. BARCIA, Michigan
EDDIE BERNICE JOHNSON, Texas
LYNN C. WOOLSEY, California
LYNN N. RIVERS, Michigan
ZOE LOFGREN, California
SHEILA JACKSON LEE, Texas
BOB ETHERIDGE, North Carolina
NICK LAMPSON, Texas
JOHN B. LARSON, Connecticut
MARK UDALL, Colorado
DAVID WU, Oregon
ANTHONY D. WEINER, New York
BRIAN BAIRD, Washington
JOSEPH M. HOEFFEL, Pennsylvania
JOE BACA, California
JIM MATHESON, Utah
STEVE ISRAEL, New York
Page 4 PREV PAGE TOP OF DOC
DENNIS MOORE, Kansas
MICHAEL M. HONDA, California
Subcommittee on Research
NICK SMITH, Michigan, Chairman
LAMAR S. SMITH, Texas
CURT WELDON, Pennsylvania
GIL GUTKNECHT, Minnesota
FRANK D. LUCAS, Oklahoma
GARY G. MILLER, California
JUDY BIGGERT, Illinois
W. TODD AKIN, Missouri
TIMOTHY V. JOHNSON, Illinois
FELIX J. GRUCCI, JR., New York
MELISSA A. HART, Pennsylvania
SHERWOOD L. BOEHLERT, New York
EDDIE BERNICE JOHNSON, Texas
BOB ETHERIDGE, North Carolina
STEVE ISRAEL, New York
LYNN N. RIVERS, Michigan
JOHN B. LARSON, Connecticut
BRIAN BAIRD, Washington
JOE BACA, California
DENNIS MOORE, Kansas
Page 5 PREV PAGE TOP OF DOC
MICHAEL M. HONDA, California
RALPH M. HALL, Texas
SHARON HAYS Subcommittee Staff Director
PETER HARSHA Republican Professional Staff Member
JIM WILSON Democratic Professional Staff Member
DIANE JONES, GREG GARCIA Professional Staff Members
NATALIE PALMER Staff Assistant
C O N T E N T S
September 6, 2001
Witness List
Hearing Charter
Opening Statements
Statement by Representative Nick Smith, Chairman, Subcommittee on Research, Committee on Science, U.S. House of Representatives
Written Statement
Statement by Representative Eddie Bernice Johnson, Member, Committee on Science, U.S. House of Representatives
Written Statement
Page 6 PREV PAGE TOP OF DOC
Panel
Dr. Rita R. Colwell, Director, National Science Foundation
Oral Statement
Written Statement
Biography
Dr. Anita K. Jones, Vice Chair, National Science Board
Oral Statement
Written Statement
Biography
Dr. Christine C. Boesz, Inspector General, National Science Foundation
Oral Statement
Written Statement
Summary Statement
Biography
Discussion
Appendix 1: Answers to Post-Hearing Questions
Dr. Rita R. Colwell, Director, National Science Foundation
Questions Submitted by Chairman Nick Smith
Page 7 PREV PAGE TOP OF DOC
Questions Submitted by Ranking Minority Member Eddie Bernice Johnson
Dr. Anita K. Jones, Vice Chair, National Science Board
Questions Submitted by Chairman Nick Smith
Dr. Christine C. Boesz, Inspector General, National Science Foundation
Questions Submitted by Chairman Nick Smith
Questions Submitted by Ranking Minority Member Eddie Bernice Johnson
NATIONAL SCIENCE FOUNDATION'S MAJOR RESEARCH FACILITIES: PLANNING AND MANAGEMENT ISSUES
THURSDAY, SEPTEMBER 6, 2001
House of Representatives,
Subcommittee on Research,
Committee on Science,
Washington, DC.
The Subcommittee met, pursuant to call, at 9:37 a.m., in Room 2318 of the Rayburn House Office Building, Hon. Nick Smith [Chairman of the Subcommittee] presiding.
75811a.eps
Page 8 PREV PAGE TOP OF DOC
HEARING CHARTER
SUBCOMMITTEE ON RESEARCH
COMMITTEE ON SCIENCE
U.S. HOUSE OF REPRESENTATIVES
National Science Foundation's
Major Research Facilities:
Planning and Management Issues
THURSDAY, SEPTEMBER 6, 2001
9:30 A.M.–11:00 A.M.
2318 RAYBURN HOUSE OFFICE BUILDING
1. Purpose
On Thursday, September 6, 2001, at 10:00 a.m. the Subcommittee on Research of the House Committee on Science will hold a hearing on the planning and management of major research facilities by the National Science Foundation (NSF). Testifying before the Subcommittee will be the Director of NSF, the Vice Chair of the National Science Board, and the NSF Inspector General.
Page 9 PREV PAGE TOP OF DOC
Recently, a number of organizations, including the National Academy of Sciences, the NSF Office of the Inspector General, Congress, the Office of Management and Budget, and the scientific community, have raised concerns over the adequacy of NSF's planning and management of large research facilities. These large facilities include astronomical observatories, supercomputer centers, the South Pole Station, and earthquake simulators, among others. Witness testimony will describe the process by which these projects are selected for funding as well as agency oversight during implementation and operation of these facilities.
2. Background
NSF's Large Research Facilities
As part of its mission, the NSF is responsible for supporting state-of-the-art facilities and equipment that enables scientific and engineering research and education. In the past, NSF has enabled the construction and operation of facilities such as telescopes, oceanographic research vessels, and a superconducting particle accelerator, and the provision of large-scale computational infrastructure, among other major research equipment and facilities.
The NSF's investment in facilities and major research equipment has grown steadily, and today the Agency spends approximately $1 billion per year on facilities and other infrastructure projects. These technically complex projects can cost a hundred million dollars or more and require years of planning, design, and construction involving multiple contractors and researchers. In addition, some of the projects—such as the Large Hadron Collider, and the Atacama Large Millimeter Array (ALMA)—involve the additional complexity provided by the participation of international partners.
Page 10 PREV PAGE TOP OF DOC
In addition to taking on large infrastructure projects such as telescopes or supercolliders, an increasing focus within the NSF on interdisciplinary science and other emerging research opportunities has resulted in the need for new kinds of facilities. The Network for Earthquake Engineering Simulation and the National Ecological Observatory Network (NEON) are examples of networks of multiple, smaller facilities the NSF is currently engaged in developing. These new kinds of facilities require novel planning, construction, and management approaches.
The NSF's approach to developing and managing large research facilities differs from that of other federal R&D agencies. For example, unlike NASA or the Department of Energy, the NSF does not directly construct or operate research facilities. Instead, the NSF makes awards to external entities—usually universities or other non-profit research organizations—which are in turn responsible for the actual construction, management, and operation of the projects. The NSF's relationship with these organizations is typically not contractual, but instead is defined in cooperative agreements. However, NSF maintains ultimate responsibility for oversight of the construction and operation of the major research facilities it funds.
Management Issues
The NSF faces a number of challenges in its oversight of construction and management of large research facilities. The sheer magnitude of large facility projects makes the grant awards associated with them significantly more complex than the typical NSF grant award. Whereas most NSF grants are made to independent investigators for performance of research projects that average less than $100,000 in total costs, large facility projects can require expenditures of $100 million dollars or more, which are spent over multiple years and require complex accounting practices in order to administer. While aided by personnel in divisions at NSF that specialize in contracts, accounting, and so forth, the NSF employees assigned responsibility for oversight of large research facilities are typically Program Officers, who are usually scientists or engineers and who may have little or no experience or training in construction management, accounting, or other areas of expertise demanded by the task of overseeing construction and/or operation of a large research facility. These personnel issues are particularly challenging for the NSF given that many Program Officers—and even their superiors—work at NSF on a temporary basis. These ''rotators'' are typically academic researchers and/or educators who spend approximately two years at NSF before returning to their university appointments.
Page 11 PREV PAGE TOP OF DOC
A few recent major facility projects at NSF have encountered financial and operational difficulties. For example, The Gemini Telescope Project, which supports the construction and operation of two telescopes in Hawaii and Chile, was cited in a December 2000 Inspector General (IG) report as being over its planned budget by at least $52.8 million. The IG determined that the Gemini project used funds designated for operating costs in order to cover the construction overruns, and the report stated that ''NSF's current policies and procedures for overseeing and administering large infrastructure awards need improvement.'' NSF disputes that the $52.8 million identified by the IG as operations costs was inappropriately used to cover construction, but agreed with many of the IG's recommendations, including those aimed at providing increased NSF oversight over large facility projects.
The Robert C. Byrd Green Bank Telescope, the world's largest fully steerable radio telescope, was completed six years later than originally planned. The project ended with a legal battle over cost overruns, ultimately settled by arbitration, between the contractor that built the telescope, COMSAT Corporation, and the Associated Universities, Inc. (AUI), which handled construction of the facility under a cooperative agreement with NSF. COMSAT had sought an additional $29 million over the contracted amount from AUI, claiming the scope of the project had changed significantly since its inception. The arbitrator ruled that AUI pay COMSAT $4.9 million; NSF is likely to end up covering these additional costs.
In A Blueprint for New Beginnings: A Responsible Budget for America's Priorities, President Bush stated that ''NSF's capability to manage proposed projects needs to be enhanced given the magnitude and costs of these projects.'' He called for the NSF to develop a plan to enhance its capabilities for managing large facility project development and construction, and to ensure that cost and schedule commitments for major facilities are met. The concerns expressed in the budget ''blueprint'' echo those of the Inspector General, who has cited management of large infrastructure projects as one of the top challenges facing the NSF.
Page 12 PREV PAGE TOP OF DOC
The NSF has responded by setting out to develop goals and strategies for addressing the management issues, which are described in the Agency's Large Facility Projects Management and Oversight Plan. The most recent draft of this plan was presented to the National Science Board at the Board's meeting last month, and the Office of Management and Budget is currently reviewing it. Among other things, the Plan calls for the creation of a new position at NSF of Deputy, Large Facility Projects. This Large Facility Projects (LFP) Deputy will report directly to NSF's Chief Financial Officer, who in turn reports directly to the NSF Director. NSF has indicated they intend to recruit an individual to fill this position who has ''extensive project management experience, including building, management, and oversight of large scientific and engineering facilities.'' NSF also intends to create several permanent positions in order to support the new LFP Deputy.
Other important aspects of the Management and Oversight Plan are not new, however, and sustain current policy. For example, the most recent draft states that ''in every large facility project, the NSF Program Manager exercises primary responsibility for all aspects of project management,'' indicating that ultimate responsibility for overseeing large facility projects will continue to lie in the hands of Program Officers—who are typically research scientists or engineers as described above. The LFP Deputy and the Deputy's support staff are described as ''a centralized resource to assist (but not supplant) NSF Program Managers with management and oversight responsibilities.'' The precise nature of the interaction between NSF personnel with expertise on the ''business side'' of large facility management and those with an understanding of the scientific issues remains unclear.
Planning and Budget Prioritization Issues
Page 13 PREV PAGE TOP OF DOC
The process by which the NSF determines which major facility projects to fund is also currently under examination. In general, the requirements of the scientific community for new facilities and equipment far outstrip the funds available for these projects; necessitating the setting of funding priorities. At present, the planning, development, formal proposal, budgeting, and contracting process for large facilities is a multi-year, multi-stage process. It involves the scientific community, NSF's individual research directorates, a special NSF MRE review panel, the NSF Director, and the National Science Board.
The process is complex enough that it currently appears to be interpreted differently by different actors. The lack of transparency surrounding this process has led to criticism from the scientific community, Congress, and others. While the Large Facility Projects Management and Oversight Plan provides information on various stages of the approval process for large facility projects, it appears to leave key aspects of the prioritization process largely unexplained.
3. Witnesses
The Committee will hear from the NSF Director, who has been asked to describe the process by which major infrastructure projects are selected for funding as well as the management and oversight of funded projects as described in the Large Facility Projects Management and Oversight Plan. The Vice Chairman of the National Science Board will describe the Board's role in the selection process and will comment on the Management and Oversight Plan, as will the NSF Inspector General.
Page 14 PREV PAGE TOP OF DOC
Witnesses testifying before the Committee will be:
Dr. Rita Colwell, Director of the National Science Foundation
Dr. Anita Jones, Vice Chair of the National Science Board
Dr. Christine Boesz, National Science Foundation Inspector General
The panelists were asked to address the following questions in their testimony before the Committee:
What are the respective roles of the Director and the National Science Board in the prioritization and funding decisions for major facilities and how are these priorities disseminated?
How is NSF working to improve its planning and management capabilities for major facilities, in response to concerns raised by the Inspector General, OMB, the National Academy of Sciences, and Congress?
Chairman SMITH. Good morning and welcome to this Research Subcommittee hearing on NSF's Oversight of Major Research Facilities. Planning and management issues are very important if we are going to accomplish the two goals of being effective and efficient in the construction of these facilities and in making sure that they serve the scientific interest in the best possible way for which they were intended.
Page 15 PREV PAGE TOP OF DOC
Today we will look at the National Science Foundation's increasing role in funding large research facilities. NSF, of course, is best known for its funding of relatively small-scale scientific research conducted by individuals or small groups. As science has progressed, however, the NSF has increased its support for larger projects and facilities that provide major infrastructure for the scientific community. With these increased requests for increasingly complex infrastructure projects comes ever-growing management challenges.
The research community has been very enthusiastic regarding—the research community has been very enthusiastic regarding NSF's track record in providing scientists with state-of-the-art facilities and large-scale research equipment. In fact, about the only complaint I have heard from researchers is that there isn't enough money and the length of the project time hasn't been extended so that they can make the kind of adequate planning that they would like to make.
While I, and many others in Congress, continue to push for more funding for NSF, we all have to recognize that the scientific community's new facilities wish list will always outstrip the resources available for funding these projects. Given this fiscal reality, prioritization of potential projects is critical. And I look forward to gaining a better understanding of how NSF intends to make these difficult decisions of prioritizing and what role the NSB (National Science Board) plays, Dr. Jones, in forming that priority list. And I firmly believe that Congress and the scientific community deserve to have a much clearer picture of NSF's priorities for major research facilities.
And to simply leave that decision to OMB to or to leave that decision to Congress, without that specific input from the scientific community on which projects should go first, I think does a disservice to our ultimate decisions.
Page 16 PREV PAGE TOP OF DOC
The second major issue we will address today is more one of nuts and bolts—in some cases, almost literally. While NSF generally does not directly build, own, or operate research facilities, the agency has the responsibility for overseeing construction and operation of these facilities in order to ensure that taxpayer dollars are spent as effectively and efficiently as possible. On a large project, even a proportionately small cost overrun can add up to big money. To offer an extreme example, I would look at the Space Station that we have been talking about for the last 8 years, and its recently announced $4.8 billion cost overrun, an amount larger, I might mention, than the NSF's entire 2002 budget.
I look forward to hearing about the new Large Facility Project Management and Oversight Plan that you have drafted—that NSF has drafted, in response to concerns over the management of large facilities, and, in general, to working closely with NSF to ensure that future projects are well-managed so that they are efficiently constructed and are able to realize their full scientific potential.
I appreciate the IG's input and interest into assuring, Dr. Boesz, of how we can best accomplish these goals and cooperation. And I hope, Dr. Colwell, you, in your testimony, will respond to some of the concerns of my letter of July 27 that I wrote to you. I have invited, and some of the staff are present from the Appropriations Subcommittee on VA HUD as we join in working together to try to accomplish our mutual goals.
And without objection, I would move that that letter of July 27 that I sent to NSF be inserted as an addendum to my opening remarks and testimony. Without objection, so ordered. And with that, I would turn the microphone to Ms. Johnson.
Page 17 PREV PAGE TOP OF DOC
[The prepared statement of Mr. Smith follows:]
PREPARED STATEMENT OF CHAIRMAN NICK SMITH
Good morning and welcome to this Research Subcommittee hearing on NSF's Major Research Facilities: Planning and Management Issues. Today we will look at the National Science Foundation's increasing role in funding large research facilities. NSF is best known for its funding of relatively small-scale scientific research conducted by individuals or small groups. As science has progressed, however, the NSF has increased its support for larger projects and facilities that provide major infrastructure for the scientific community. With these increased requests for increasingly complex infrastructure projects come ever-growing management challenges.
The research community has been overwhelmingly enthusiastic regarding NSF's track record in providing scientists with state-of-the-art facilities and large-scale research equipment. In fact, about the only complaint I've heard from researchers is that there isn't enough money to fund more of these large-scale projects! While I and many others in Congress continue to push for more funding for NSF, we all have to recognize that the scientific community's new facilities ''wish list'' will always outstrip the resources available for funding these projects. Given this fiscal reality, prioritization of potential projects is critical. I look forward to gaining a better understanding of how the NSF makes these difficult decisions, and what role the NSB plays in informing them. I firmly believe that increased transparency can only benefit the process.
Page 18 PREV PAGE TOP OF DOC
The second major issue we will address today is more one of ''nuts and bolts''—in some cases, almost literally. While NSF generally does not directly build, own, or operate research facilities, the agency has the responsibility for overseeing construction and operation of these facilities in order to ensure that taxpayer dollars are spent as efficiently and effectively as possible. On a large project, even a proportionately small cost overrun can add up to big money. To offer an extreme example, we need only consider the International Space Station and its recently announced $4.8 billion cost overrun—an amount larger than the NSF's entire FY 2002 budget.
I look forward to hearing about the new Large Facility Projects Management and Oversight Plan that the NSF has drafted in response to concerns over the management of large facilities, and, in general, to working closely with the NSF to ensure that future projects are well-managed so that they are able to realize their full scientific potential.
I thank the witnesses for appearing today, and I look forward to your testimony.
Ms. JOHNSON. Thank you very much, Mr. Chairman. And I congratulate you on calling this oversight hearing on NSF's management of large research facility projects. I believe that the NSF has a good overall track record in managing major facility construction projects. But we are now seeing a significant increase in the number and complexity of such projects being proposed. Clearly, it was prudent and timely for the NSF to re-examine its current processes and procedures, both for the selection of new construction projects and for management and oversight of projects selected for construction.
Page 19 PREV PAGE TOP OF DOC
The Committee has had the opportunity to review NSF's Large Facility Projects Management and Oversight Plan which was presented to the National Science Board at its recent meeting. I believe that the Plan has positive aspects that will help to strengthen and improve facility target management. It recognizes the need for building up in-house project management expertise in NSF and for developing comprehensive management plans for all proposed facility projects.
It also stresses the importance of putting in place a more systematic oversight process for construction projects. However, there are aspects of the Plan that would benefit from greater elaboration. Relative to the new institutional arrangements proposed in the Plan, I am concerned about whether the lines of authority for facility project management are sufficiently clear.
For example, what exactly is the relationship between the proposed Deputy for Large Facility Projects who will report to the Chief Financial Officer and the Program Manager for a project who reports to an Assistant Director? This arrangement appears to create separate lines of authority to business oversight in technical management of a project. It is important that the authority and responsibility for the management of large construction projects be unambiguous.
The institution arrangement should also ensure that an adequate and timely flow of information is provided to senior management at NSF and to the National Science Board regarding contract performance, as measured against cost, schedule, and technical goals.
I would also like to learn more about the types of skills that will be represented among the new staff of the Office of the Deputy for Large Facility Projects. I assume the personnel in this office will include individuals with expertise in technical areas such as construction engineering and not just in financial management areas.
Page 20 PREV PAGE TOP OF DOC
Finally, I am interested in the process for selecting new facility construction projects from among the meritorious projects proposed. The facility Plan leaves out the general process and the sequence of events. However, I am still uncertain about how the final prioritization among approved projects occur. That is, what are the criteria and how are they applied to identify the projects that are included in NSF's annual budget request? I gather there is no prioritized list of projects judged to be ready for development and there are projects that appear in the budget request for one year, but it is not funded by Congress, making missing from the next year's request. I believe that NSF should strive to make this prioritization process for approved facility projects as transparent as possible.
Mr. Chairman, I join you in welcoming our witnesses today and I look forward to discussing the NSF Large Facility Plan and learning more about how it will be implemented. Thank you.
[The prepared statement of Ms. Johnson follows:]
PREPARED STATEMENT OF REPRESENTATIVE EDDIE BERNICE JOHNSON
Mr. Chairman, I congratulate you on calling this oversight hearing on NSF's management of large research facility projects.
I believe that NSF has a good overall track record in managing major facility construction projects. But we are now seeing a significant increase in the number and complexity of such projects being proposed. Clearly, it was prudent and timely for NSF to reexamine its current processes and procedures both for selection of new construction projects, and for management and oversight of projects selected for construction.
Page 21 PREV PAGE TOP OF DOC
The Committee has had the opportunity to review NSF's Large Facility Projects Management and Oversight Plan, which was presented to the National Science Board at its recent meeting. I believe that the plan has positive aspects that will help to strengthen and improve facility project management. It recognizes the need for building up in-house project management expertise at NSF and for developing comprehensive management plans for all proposed facility projects. It also stresses the importance of putting in place a more systematic oversight process for construction projects.
However, there are aspects of the plan that would benefit from greater elaboration. Relative to the new institutional arrangements proposed in the plan, I am concerned about whether the lines of authority for facility project management are sufficiently clear. For example, what exactly is the relationship between the proposed new Deputy for Large Facility Projects, who will report to the Chief Financial Officer, and the Program Manager for a project, who reports to an Assistant Director? This arrangement appears to create separate lines of authority for business oversight and technical management of a project.
It is important that the authority and responsibility for the management of large construction projects be unambiguous. The institutional arrangements should also ensure that an adequate and timely flow of information is provided to senior management at NSF and to the National Science Board regarding contractor performance as measured against cost, schedule and technical goals.
I would also like to learn more about the types of skills that will be represented among the new staff for the office of the Deputy for Large Facility Projects. I assume the personnel in this office will include individuals with expertise in technical areas such as construction engineering, and not just in financial management areas.
Page 22 PREV PAGE TOP OF DOC
Finally, I am interested in the process for selecting new facility construction projects from among the meritorious projects proposed. The facility plan lays out the general process and the sequence of events. However, I am still uncertain about how the final prioritization among approved projects occurs. That is, what are the criteria and how are they applied to identify the projects that are included in NSF's annual budget request? I gather there is no prioritized list of projects judged to be ready for development, and that a project that appears in the budget request one year, but is not funded by Congress, may be missing from the next year's request.
I believe that NSF should strive to make this prioritization process for approved facility projects as transparent as possible.
Mr. Chairman, I join you in welcoming our witnesses today. I look forward to discussing the NSF's Large Facility Plan and learning more about how it will be implemented.
Chairman SMITH. Thank you, Representative Johnson. If there is no objection, if any of the other members have prepared statements, without objection, those statements submitted in writing will be entered into the record. Thank you.
At this time, I would like to introduce our panelists. Our first witness is Dr. Rita Colwell, Director of the National Science Foundation. Our second witness is Dr. Anita Jones, who is Vice Chairman of the National Science Board. And Dr. Jones is also a Professor of Engineering, Applied Science in the Department of Computer Science at the University of Virginia. And our third witness is Dr. Christine Boesz, the National Science Foundation Inspector General. I thank you all for appearing and being with us today. And, as many of you know, your written testimony will be entered into the record in its entirety, and I would ask you all to try to limit your comments, opening comments, to about five minutes. And Dr. Colwell, we will start with you.
Page 23 PREV PAGE TOP OF DOC
STATEMENT OF DR. RITA R. COLWELL, DIRECTOR, NATIONAL SCIENCE FOUNDATION; ACCOMPANIED BY MR. THOMAS N. COOLEY, CHIEF FINANCIAL OFFICER, AND DIRECTOR, OFFICE OF BUDGET, FINANCE, AND AWARD ADMINISTRATION, NATIONAL SCIENCE FOUNDATION
Dr. COLWELL. Chairman Smith, and, Ranking Member Johnson, members of the Subcommittee, I am pleased to be here today to discuss the NSF's management and oversight of large facilities. I am very, very proud of the track record of the National Science Foundation. Over the past 50 years, the NSF has supported a number of large research facilities. These include the National Oceanographic Laboratory System, UNOLS, the Deep Sea Drilling Project, its successors, the U.S. Antarctic Research Program, which I believe is a stellar operated program, and supercomputing facilities.
In fact, we have recently made an award to provide large-scale computational infrastructure that will support virtually all fields of science and engineering. Currently, the NSF supports about or invests about $1 billion annually in facilities and other infrastructure projects.
The NSF's approach to facilities management differs from the other R&D agencies because NSF, as you pointed out, Mr. Chairman, does not directly construct or operate the facilities that it supports. Typically, the Foundation makes awards to universities or nonprofit organizations that undertake the construction and management and operation of the facility projects. However, NSF does retain the final responsibility for overseeing management and successful performance of the project.
Page 24 PREV PAGE TOP OF DOC
In setting priorities, the question is, is it the right thing to build? My written testimony provides a detailed analysis of our research facilities priority-setting process. That will be in the record I hope. And let me summarize just several points that are key to the process.
Major research facilities proposals undergo very rigorous evaluation and merit review, the most of any research activity that I am aware of. An idea for a major facility project has to survive intense scrutiny from within the community that proposes it. And then it passes through an external review and an evaluation by the NSF's senior executives. And some of the executives, the Assistant Directors, have requested support by competing projects within their area of responsibility—math, chemistry, physics, for example, in the mathematics, physical sciences directorate.
The Director or I then review the recommended projects that come up with proposed promising ones as candidates to the Science Board for further evaluation and consideration. And, finally, even the most meritorious candidates have to fit into the Administration's overall budgetary priorities for the NSF.
So the combined expertise and experience of the research and education community, the NSF Program Manager, the MRE Panel, the Deputy Director, the Director, the National Science Board—all of these have input into the—every single one of the final decisions. They evaluate each candidate project, not only from the intellectual merit of a project, but also to importance to science and engineering, potential costs of benefits, the need for balance across fields and disciplines, the connection to related projects in other agencies and in other nations, for that matter, and the readiness for implantation.
Page 25 PREV PAGE TOP OF DOC
Facilities management is the question of whether it is built right. After a project is selected and funded, we begin the critical job of ensuring that the research facility is constructed on time and within the cost estimates. Given the increasing complexity and scope of a facility, NSF has recognized the need to minimize risks by ensuring management and oversight benefit from contemporary best practices. To accomplish this, we have developed the Large Facility Projects Management and Oversight Plan. Actually, we have been working together, Joe Bordogna, the Deputy, and I, and our senior staff, for the last couple of years to address the issue of management and oversight.
We have shared drafts of the Plan with your staff, as well as with OMB, with the NSF's Assistant Directors, the Office of the Inspector General, the Science Board, and, on September 10, we will submit the Plan to OMB.
Now, oversight and decision authority is the question, is it run right? One purpose of the Plan is to ensure clear lines of authority, responsibility, and communication from the NSF Director to the NSF Program Manager to the awardee Project Director. And in every large facility project, the NSF Program Manager exercises primary responsibility for all aspects of project management. Working closely with the NSF Program Manager, the awardee designates a person, with strong management experience, as the Project Director, and this individual has overall control and responsibility for the project in the awardee organization.
Now, to enable the efficient evolution of NSF's large facility projects, NSF will establish will a new position for a Deputy, Large Facilities Projects, reporting to the Chief Financial Officer, who reports directly to me. Through the—a nationwide search, the NSF will recruit an individual with extensive experience in building, management, and oversight of large science and engineering facilities.
Page 26 PREV PAGE TOP OF DOC
The new Large Facilities Projects Deputy will provide a watchful eye over all the large projects to ensure that potential challenges are identified early on and handled quickly and effectively.
On the 21st of September, NSF is sponsoring a best practices workshop for NSF's management and oversight of large facility projects. We will have representatives from federal agencies involved in the management of large research facilities, and managers of existing facilities that we support, who will share their experiences in construction in operations of the facilities for which they are responsible. And I invite you and your members of the staff to attend that meeting on the 21st of September.
Now, in conclusion, Mr. Chairman, NSF is keenly focused on maintaining excellence in selecting, managing, and overseeing large facility projects for science and engineering. Our Facilities Management and Oversight Plan includes an aggressive schedule for developing and implementing measures to manage a growing number of complex, large projects that will enable us to remain on the forefront of science and engineering.
And, as one final point, I would like to say, Mr. Chairman, that in reviewing the history of the National Science Foundation and in reviewing some of my predecessor Directors' comments, it seems that the—this large facilities is a recurrent theme in discussion and has been for the last 50 years.
In 1955, the National Science Board adopted the following recommendations of a committee to which the NSF had reported. Mainly, that the NSF should recommend, as a national policy, the desirability of government support of large-scale, basic scientific facilities when the need is clear, when it is in the national interest, and when the merit is endorsed by panels of experts, and when the funds are not readily available from other sources. So we continue to evaluate and discuss the subject that is of intense interest and has been so for 50 years. Thank you.
Page 27 PREV PAGE TOP OF DOC
[The prepared statement of Dr. Colwell follows:]
PREPARED STATEMENT OF DR. RITA R. COLWELL
Chairman Smith, Ranking Member Johnson, Members of the Research Subcommittee, I appreciate the opportunity to be here today to discuss the National Science Foundation's management and oversight of large facilities. My colleagues, Dr. Boesz and Dr. Jones, will also provide comments.
INTRODUCTION:
Throughout its 50-year history, NSF has enjoyed an extraordinarily successful track record in providing state-of-the-art facilities for science and engineering research and education. NSF management and oversight have enabled not only the establishment of unique national assets, but have also ensured that they serve the science and engineering communities and the discovery process as intended.
NSF has enabled construction of unique and scientifically productive telescopes and managed their operation by funding non-governmental institutions created for that purpose. For decades, the University National Oceanographic Laboratory System of research ships has made outstanding contributions to multidisciplinary research and education worldwide and at the most remote regions of the Earth. The Deep Sea Drilling Project and its successors have operated with a truly exemplary record of scientific discovery and contract management.
Page 28 PREV PAGE TOP OF DOC
For more than 40 years NSF has managed the Nation's research program in Antarctica. Since the 1970s it has been widely recognized for its impressive management of the overall U.S. Antarctic Program that now encompasses planning and budgeting for logistics, facilities, and research support. More recently, NSF has enabled major advances in building and managing large-scale computational infrastructure supporting virtually all fields of science and engineering. Last month NSF awarded $53 million to a consortium of universities and institutes for a distributed terascale computing system.
NSF's approach to facilities management differs from most R&D agencies. NSF does not directly construct or operate the facilities it supports. Typically, the Foundation makes awards to external entities, primarily universities, consortia of universities or nonprofit organizations, to undertake construction, management and operation of facilities. NSF retains responsibility for overseeing the development, management and successful performance of the projects. NSF decision-making not only involves establishing new facilities but also reconstituting or phasing out support for facility projects that have reached the end of their useful lives.
These partnerships have proven remarkably effective in terms of overall cost and performance. The reasons include the talented program officers at NSF, the great creativity of the Nation's academic enterprise and the ''enlightened self-interest'' of a community wishing to use the facilities for research. An essential added benefit of NSF's model is the opportunity to train young scientists and engineers by allowing them direct involvement in planning, construction and operation of major facilities and large-scale instrumentation.
Currently, NSF invests over $1 billion annually in facilities and other infrastructure projects. Overtime, the portfolio of facilities has grown and diversified. For example, our portfolio now includes distributed projects that challenge traditional management and oversight approaches. With emerging multidisciplinary science and engineering opportunities, large facility projects are becoming increasingly complex and more challenging technical and management issues are on the horizon.
Page 29 PREV PAGE TOP OF DOC
Growth and diversification require that NSF remain attentive to the ever-changing issues and challenges inherent in planning, construction, operation, management and oversight of large facility projects. NSF's current review, management, and oversight processes are designed to ''build right those facilities that are the right ones to build.'' Today I will describe some additional steps we are taking to improve coordination, collaboration, and learning among NSF staff and external partners in order to facilitate a process of continuous improvement, based upon the learning of best practices.
PRIORITY SETTING: ''Is it the right facility to build?''
Many of NSF's large facility, projects are supported through the Major Research Equipment, or MRE, account—an agency-wide capital asset account that funds the construction or acquisition of projects with costs ranging from tens to several hundreds of millions of dollars. Large facility projects are also funded through the Research & Related Activities (R&RA), and/or the Education and Human Resources (EHR) accounts.
Currently, NSF is funding the following MRE projects:
Atacama Large Millimeter Array R&D (ALMA)
High-performance Instrumented Airborne Platform for Environmental Research (HIAPER)
Large Hadron Collider (LHC)
Page 30 PREV PAGE TOP OF DOC
George E. Brown Network for Earthquake Engineering Simulation (NEES)
Polar Support Aircraft Upgrades
South Pole Station Modernization (SPSM)
Terascale Computing Systems
The following projects have been included in prior budget requests and remain scientific priorities:
ALMA Phase II ($9 million for maintenance of the established infrastructure for ALMA was included in the Research and Related Activities account in our FY 02 request, pending a review of overall facilities management issues.)
EarthScope I (U.S. Array and San Andreas Fault Observatory at Depth)
National Ecological Observatory Network (NEON)
Finally, the following projects have been approved by the National Science Board and are priorities for consideration in future budget requests:
EarthScope II (Plate Boundary Observatory)
Ice Cube Neutrino Detector
Page 31 PREV PAGE TOP OF DOC
Ocean Observatories
Rare Symmetry Violating Processes
I would now like to explain how these and other candidate MRE projects are reviewed, prioritized and eventually selected for inclusion in NSF's budget request. (This process is more fully described in Guidelines for Planning and Managing the Major Research Equipment Account, which provide the basis for budgeting for MRE facilities, and can be found at http://www.nsf.gov/home/about/mre01.html)
First I would like to describe the criteria NSF uses to select large facility projects for inclusion in a budget request and to determine which projects it will eventually fund. These criteria are listed below. The first two are the general NSF merit review criteria used in the merit review process. The next six are additional criteria that NSF and the NSB use to review and approve projects for inclusion in a budget request. The relative importance of these criteria is not fixed but depends upon the nature of the facility project and the changing context that envelopes it.
Intellectual Merit
Broader Impacts
Importance to science and engineering
Page 32 PREV PAGE TOP OF DOC
Cost-benefit and risk analysis
Readiness to implement and manage
Appropriateness for NSF
Balance across fields and disciplines
Synergy with other large facilities supported by NSF, other agencies, and other nations
The chart below depicts NSF's process for the review and approval of large facility projects considered for funding through the MRE account. Several steps are involved as a project moves from an idea to a line in the NSF annual budget.
75811b.eps
The first step in the process is the early identification of an MRE Pipeline Project as such. These potential projects are conceived of in the science and engineering community, often as a result of emerging opportunities, and are often many years in development following initial conceptualization. Currently, NSF has about 20 projects in the discussion phase. These projects span the range from ''gleam in the eye'' to projects that are in the conceptual development stage.
At the conceptual development stage, an NSF Program Manager works with the scientific and engineering communities to develop a shared understanding of the project's concept, resource requirements, and schedule and performance issues. As the project moves toward a more formal definition, the NSF Program Manager consults with members of the scientific and engineering communities to help refine project design and definition and to formulate scientific/technical and management plans. It is important to note that, although NSF may be providing some funding for early planning and definition, it has not made any commitment to support the establishment of the facility.
Page 33 PREV PAGE TOP OF DOC
Developed projects are then proposed by an NSF Originating Organization(s) for consideration by the MRE Panel. These projects may be based on a proposal already submitted and evaluated using NSF's merit review process. When an Assistant Director believes a project is ready to be considered for inclusion in an NSF budget request, it may be brought to the MRE Panel for consideration. The project is considered by NSF's MRE Panel, which is chaired by the Chief Operating Officer and includes the Assistant Directors, Program Office Heads and the Chief Financial Officer. It is during these discussions that issues such as appropriateness for NSF, balance across fields and disciplines and readiness to implement and manage are first addressed.
The MRE Panel considers the projects on the basis of the review criteria specified earlier and makes recommendations to the Director. Using the review criteria, the Director selects candidates for the NSB to consider during one of its five meetings throughout the year. Once a project has been approved by the NS13, the Director, at her discretion, may recommend the project for inclusion in a future budget request to OMB. The project is generally considered a viable candidate until it is funded. However, significant changes to project definition, scope, and funding must be reviewed and approved by NSF and the NSB.
Once funded, changes in project cost are governed by a Delegation of Authority from the NSB. Under this authority, the Director may subsequently amend the award to commit an additional 20 percent or $10 million, whichever is less. Any funding that exceeds this requires additional NSB review and approval. Our policies also require that the Director be notified in writing, by the cognizant Assistant Director, prior to the expenditure of additional funding. NSB-approved awards are now coded in our electronic proposal and award system, and we have implemented an automated alert—for program managers and grants officers—identifying the awards and the NSB delegation conditions.
Page 34 PREV PAGE TOP OF DOC
OMB gives NSF budgetary guidance that reflects the Administration's priorities. The Director submits a proposed budget to OMB within the guidance—a budget that reflects priorities established in conjunction with the NSB. OMB and NSF then enter into a negotiation phase, which results in the priorities outlined in the final funding request. Following the appropriations process, the Director allocates funds to the relevant projects. If necessary, a program solicitation or RFP is prepared and, following receipt and merit review of the proposals, one or more awards are made.
This is the overall process by which large facility projects are selected and funded. It is important to note that during this process candidate facilities are not given specific priority ranks. In any given year, the MRE projects that appear in the President's budget request will depend on NSF's current assessment of opportunities and needs and the parameters provided by OMB. The highest priority projects appear in the budget request.
This process is very open and involves a great deal of interaction with the external scientific community and other stakeholders. This selection process has served NSF and the nation well in providing the most competitive state-of-the-art science and engineering facilities—ones that are certain to contribute significantly to the nation's research and education endeavors.
FACILITIES MANAGEMENT AND OVERSIGHT: ''Is it built right?''
Given the increasing complexity and scope of its facilities, NSF has recognized the need to mitigate risks by ensuring management and oversight benefit from contemporary best practices. Improving coordination, collaboration and shared learning among NSF staff and external partners facilitates identification and incorporation of best practices. To this end, and to comply with instructions in A Blueprint for New Beginnings: A Responsible Budget for America's Priorities (February 2001), NSF developed the Large Facility Projects Management & Oversight Plan.
Page 35 PREV PAGE TOP OF DOC
The Plan outlines NSF's goals and strategies for integrating its current procedures and processes into a next-generation system for selecting, managing and overseeing large facility projects. In answering the President's request, the Plan focuses on the management and oversight of large infrastructure and facility projects.
In its Plan, NSF addresses improvements in four areas:
Organizational and staff capabilities enhancement to improve coordination, collaboration and shared learning among NSF staff and external partners,
Comprehensive guidelines and procedures for all aspects of facilities planning, management and oversight,
Review and approval of large facility projects, and
Coordinated and proactive oversight of all facility projects to ensure success.
The draft Plan has been reviewed by OMB, NSF's Assistant Directors, the Office of Inspector General, and the NSB. On September 10th, the finalized plan will be submitted to OMB.
The Plan preserves clear lines of authority, responsibility and communication from the NSF Director to the NSF Program Manager to the awardee Project Director. In every large facility project, the NSF Program Manager exercises primary responsibility for all aspects of project management. Working closely with the NSF Program Manager, the awardee designates one person—with strong management experience—as the Project Director, who has overall control and responsibility for the project in the awardee organization. NSF vests responsibility for monitoring the business operations of large facility projects in its Chief Financial Officer (CFO). NSF personnel reporting to the CFO ensure that all policies, guidelines and procedures are followed and that the awardee is in compliance with the business, operations and financial requirements.
Page 36 PREV PAGE TOP OF DOC
To enable the efficient evolution of NSF's large facility projects from their pre-formulation through operations, NSF will establish a new position for a Deputy, Large Facilities Projects. The LFP Deputy will report directly to the CFO. Through a nationwide search, NSF will recruit an individual with extensive experience in the building, management and oversight of large science and engineering facilities.
The LFP Deputy will be responsible for:
Providing expert assistance to NSF's science and engineering staff on non-scientific aspects of project planning, budgeting, implementation, and management;
Developing, implementing and managing, with NSF-wide input and concurrence, management and oversight policies, guidelines and procedures; and,
Ensuring shared learning of best practices by fostering coordination and collaboration throughout NSF to facilitate application of lessons learned from each project.
The LFP Deputy will be supported by permanent NSF staff with experience in project management, planning and budgeting, cost analysis and oversight. The personnel will be a centralized resource assembled to assist NSF Program Managers with their management and oversight responsibilities and to develop and conduct post-award oversight of business operations, financial and internal control systems, and cost and schedule performance. The LFP Deputy and his or her staff will provide a watchful eye over all of the MRE projects and many of NSF's other facility projects to ensure that potential challenges are identified early and handled quickly and effectively.
Page 37 PREV PAGE TOP OF DOC
I would like to invite members of the committee or your staff to attend the best practices workshop for managing large facility projects that we have scheduled for September 21st. The workshop will focus on the development of comprehensive guidelines for NSF's management and oversight of large facility projects. Representatives from NSF and Federal agencies involved in the management of large research facilities, including Department of Energy and NASA, and managers of existing NSF and Federal facilities will share their experiences in construction and operation of the facilities for which they are responsible.
CONCLUSION:
As I have stated, we submit the NSF's Large Facility Projects Management and Oversight Plan to OMB on September 10. This Plan includes an aggressive schedule for developing and implementing each of the major components. We have worked closely with the National Science Board in the development of the plan, and will continue that close collaboration in its implementation. We are pleased with the dialogue we have had with this Committee, and with the OMB.
Mr. Chairman, in concluding, NSF is keenly focused on maintaining excellence in selecting, managing and overseeing large facility projects for science and engineering. Science and engineering facilities are fundamental and a much needed resource for our country. They expand the frontier of knowledge and are highly sophisticated in construction and operation. We are confident our current processes and the measures we have formulated clearly demonstrate NSF's leadership in planning, executing and overseeing a growing number of complex, large projects for science and engineering for our Nation.
Page 38 PREV PAGE TOP OF DOC
Thank you.
BIOGRAPHY FOR DR. RITA ROSSI COLWELL
Director, National Science Foundation, 4201 Wilson Boulevard, Arlington, Virginia 22230
Dr. Rita Colwell became the 11th Director of the National Science Foundation on August 4, 1998.
Since taking office, Dr. Colwell has spearheaded the agency's emphases in K–12 science and mathematics education, graduate science and engineering education/training and the increased participation of women and minorities in science and engineering.
Her policy approach has enabled the agency to strengthen its core activities, as well as establish support for major initiatives, including Nanotechnology, Biocomplexity, Information Technology, and the 21st Century Workforce. In her capacity as NSF Director, she serves as Co-chair of the Committee on Science of the National Science and Technology Council.
Under her leadership, the Foundation has received significant budget increases, and its funding recently reached a level of more than $4.5 billion.
Before coming to NSF, Dr. Colwell was President of the University of Maryland Biotechnology Institute, 1991–1998, and she remains Professor of Microbiology and Biotechnology (on leave) at the University Maryland. She was also a member of the National Science Board (NSF's governing body) from 1984 to 1990.
Page 39 PREV PAGE TOP OF DOC
Dr. Colwell has held many advisory positions in the U.S. Government, non-profit science policy organizations, and private foundations, as well as in the scientific research international community. She is a nationally respected scientist and educator, and has authored or co-authored 16 books and more than 600 scientific publications. She produced the award-winning film, Invisible Seas, and has served on editorial boards of numerous scientific journals.
She is the recipient of numerous awards, including the Medal of Distinction from Columbia University, the Gold Medal of Charles University, Prague, and the University of California, Los Angeles, and the Alumna Summa Laude Dignata from the University of Washington, Seattle.
Dr. Colwell has also been awarded 18 honorary degrees from institutions of higher education, including her Alma Mater, Purdue University. Dr. Colwell is an honorary member of the microbiological societies of the UK, France, Israel, Bangladesh, and the U.S. and has held several honorary professorships, including the University of Queensland, Australia. A geological site in Antarctica, Colwell Massif, has been named in recognition of her work in the polar regions.
Dr. Colwell has previously served as Chairman of the Board of Governors of the American Academy of Microbiology and also as President of the American Association for the Advancement of Science, the Washington Academy of Sciences, the American Society for Microbiology, the Sigma Xi National Science Honorary Society, and the International Union of Microbiological Societies. Dr. Colwell is a member of the National Academy of Sciences.
Page 40 PREV PAGE TOP OF DOC
Born in Beverly, Massachusetts, Dr. Colwell holds a B.S. in Bacteriology and an M.S. in Genetics, from Purdue University, and a Ph.D. in Oceanography from the University of Washington.
Chairman SMITH. Thank you very much. Dr. Jones.
STATEMENT OF DR. ANITA K. JONES, VICE CHAIR, NATIONAL SCIENCE BOARD
Dr. JONES. Chairman Smith, Ranking Minority Member Johnson, members of the staff, I appreciate the opportunity to testify. My comments will focus on the National Science Board's role as the governing board of the Foundation, but particularly on our oversight of the Foundation's large-scale research facilities.
First, I would like to emphasize that the Foundation has an excellent record, spanning 50 years, of supporting such facilities, both in terms of the quality of the research accomplished at those facilities, and in terms of the NSF management.
These facilities are quite diverse. For the most part, they house large research instruments that make possible research advances that can be accomplished in no other way. The facilities are large. They are complex. They are costly. They serve many researchers, often across disciplines. And often international collaboration is involved, and I will come back to that.
Page 41 PREV PAGE TOP OF DOC
Many research fields are on the edge of exciting discoveries that can only be made with such facilities. I anticipate a growth in that. And, as Ms. Johnson said, it is timely, the focus on NSF's management processes.
The National Science Board plays a crucial role in the oversight of large facilities. The Board takes two kinds of formal actions, although it is routine at almost each meeting for either the Board or some of its committees to discuss one or more large facility projects. Those two kinds of formal actions are approval of all large funding awards over a defined threshold, and, second, the approval or the authorization of candidates for the Major Research Equipment account. And I want to talk about each one.
As I said, the Board approves funding for all major projects, including facilities. And by major, that means, one whose cost exceed 1 percent of the budget of the sponsoring directorate or office. And that approval for funding is for a limited time. The Board also approves the major programs whose budget exceeds three percent of the budget of the sponsoring directorate or office.
We have a process utilizing the Committee on Programs and Plans to review large projects at various stages of their development and implementation. That committee makes recommendations to the full Board for the initiation of new awards and new programs. In addition, the Committee on Programs and Plans reviews projects for adherence to the NSB-approved criteria for merit review and the Board's policy regarding competition, recompetition, and renewal.
The Board's Committee on Audit and Oversight reviews specific financial and business management issues raised either by the Inspector General or by the Foundation management.
Page 42 PREV PAGE TOP OF DOC
Now, that is approval of major awards. Let me turn to candidates for the Major Research Equipment Account, the capital asset account. The Director brings facility projects to the Board for consideration. If the Board approves such a project, and it is like an authorization of a candidate project, that project may be included in future budget requests. All the projects authorized by the Board are excellent and worthy of Foundation support. In giving approval, the Board considers elements, such as intellectual merit, the societal impact of the projects, their importance to engineering and science, balance across disciplines of the many facility projects, readiness to be implemented, and cost-benefit and risk analyses.
The Board, as you have observed, does not rank-order candidate projects. This gives the Foundation flexibility. When the Board approves the Foundation's budget submission, it is reaffirming its support for the MRE projects that are included for funding.
As I said before, before a project is actually funded, it comes back to the Board for approval of funding at a specific level. That approval is for the initial construction. And the Board's Committee on Programs and Plans reviews a project's progress at the midpoint of construction and whenever significant issues arise. The Director comes back to the Board if it appears that there will be an exceeding or projected exceeding of the approved dollar amount by more than 20 percent or $10 million, whichever is less.
Now, let me pause. There has been a lot of discussion of a process. There is a prioritization process. It is collegial. It involves the community, the National Science Foundation, but it is the Board that makes priority decisions. Given all the set of projects that could be considered, the Board approves a number, certainly a smaller—much smaller number than are being thinked about—or thought about—and those are the authorized candidate projects.
Page 43 PREV PAGE TOP OF DOC
Historically, all of those MRE projects are eventually funded. The Board does not rank-order the projects, and that is for a very good reason. If you had a small number of priority approved or authorized projects, but there may be changes in the performing organization, for example, loss of personnel, the arrangement of international co-funding may take more time than expected to put into place, and there is discussion with the OMB, for example, on the size of the MRE account in any given year. If there were a rank-order, for all three of these reasons, it might be appropriate to reorder such a list, and there would be must consternation and, in fact, anger, I think, in the community. So I think we are wiser to have a small priority group, but allow the Foundation flexibility for the reasons that I just enunciated. So there is a prioritization of projects, but is a prioritization by group, and the Board believes that is the best way to move forward.
As you have mentioned, Mr. Chairman, there is now a draft Oversight Plan by the Foundation. The Board saw that Plan this summer. The Board received a briefing on the Plan and was pleased with the direction, the framework, and the elements set forth. That Plan will codify practices in place. It will ensure project management stays with the scientists and engineering—engineers. But, in particular, it will strengthen Foundation oversight of financial and business functions. Dr. Colwell mentioned the creation of the Deputy for Large Facility Projects in the line of authority underneath the Chief Financial Officer.
So, in summary, Mr. Chairman, the Board supports the general direction of this Plan. The Board will assess the Foundation's projects—progress in refining and implementing the Plan, particularly to ensure the integrity of the evaluation and oversight of financial and business aspects. Thank you for the opportunity to present these remarks. And I would be glad to answer questions.
Page 44 PREV PAGE TOP OF DOC
[The prepared statement of Dr. Jones follows:]
PREPARED STATEMENT OF DR. ANITA K. JONES
Chairman Smith, Ranking Minority Member Johnson, members of the Committee, I appreciate the opportunity to testify today. My name is Anita Jones. I am Vice Chair of the National Science Board and Chair of the Board's Committee on Strategy and Budget. I am also the Quarles Professor of Engineering and Applied Science at the University of Virginia. From 1993 to 1997 I served as Director of Defense Research and Engineering at the U.S. Department of Defense. In that position I was responsible for the science and technology program of the Department of Defense, including the Defense Advanced Research Projects Agency and oversight of the Department's laboratories.
On behalf of the National Science Board, I thank the Committee for its long-term support for science and engineering research and education activities, which have contributed so substantially to our Nation's well being.
The National Science Board has two statutory roles: to serve as the governing board of the National Science Foundation, and to advise the Congress and the President on national policy issues for science and engineering research and education.
Today, my comments will focus on the Board's role as governing board of the Foundation, specifically on our oversight and approval of the Foundation's support for large-scale research facilities.
Page 45 PREV PAGE TOP OF DOC
First, I would like to emphasize that the Foundation has an excellent record—spanning 50 years—of supporting such facilities, in terms of both the quality of their research and their management. Today, NSF invests over $1 billion annually in facilities and other infrastructure projects. With the exception of U.S. research facilities in the Antarctic, which are directly operated by the National Science Foundation, NSF typically makes awards to other organizations for the construction and operation of facilities.
The following are examples of major facilities:
The Large Hadron Collider is a superconducting particle accelerator. Its purpose is to help scientists advance the fundamental understanding of matter. The Collider's construction and operations are funded through an international collaboration.
The Laser Interferometer Gravitational-Wave Observatory, or LIGO, will allow physicists and engineers to collaborate to test the dynamic features of Einstein's theory of gravity and to study the properties of intense gravitational fields.
The National Astronomy Center in Arecibo, Puerto Rico, supports observations in radio and radar astronomy and atmospheric sciences.
Research facilities at the South Pole Station, currently under renovation, support a variety of diverse but important research activities that can only be conducted in the unique cold and pristine environment at the South Pole.
Page 46 PREV PAGE TOP OF DOC
The Ocean Drilling Program, involving 20 countries, supports research in areas including deep ocean structures, hydrology and geochemical cycles.
These five examples are all major research facilities. For the most part, they are the research instruments that make possible research advances that can be accomplished in no other way. They are all large; each one opens new research frontiers that could not be entered without these tools. They are complex; each one involves challenging engineering tasks in its design, construction and operation. Hence, each is very costly. As I have illustrated, the U.S. frequently teams with international collaborators, not just to assure that the best research is pursued, but also to help make the facilities more affordable. And, very importantly, each facility has a very broad base of researchers who are the users; they frequently come from multiple disciplines.
Many scientific fields are on the edge of exciting discoveries that require such facilities. I anticipate that in the 21st century, the need for such large, complex research facilities will grow. In recent years, the Foundation's portfolio of facilities has grown and diversified to include distributed projects and complex multidisciplinary projects like terascale computing systems and ocean observatories that challenge traditional management and oversight approaches.
National Science Board's Oversight of Large Facility Projects
The National Science Board plays a critical role in the oversight and approval of large NSF-supported facilities. The NSB is well constituted to exercise its oversight and approval responsibilities. Members of the Board include executives from industry and presidents of universities, individuals who have extensive experience in managing large, cutting-edge research facilities and instrumentation. Of course, the Board includes members who have used such facilities.
Page 47 PREV PAGE TOP OF DOC
The Board conducts two activities that focus on the approval and oversight of facilities. They are the approval of large awards and the approval of candidates for the Major Research Equipment account. Typically, the Board hears briefings from NSF management at almost every NSB meeting on the subject of large facilities—existing and candidate.
NSB Approval for Major Awards:
The Board approves all major projects, including facilities, whose costs exceed one percent of the budget of the sponsoring directorate or office budget. The Board also approves new major programs whose budget exceeds three percent of the budget of the sponsoring directorate or office.
The Board's Committee on Programs and Plans (CPP) reviews large projects at various stages of their development and makes recommendations to the full Board for the initiation of new awards and programs. In addition, the CPP reviews projects for adherence to the NSB approved criteria for merit review and the Board's policy regarding the competition, recompetition and renewal of NSF awards. Throughout the implementation phase of a project, the CPP reviews its progress and informs the Board of its status and any issues that arise.
The Board's Committee on Audit and Oversight (A&O) reviews specific financial and business management issues raised by the Inspector General and by Foundation management. Like the CPP, A&O informs the Board of any issues that arise.
The Major Research Equipment Account:
Page 48 PREV PAGE TOP OF DOC
The Major Research Equipment (MRE) account is an agency-wide capital asset account used to fund major science and engineering infrastructure projects that cost far more than one program's budget could support. The costs of MRE facilities range from several tens to hundreds of millions of dollars. The Board sees these projects multiple times over their lifetime. The Board takes two kinds of actions. First, it authorizes a candidate project for possible inclusion by the Foundation in a future budget. Later, the Board approves specific funding for an organization or consortium to design and construct the facility.
Let me briefly outline how the Board oversees MRE projects. The Director selects candidates for the NSB to review during one of its five meetings throughout the year. The Board receives, for approval, candidate projects that may be included in a future budget request, subject to availability of funding. Board authorization signifies that the projects are meritorious and that planning is sufficiently advanced to justify funding. In giving its approval, the Board considers the intellectual merit, societal impacts of the projects, their importance to science and engineering, balance across disciplines, readiness to be implemented, and cost-benefit and risk analyses.
The Board authorizes the MRE projects for possible inclusion in future budgets, but does not rank-order them to preserve the Foundation's flexibility in a given budget year. We believe that all projects authorized by the Board are of unquestioned excellence and worthy of Foundation support. When the Board approves the Foundation's budget submission to the Office of Management and Budget, it reaffirms its support for any MRE projects included for funding.
Page 49 PREV PAGE TOP OF DOC
After NSF has run a competition and detailed plans are in place for design, construction and operations, the project comes back to the Board for the award of funding at a specific level. Board oversight of MRE projects continues after an award is made. The Board's CPP reviews a project's progress at the midpoint of construction and whenever significant issues arise. If it appears that a project will exceed the Board's approved dollar amount by over 20 percent or $10 million, whichever is less, the Director must return to the Board to request approval for a higher level of funding. CPP also receives periodic reports on the status of all major projects.
NSF Large Facility Projects Management and Oversight Plan
As part of its oversight responsibilities, the Board began a dialogue with Foundation management about large facility issues more than a year and a half ago. Together, we have been discussing the improvement of the process for identifying candidate projects and for Foundation oversight of the management of construction and operation of such facilities.
The Foundation has created a new Large Facility Projects Management and Oversight Plan. That Plan, which was requested by the Administration, does two things. It incorporates and builds upon an existing facility management process. In addition, it strengthens financial oversight.
I would like to comment briefly on the Board's participation in the development of the Plan. The full Board received a draft Plan for comment this summer. At the August 8–9 Board meeting, our Committee on Programs and Plans, which has the responsibility to review major projects and facilities, received a briefing on the Plan from the Deputy Director. Members of our Audit and Oversight Committee participated in those discussions. Board members were pleased with the direction, framework, and elements set forth in the briefing and encouraged Foundation management to proceed with the Plan's development. The Board will continue to assess the Foundation's progress in refining and implementing the Plan.
Page 50 PREV PAGE TOP OF DOC
The Plan institutionalizes and builds on long-standing management practices.
(1) It codifies sound practices already in use, augments the existing MRE process, and documents principles of management.
(2) It ensures that project management will stay with the scientists and engineers, from planning through operation. The overall NSF Program Manager for a particular facility project is an individual in one of the research directorates of the Foundation.
(3) It strengthens Foundation oversight of financial and business functions. This requires organizational and managerial changes within the Foundation. In particular, it calls for the creation of a Deputy for Large Facility Projects who reports directly to the Chief Financial Officer and is responsible for ''developing, implementing and managing, with NSF-wide input and concurrence, management oversight policies, guidelines and procedures.''
In summary, Mr. Chairman and Members of the Committee:
The National Science Board supports the general direction laid out in NSF's Large Facility Projects Management and Oversight Plan. The Board will assess the Foundation's progress in refining and implementing the elements of the Plan, particularly to ensure the integrity of the evaluation and oversight of the financial and business aspects of the facility project throughout its life.
Page 51 PREV PAGE TOP OF DOC
The implementation of the Plan will ensure that the Foundation, with Board oversight, has the policies and organization required for sound management of unique, complex, world-class research facilities.
Thank you for the opportunity to present these remarks. I would be glad to answer any questions you may have.
BIOGRAPHY FOR ANITA K. JONES
Anita Katherine Jones was born in Fort Worth, Texas. She received a Bachelor's degree from Rice University in mathematics, a Master of Arts degree in literature from the University of Texas, Austin, and a Ph.D. degree in computer science from Carnegie Mellon University. She then joined the faculty at Carnegie Mellon University. In 1988, Jones became a professor and chair of the Department of Computer Science at the University of Virginia. Her research focuses on the design and implementation of programmed systems on computers, including enforcement of security policies on computers, operating systems, and scientific databases. She has published two books and more than 40 technical articles in the area of computer software and systems.
In 1993, Jones was sworn in as Director of Defense Research and Engineering at the U.S. Department of Defense. In 1997, she returned to the University of Virginia as a University Professor. She was later appointed as Lawrence R. Quarles Professor of Engineering and Applied Science. She currently serves on the Board of Directors for Science Applications International Corporation, and as a member of the Defense Science Board, Business Executives for National Security, and the Council on Foreign Relations.
Page 52 PREV PAGE TOP OF DOC
Jones was elected to the National Academy of Engineering in 1993. She is a fellow of the Association for Computing Machinery (1995), the Institute of Electrical and Electronics Engineers (1996), and the American Academy of Arts and Sciences (2000). She received the Department of Defense Award for Distinguished Public Service in 1996 and the Distinguished Service Award from the Computing Research Association in 1997. She was appointed to the National Science Board in 1998 and elected Vice Chair in 2000.
Chairman SMITH. Before you make your comment, Dr. Boesz, Representative Johnson.
Ms. JOHNSON. Thank you, Mr. Chairman. I apologize for having to run to a meeting to look after my district. But I wanted to call on and give attention to the fact that the three major witnesses are female today.
Chairman SMITH. I didn't notice.
Ms. JOHNSON. Thank you.
Chairman SMITH. Dr. Boesz.
STATEMENT OF DR. CHRISTINE C. BOESZ, INSPECTOR GENERAL, NATIONAL SCIENCE FOUNDATION
Dr. BOESZ. Chairman Smith, Ranking Member Johnson, and, distinguished members of the Subcommittee, I appreciate the invitation to appear before you today. As the National Science Foundation continues to increase its investments in large infrastructure projects, its oversight of timely and cost-effective construction management, while achieving the intended science, becomes ever more important. I am pleased to inform you that NSF and my office are working together on this challenge.
Page 53 PREV PAGE TOP OF DOC
Following an audit of Gemini Telescopes last year, my office recommended that NSF develop clearer policies and procedures for managing all aspects of large facility projects, emphasizing fund control and effective project management. In developing these recommendations, my staff interviewed NSF employees and reviewed its current written policies and procedures, and we also looked at the guidance used in other federal agencies.
In addition, the President mandated, in his budget blueprint, that NSF develop a plan to enhance the capability to estimate costs and to provide oversight of large facility project development and construction to ensure that NSF was able to adhere to cost and schedule commitments.
In response to both the President's mandate and our audit recommendations and Dr. Colwell's general concerns, NSF has been developing a Large Facility Project Management and Oversight Plan. This Plan covers both pre-award and implementation phases. My comments are focused on the implementation phase.
I believe that the most constructive role for my office is to ensure that sound business and management practices are in place in order to advance NSF's scientific goals. The Plan clearly represents progress, particularly in raising the profile of project construction and management within the agency. However, key areas of implementation still need to be addressed.
First, NSF should clarify who will have responsibility for final oversight and accountability for each large facility project. The Plan envisions two distinct organizational structures, one for scientific project management and the other for administrative oversight. There may be times when there will be differences on how to proceed. So clarity is needed on how any differences will be resolved.
Page 54 PREV PAGE TOP OF DOC
Second, the Plan specifies an awardee institution will manage all aspects of project implementation and report to NSF on its progress. Clarification is needed on how NSF will ensure that an awardee has the necessary project implementation capabilities. I believe that such capabilities could even become a decision criteria for funding. Because NSF relies heavily on the management skills of its awardees, we recommend that NSF evaluate the required awardee management Plan as an interval part of the selection process.
Third, the Plan should include information on a comprehensive training program for all aspects of large facility construction and management. Training should address basic project and financial management skills, as well as NSF's new guidelines and procedures as they are developed and implemented.
Fourth, once NSF makes an award, it will need to ensure that the management Plan is implemented and that its results are monitored. The management Plan is a focus point that could and should be used as a benchmark in an ongoing oversight process. While a thorough and effective planning process is critical, active and ongoing project oversight is equally important.
Finally, I am mindful that NSF has set for itself ambitious and challenging milestones for implementation of the Plan. Because of the importance of this effort, NSF should continue to use the necessary expertise to develop guidelines and build project management capabilities while it recruits for a Deputy for Large Facility Projects.
As NSF moves forward with implementation of its Plan and the corresponding guidelines and procedures, my office will continue to have a role. First, because the Plan was partially developed in response to an audit recommendation, we will conduct a follow-up review to ensure that it has been fully implemented.
Page 55 PREV PAGE TOP OF DOC
Second, my office will periodically review NSF's oversight process and, if necessary, make recommendations for improved efficiency and effectiveness. I believe that it is NSF's responsibility to manage and oversee its awards, including assessing whether performance project funds are adequate to achieve scientific goals, ensuring that awards are—awardees are exercising proper project management, determining whether an entity has proper accounting systems in place, and ensuring that the scientific goals of the project are achieved.
NSF should conduct regular reviews to make certain that the management plans submitted to and approved by NSF are being followed or that variances are justified. Because management of large scientific facilities is one of the agency's top ten management challenges identified by my office, we will continue to make these projects priorities in formulating future audit work.
In addition, my office will remain available to assist in specific areas for which we have expertise, such as pre-award reviews of budgets and indirect cost rates.
In conclusion, I want to emphasize that NSF's development of this Plan is an important first step to ensuring that these projects provide their intended research benefit while also providing appropriate stewardship over public funds. I recognize that balancing these two objectives may sometimes be difficult. This Plan lays the groundwork for NSF's efforts to strike this balance and thereby meet one of its management challenges.
Mr. Chair, that concludes my statement. Thank you for the opportunity to share this information with you. I would be pleased to answer any questions that you may have.
Page 56 PREV PAGE TOP OF DOC
[The prepared statement of Dr. Boesz follows:]
PREPARED STATEMENT OF CHRISTINE C. BOESZ
Chairman Smith, Ranking Member Johnson, and distinguished Members of the Subcommittee, I appreciate the opportunity to appear before you today. As you know, the National Science Foundation (NSF) is an innovative agency dedicated to maintaining American leadership in discovery and the development of new technologies across the frontiers of scientific and engineering research and education. As the scientific enterprise changes and research evolves, new challenges arise. Consequently, my office has worked closely with NSF management to identify and begin to address issues that are important to the success of NSF achieving its mission. One of these areas is the construction and management of large facility projects.
NSF is increasing its investments in large infrastructure projects such as accelerators, telescopes, research vessels, supercomputers, digital databases, and earthquake simulators. Currently, NSF spends approximately $1 billion per year for such cutting-edge projects, some of which individually cost hundreds of millions of dollars. Many of these projects are large in scale, require complex instrumentation, and involve partnerships with other Federal agencies, international science organizations, and foreign governments. Some, such as the new South Pole Station, present additional challenges because they are sited in harsh and remote environments.
Awards for large facilities (i.e., Tools) are inherently different than those that NSF makes to single investigators for individual research projects (i.e., Ideas).(see footnote 1) In exploring Ideas, researchers need to be given the freedom and independence to allow their research to evolve and move in new directions. In large facility projects, or Tools, that same degree of freedom may sometimes be at odds with cost and schedule requirements. In one of our recent audits, we found evidence that the tension between research interests and cost and schedule requirements delayed the delivery of key scientific instrumentation. In her book Einstein's Unfinished Symphony: Listening to the Sounds of Space-Time, Marcia Bartusiak describes this tension as it relates to the NSF-funded Laser Interferometer Gravitational Wave Observatory (LIGO). Bartusiak states that
Page 57 PREV PAGE TOP OF DOC
''[r]esearch scientists, long used to laboratory independence—the freedom to change an experiment at will—became upset over LIGO's rigorous schedules and their inability to make last-minute changes in the instrument. Scientific considerations suddenly had to bow to financial and engineering concerns. This meant that certain technologies had to be locked in early, even though advances may have developed later. Some adapted; others left.''(see footnote 2)
Overseeing the construction and management of these large facility projects and programs, while still being sensitive to the scientific endeavor, requires much more diligence than simply allowing for research independence and freedom. It requires a disciplined project management approach including meeting deadlines and budgets, and working hand-in-hand with scientists, engineers, project managers, and financial analysts. Furthermore, although NSF does not directly operate or manage these facilities, it is NSF that is ultimately responsible and accountable for their success. Consequently, NSF must exercise proper stewardship over the public funds invested in these large projects.
NSF Needs Strong Management of Large Facility Projects
NSF's management of large facility projects has received increased attention recently. Last year, my office conducted an audit of one of these large projects and made several recommendations to NSF management. NSF has certain policies and procedures governing large projects, but we found that these were not comprehensive and did not cover all key aspects of managing such projects. We also found that important information about the status of projects did not always reach appropriate senior policy makers. We recommended that NSF develop strong policies and procedures for managing all aspects of large facility projects, including giving more extensive and higher level NSF management responsibility for oversight of Major Research Equipment (MRE) appropriation account projects, and training staff involved with these large projects on fund control and effective project management. In developing these recommendations, my staff interviewed NSF employees currently working on large facility projects and reviewed NSF's current written policies and procedures. We also researched other Federal guidance available, such as the General Accounting Office's Executive Guide: Leading Practices in Capital Decisionmaking,(see footnote 3) Office of Management and Budget Circular A–11 Capital Programming Guide(see footnote 4) and project management structures in place at other agencies such as the National Aeronautics and Space Administration(see footnote 5) and the Department of Energy.(see footnote 6)
Page 58 PREV PAGE TOP OF DOC
Shortly after the release of our report,(see footnote 7) the President mandated in A Blueprint for New Beginnings: A Responsible Budget for America's Priorities that NSF develop ''a plan to enhance its capability to estimate costs and provide oversight of [large facility] project development and construction. This plan should help ensure that NSF is able to meet and stick to cost and schedule commitments for major facility projects.''(see footnote 8)
Because of the need for increased management and oversight of large facility projects, and because of the increasing level of funding being allocated to these projects, my office last year identified large facility project management as one of NSF's top ten management challenges.(see footnote 9)
Facilities Management and Oversight Plan
In response to both the President's mandate and recommendations in my office's December 2000 report, Audit of the Financial Management of the Gemini Project, NSF has been developing a Facilities Management and Oversight Plan (Plan). This Plan, while still in draft form,(see footnote 10) is a good start on the road to addressing NSF's large facility project management needs. The Plan has four major goals: (1) to address organizational needs within NSF to effectively manage large facility projects; (2) to implement guidelines and procedures for all aspects of facilities planning, management, and oversight; (3) to improve the process for reviewing and approving large facility projects; and (4) to properly oversee facility projects to ensure their success.
Page 59 PREV PAGE TOP OF DOC
First, the Plan primarily addresses organizational changes by creating a new position, Deputy CFO for Large Facility Projects, that will lead a Business Oversight Team. This Deputy is intended to be a point of accountability for the non-scientific aspects of large projects. The Business Oversight Team will consist of permanent NSF staff directly supporting the Deputy CFO for Large Facility Projects teamed with experts from other non-scientific NSF Divisions and Offices. NSF also plans to use Project Advisory Teams to give advice and assistance to the project managers responsible for individual large facility projects in the planning, review, management, and oversight of the project to assure cost, schedule, and technical performance. These Teams will be led by the NSF program officer, who is organizationally within an NSF scientific directorate or office, and will include members of the Business Oversight Team. In addition, the Business Oversight Team is expected to facilitate interactions and learning across all the Project Advisory Teams. Through the creation of these multidisciplinary teams, NSF hopes to enhance oversight management practices.
Second, NSF intends to develop new and refine existing guidelines and procedures for oversight and management of large facility projects. These guidelines are intended to be applicable to all large facility projects, cover the entire project life cycle, and give greater emphasis to post-award project oversight of construction and management. NSF also plans for these guidelines and operational procedures to be incorporated into a Large Facility Projects Manual for use by NSF staff working with these projects.
Third, NSF intends to strengthen its predecisional review process as it relates to large facility projects. To achieve this, NSF plans to require all large facility project proposals to undergo a predecisional review and approval process similar to that which is in place currently for MRE projects. NSF will develop criteria for making funding decisions for large facility projects and also plans to require that all proposals for large facility projects include a management plan. I believe that such a management plan is a keystone for successful project implementation and future monitoring of progress during the project implementation phase.
Page 60 PREV PAGE TOP OF DOC
Fourth, NSF plans to develop a model template for post-award project review and oversight. This is intended to be an adaptable model of protocols for oversight reviews of large facility projects. Using these protocols, teams of NSF staff comprised of scientific, technical, management, business, and legal experts will conduct regular oversight reviews. Through these reviews, NSF hopes to identify and address potential performance and financial risks before they become problematic.
The Plan also includes milestones for its implementation. Currently, these milestones have NSF initiating its search for the Deputy CFO for Large Facility Projects in September 2001. By the end of January 2002, NSF plans to have developed the comprehensive guidelines and begin project oversight reviews. Finally, NSF plans to produce its Large Facility Projects Manual by the end of February 2002.
OIG Comments on the Plan.
NSF has sought my office's input as it has developed the Plan over the past several months, and we have provided comments to NSF throughout the process. We are pleased to see that NSF has found our audit recommendations helpful and we are looking forward to full implementation of the Plan. We also recognize and appreciate that NSF is undertaking a significant task. This process is an important investment. Although there are costs now in terms of time and resources, the returns are sure to be significant. NSF will complete this process with a greater understanding of the potential challenges involved with these projects and how best to avoid and/or overcome them. The research community, and the public at large, will be able to have confidence that the large facility projects NSF undertakes are managed to ensure the accomplishment of the intended research while maintaining accountability for public funds.
Page 61 PREV PAGE TOP OF DOC
While the Plan covers both pre-award and implementation phases of large facility projects, my comments are generally focused on issues concerning the implementation phase. I believe that my office can be most useful in helping NSF put sound business and management practices in place that ensure that the intended scientific research is achieved with the funding NSF provides, not in evaluating the scientific merits of proposed projects. This Plan, as the blueprint for NSF's oversight of large facility project construction and management, lays the groundwork for all that is to come. In general, the Plan represents progress, particularly in terms of raising the profile and importance of project management within the agency. However, NSF will still need to address some key questions concerning implementation.
First, NSF should clarify who will have final oversight and accountability responsibility for each large facility project. The Plan envisions two organizational structures: one for scientific project management and the other for administrative oversight. For many issues that arise during the life of these projects, one or the other of these organizational components should be able to be address them. However, as we have already seen, there are times when these two functional structures will tend to differ on how to proceed. Although the Plan does establish Project Advisory Teams to integrate these two viewpoints, these teams are advisory in nature and will be led by the cognizant program manager. It is important that NSF clearly identify the ultimate authority so that it can strike the appropriate balance between good science and good project management. The Plan should address this key issue and provide guidance for how conflicting views on science, construction, management, and funding will be resolved.
Second, although the Plan specifies that the awardee institutions will manage all aspects of project implementation and report to NSF, there is only a general indication of how NSF will ensure that awardees have the necessary project implementation capabilities (i.e., construction and management). In fact, project implementation capability is not currently listed in the Plan as a criterion for a funding decision. Although the Plan states that NSF will require every large facility proposal to contain a management plan that covers all phases of the project, it does not specify whether or how this plan will be evaluated or used. Further, it appears that the management plan will be approved after the award is made. Because NSF relies heavily on the project management skills of its awardees in conducting the projects, we recommend that NSF evaluate and consider the management plans as part of the selection process, prior to handing over Federal dollars.
Page 62 PREV PAGE TOP OF DOC
Third, the Plan should include detailed information on a comprehensive training program. One of recommendations coming out of my office's audit of the Gemini Project focused on the need for project management training for NSF staff engaged in large facility projects. Training is mentioned briefly in the Plan but there are no details as to who will be trained, by whom, and on what subjects. NSF should develop a comprehensive training program for its staff working on all aspects of large facility projects. This program should include training on basic project and financial management skills as well as specific training on NSF's new guidelines and procedures as they are developed and implemented.
Fourth, once NSF makes an award, it will need to ensure that the management plans are implemented and monitored as part of its ongoing oversight process. While a thorough planning process is critical, active and ongoing project oversight is equally important. The Plan includes, in an appendix, a graphic description of NSF's pre-award review and approval process currently in place for MRE projects, with a brief description of intended post-award activities. While the Plan indicates that oversight teams will conduct progress reviews, there is no detail on the scope of these reviews (e.g., content and frequency) and the role their results will play in taking the necessary actions to ensure the successful and timely completion of the projects. The Plan should clarify how these reviews could lead to changes in the management of the particular project being reviewed and how NSF will use the project management plans to monitor construction progress and assess ongoing management capability.
Finally, we note that NSF has set for itself ambitious and challenging milestones for implementation of the Plan. The Plan states that the Deputy CFO for Large Facility Projects will lead the effort to develop and implement the guidelines and begin the oversight reviews. These guidelines are expected to be developed and reviews started by the end of January 2002. However, the Deputy has not yet been recruited. Conducting a ''nationwide search'' for this position is an effort that could take a significant amount of time, and waiting for this person to develop the guidelines for reviewing projects may not allow NSF to meet its intended milestones. Because of the urgent nature of implementing this Plan, NSF should consider gathering the necessary expertise now, on a temporary basis if necessary, to develop the guidelines, rather than awaiting the arrival of the Deputy. This may give NSF the opportunity to meet its ''aggressive schedule.''
Page 63 PREV PAGE TOP OF DOC
Continuing OIG Role
As NSF moves forward with implementation of its Plan and the corresponding guidelines and procedures for large facility project management, my office will continue to have a role in the process. First, because the Plan was partially developed in response to an audit recommendation, my office will conduct a follow-up review to ensure that the Plan has been fully implemented and is serving its intended purposes.
Second, my office will continue to monitor the agency's management of large facility projects. I believe that it is NSF's responsibility to manage and oversee its awards, including determining whether proposed project costs are realistic, ensuring that awardees are exercising proper project management, determining whether an entity has proper accounting systems in place to adequately account for Federal funds, and ensuring that the scientific goals of the project are achieved. NSF should conduct reviews of ongoing large facility projects to ensure that the awardees are following the management plans submitted to and approved by NSF. This should be a key component of NSF's oversight process. My office will periodically revisit issues related to NSF's award oversight process and, where necessary, make recommendations for improved efficiency and effectiveness. Because management of large infrastructure projects is one of the agency's top ten management challenges, we will continue to prioritize these issues in formulating future audit work.
My office will continue to be available to NSF, upon request, to assist in specific areas for which we have particular expertise. For example, we have recently conducted, at NSF's request, a review of an awardee's proposed indirect cost rate to be applied to a large facility project. We have also in the past conducted proposal reviews and will continue to be available to NSF, resources permitting, for these types of consultations.
Page 64 PREV PAGE TOP OF DOC
Finally, we will continue to carry out our statutory role of detecting and deterring fraud, waste, and abuse. We are confident that if significant and serious problems arise with large facilities or any other projects, NSF will continue to refer these matters to us. Our doors are always open and we have telephone and e-mail hotlines available for such reports.
Conclusion
In conclusion, the Large Facility Projects Management and Oversight Plan is an important first step in the process of ensuring that NSF's large facility projects provide their intended research benefits while also providing appropriate stewardship over public funds. We look forward to NSF addressing the fundamental issues of accountability, authority, and post-award project management. We also look forward to NSF meeting its timelines while producing a thorough and complete end-product. This Plan will lay the groundwork for all of NSF's efforts in this area and as such provide a blueprint for future actions. The NSF Plan represents progress toward that end.
Mister Chair, that concludes my statement. Thank you for the opportunity to share this information with you. I would be pleased to answer any questions that you may have.
SUMMARY STATEMENT OF DR. CHRISTINE C. BOESZ
The National Science Foundation continues to increase its investments in large infrastructure projects. Overseeing the timely and cost effective construction and management of these large facility projects and programs, while serving the scientific endeavor, requires a disciplined project management approach including meeting deadlines and budgets, and working hand-in-hand with scientists, engineers, project managers, and financial analysts. To address this management challenge NSF has been developing a Facilities Management and Oversight Plan. While the Plan covers both pre-award and implementation phases of large facility projects, my comments on this Plan are focused on issues concerning the implementation phase. In general the Plan represents progress. However, NSF will still need to address some key questions concerning implementation. First, who will have final oversight and accountability responsibility for each large facility project. Second, how will NSF ensure that successful awardees have the necessary project implementation capabilities. Third, what are the details of a comprehensive training program. Fourth, how will NSF ensure that the management plans are implemented and monitored as part of its ongoing oversight process. NSF's Large Facility Projects Management and Oversight Plan is an important first step in the process of ensuring that large facility projects provide their intended research benefits while also providing appropriate stewardship over public funds.
Page 65 PREV PAGE TOP OF DOC
BIOGRAPHY FOR CHRISTINE C. BOESZ
Christine C. Boesz assumed the duties as Inspector General of the National Science Foundation (NSF) in January 2000, reporting to the National Science Board and the Congress. As head of the Office of Inspector General (OIG) she recommends policies for promoting economy, efficiency and effectiveness of NSF programs and operations. She leads efforts to prevent and detect fraud, waste, and abuse, improving the integrity of NSF programs and operations and investigating allegations of misconduct in science. Dr. Boesz chairs the Misconduct in Research Working Group for the federal IG community, responsible for setting standards and training for investigations into research misconduct allegations.
Prior to this position, Dr. Boesz served as Head of Regulatory Accountability, at Aetna U.S. Healthcare (AUSHC), a subsidiary of Aetna, Inc. While there she had broad responsibilities for establishing and maintaining a compliance program for the managed care Medicare program. She also served as senior policy analyst on federal legislative and regulatory activities. Dr. Boesz joined AUSHC with its purchase of NYLCare Health Plans, a subsidiary of New York Life, in July 1998. From November 1995 she served as Vice President of Government Programs, principally responsible for developing and operating Medicare and Medicaid managed care programs in ten states. Her duties included oversight of product development, pricing, marketing, and compliance with government contracting requirements. She also developed legislative policy for the company.
Prior to her appointment at NYLCare, Dr. Boesz held several Federal Government compliance and oversight positions over an 18-year period with the Department of Health and Human Services. In her last position she served as Director of Operations and Oversight for the Medicare managed care program. In this capacity she was responsible for awarding and overseeing contract compliance. Earlier in her federal career she was instrumental in developing the regulatory framework for health maintenance organizations (HMO), in operating the HMO loan fund, and in building a compliance staff for managed care oversight.
Page 66 PREV PAGE TOP OF DOC
Dr. Boesz is a past president of the National Association of Managed Care Regulators (NAMCR), a member of the American Statistical Association, and the American Public Health Association. Over her career she has received numerous awards, including the Lifetime Service Award from NAMCR, and has been recognized as a Distinguished Alumna.
Dr. Boesz received her B.A. in mathematics from Douglass College (1966) and a M.S. in statistics from Rutgers University (1967). Her Doctorate in Public Health (health policy) was awarded by the University of Michigan School of Public Health (1997) where she was a Pew Fellow.
Discussion
Chairman SMITH. Dr. Boesz, thank you. I think maybe my first question is what Representative Johnson and I both mentioned, is the prioritization. Dr. Colwell, do you now, or do you plan to in the future, give the National Science Board your recommendation of with what you recommend to be the first and second and third priority depending on funding availability?
Dr. COLWELL. We present to the Board the projects that are—that meet the criteria of intellectual merit, that is, they are outstanding, in terms of scientific and engineering expertise——
Chairman SMITH. Yeah. Yeah. But do you say, here is my recommendation of the first thing that should be constructed, and, if we have enough money, here is the number two project that I recommend? I mean, it is a politically tough decision.
Page 67 PREV PAGE TOP OF DOC
Dr. COLWELL. Yeah. If the—in recommending to the Board the projects, we have to take into account balance. And so you can't really say that the astronomy project is number one over the biology project. You have to take into account——
Chairman SMITH. Who is going to make that decision if the National Science Board doesn't have your input of what should be number one and number two?
Dr. COLWELL. My discussion with the Board is to make sure that the projects that are presented are, in fact—meet the criteria and the Board approves them. Then, in my discussions with the OMB, within the budget, then I have to make that decision and it has to be of the criteria of readiness and balance, and so forth. So, yes, I do have to make that decision and the Director should.
Chairman SMITH. Do you put number one priority, number two priority? I mean——
Dr. COLWELL. To the Board?
Chairman SMITH.—I want to try to understand this a little better. To the Board. Do you give their recommendation if there is only enough money for one project—this is number one and this is number two—do you prioritize?
Dr. COLWELL. I prioritize by the fact that we have to complete the projects that are in process. They are number one in priority.
Page 68 PREV PAGE TOP OF DOC
Chairman SMITH. Yeah. Yeah. Right. And I——
Dr. COLWELL. And then while projects that have been submitted in previous budget requests continue to be a top priority—and they are listed as a top priority. And then we have——
Chairman SMITH. But do you—okay—help me—do you prioritize existing projects that are continuing? It came out of OMB in this last budget some prioritization. Some projects were left out that were on your list. And——
Dr. COLWELL. Yes. And they have become top of the list. Yes.
Chairman SMITH. Does the—okay. Let me ask you, Dr. Jones—on the prioritization decisions of the Board, in your work with NSF, does the Board make—have discussions and make some decisions of what should be—do prioritization, one, two, three, four?
Dr. JONES. Let us see, as I said earlier, I think that the number one decision, in terms of prioritization, is to go from a huge list of projects down to a very small set. And the expectation of the Board is that those will be commenced very soon, budget permitting, and so all of those should go forward. As I said in my testimony, a project may, for example, depend on supportive funding from international partners. In a particular month, in a particular year, it may be hard to determine when that will be in place. And so while all of a small set of projects are the highest priority, it is issues like that that will decide what next gets in a budget.
Page 69 PREV PAGE TOP OF DOC
Chairman SMITH. See, I am somewhat concerned that if you say these are all important projects and we have narrowed this list down so that these eight now are our most important projects, and that goes to OMB, do we really want OMB to prioritize that decision without a more exact prioritization by the Board and by NSF? And if OMB continues that limited list, do we want then politicians to make the kind of decisions of how they think they are going to prioritize, sometimes based on the influence of some, you know, individual Member of Congress that knows their university is working on a particular project and wants to continue that project for——
Dr. JONES. The Board expects each of those projects to go forward very shortly.
Dr. COLWELL. May I be specific, because I think this would help? For example, the priority is to complete the funding of existing projects.
Chairman SMITH. Do—in that, I want to——
Dr. COLWELL. The Large Hadron Collider—yes.
Chairman SMITH. In that, in seems to be——
Dr. COLWELL. Absolutely.
Chairman SMITH.—still it should be number one, number two, number three, number four.
Page 70 PREV PAGE TOP OF DOC
Dr. COLWELL. That is right. The Large Hadron Collider, the National Environmental Engineering System, NEES, the high-performance aircraft, we have to complete those with expended funds and it is cost-effective to do that. Then they have met all the other criteria. And let me bring forward the projects that are already now ready to go that have—for example, ALMA, the Atacama Large Array, that has—the R&D of it is done and it is ready to go forward. EarthScope and NEON have been proposed and Congress has stated that it has been deferred without prejudice. These are still very high priority. And then depending if there are additional funds, the projects that are ready to go forward are RSVP, Ice Cube. So there is a prioritization, very clearly.
Chairman SMITH. It wasn't that clear to me, Madam Director, in terms of the decision that was made by the Appropriations Committee to go ahead with some of the projects that weren't in your—that weren't in OMB's recommendation, but that you encouraged the Appropriations Subcommittee, in your mind, were very important. I don't know of the influence of the Board, but I think we need a second round of questions.
Dr. COLWELL. Okay.
Chairman SMITH. And, Bob—Mr. Etheridge, with that, your five minutes.
Mr. ETHERIDGE. Thank you, Mr. Chairman. Let me move to a little broader issue, if I might, for your comment. Because inasmuch as NSF employees who are assigned the responsibility for oversight on really large research facilities are typically program officers in the agency and usually they are scientists and engineers and I assume they are very capable people, otherwise they wouldn't be there. But they very well have a little expertise or training in the specific areas of construction management, accounting, and really other areas of expertise, as you are talking about oversight in areas of construction or operation of large research facilities.
Page 71 PREV PAGE TOP OF DOC
It seems to me, as we talk about this as being a major piece, the personnel issue, I believe, would be particularly challenging for NSF given that any program office, even those in superior ranks, are on a rotating basis, as well. So that being said, what effect has been—what efforts, I guess, have been made and—or will be made, if have not, by NSF to put in place NSF personnel with the specialized skills to manage effectively large facility construction projects?
Dr. COLWELL. Well, let me point out first that, again, I must iterate that the—reiterate that the NSF's record is extremely good. Now, that the future means larger projects, more complex. And that is exactly why we are putting our attention to this. And so in elaborating the process, it is clear we have to have the right people, and we have to develop the people at the Foundation who have the mix of training for the future. And that is one of the reasons that we have established—we are in the process of establishing an NSF academy—that is, an internal training program and opportunity for the expertise of the staff who are there to move within the organization, to retain the good people instead of having them move on to other agencies.
But we also have looked at the last facility projects and we believe that it is important to share the experience and the capability of having large projects, why we are having this workshop, to bring together our folks and the folks from other agencies to do this.
In addition, we have formed a new advisory committee. No other agency has, I don't—to the best of my knowledge. And that is to bring the financial officers and the construction officers from the other agencies and from private sectors to get an advisory body to the NSF financial office and the office of information and so forth. Now, this is a new advisory committee and it is a mechanism for us to have this kind of continued training.
Page 72 PREV PAGE TOP OF DOC
And then we have also determined that it would be useful to have the Deputy for the large projects. And this would be a person who reports to the Chief Financial Officer who reports directly to me. And this person will have the facility project construction experience or the management experience to be able to coordinate the construction of the many large projects. But at the risk, you know—Mr. Chairman, stop me when I need to.
But I would like to also point out that it is important to understand what—the National Science Foundation addresses all of science and we, therefore, have to have a mix of expertise because the projects that will be constructed, let us say, for NEON and biology, are not necessarily the same exact kind of project that is going to be constructed for another massive telescope. So we have to ensure that we have the mix and the sharing of expertise to go on to get these constructed.
And then, finally, the point that is really important is that you have got to balance of science that you are trying to achieve with the excellent management and the cost-effectiveness and responsibility to the taxpayer. So you don't want to have a thoroughly managed, exquisitely managed project, that by the time you get it built, maybe a decade out of date, because you haven't had the flexibility to incorporate the changes in technology. So it is a very, very difficult balancing act.
Mr. ETHERIDGE. Given that, talking about your advisory committee, before my time runs out, I understand—and from what I understand from your comments, it is just an advisory committee on the front side, and I assume it is ongoing as the project continues to move as it is——
Page 73 PREV PAGE TOP OF DOC
Dr. COLWELL. Indeed.
Mr. ETHERIDGE.—monitored in the process and the changes that take place along the way.
Dr. COLWELL. Indeed.
Mr. ETHERIDGE. Okay. Thank you, Mr. Chairman.
Chairman SMITH. The gentleman from New York, Mr. Grucci.
Mr. GRUCCI. Thank you, Mr. Chairman. I would like to direct these remarks to Dr. Colwell and to Dr. Jones, if I may. As you are probably both are aware in my district there is an MRE project that is awaiting funding, known as the RSVP, the Rare Symmetry Violating Processes. It is a large-scale particle physics project that has been applied, been peer-reviewed, and has been approved by the National Science Board. It is being conducted out at the Brookhaven National Laboratory which is part of the Department of Energy. It has done an enormous amount of work in the past.
And not only has the facility been recognized, but its management has been recognized. Dr. Marburger here has been selected by President from that facility, to be the National Science Advisor to the President. But what I am concerned about is that this project seems to be on hold. It seems to be stalled, for whatever reason.
Page 74 PREV PAGE TOP OF DOC
And, Dr. Colwell, if I may refer to your testimony, you do recognize the Rare Symmetry Violating Processes in your testimony on page 3, as a project that has been approved by the National Science Board. And you go on in your testimony on page 5 to reference the fact that it is your decision to make the recommendation for the projects for inclusion in future budget requests to the OMB. And while they seem to have followed the procedure and they seem to be playing by the rules, there seems to be others that—and I am not suggesting others aren't playing by the rules—that are seem to be jumping ahead of this project.
And this is a very important project, not only to the science community, to the country, but certainly to my district and to me personally. I would like very much to understand why this isn't moving forward with a little bit more acceleration than it has been in the past. And I would hope that the both of you would give this project very serious consideration and break through that log jam, get it into the OMB, get it into the funding process, so that the work that is being done can continue to be done. And I will allow whoever wants to take the first stab at the question, can respond.
Dr. COLWELL. Well, just let me point out that the RSVP and Ice Cube are projects that have been approved and consideration is yet to be determined. But the category of primary priority is completion of the projects that we have already started. As I mentioned, projects like the Large Hadron Collider and the Engineering System and the high-performance aircraft. And then we need to bring forward the projects that are ready now, ALMA, EarthScope, NEON. And then, the other projects that are definitely in the pipeline are RSVP and Ice Cube.
Mr. GRUCCI. Plus, I did hear your response to the Chairman in that order. And I appreciate that and I respect it. I guess what I am asking, and what is being asked of me by the lab and its management, is exactly when will you be making that determination? If—and we are told that it is a year, two years, three years down the road, we could accept that. What we are not prepared to accept is being hung out in limbo and not understanding when this project is going to get the recommendation that it needs to continue to move through the process. It is at a stalemate right now.
Page 75 PREV PAGE TOP OF DOC
Dr. COLWELL. Well, our budget is going to OMB next Monday. And, at that time, we—the negotiations will take place. And so I think——
Mr. GRUCCI. Is this going to be one of your recommendations in the budget?
Dr. COLWELL. I can't answer the question at this point. We are still in the process of doing our final determinations.
Mr. GRUCCI. When do you think you would be able to answer it?
Dr. COLWELL. When the budget is submitted.
Mr. GRUCCI. And when might that be?
Dr. COLWELL. Next Monday.
Mr. GRUCCI. Would you be able to respond back to my office with an answer?
Dr. COLWELL. This will depend on the budgetary considerations with OMB.
Mr. GRUCCI. I am not sure I understand the answer.
Page 76 PREV PAGE TOP OF DOC
Dr. COLWELL. Well, I will give you an answer within the next week, if that is satisfactory to you, sir.
Mr. GRUCCI. Thank you. I appreciate that.
Dr. JONES. Mr. Grucci——
Mr. GRUCCI. Yes.
Dr. JONES. This project, as you know, has been selected as a candidate MRE project by the Board, which puts it in a very special category. There are only a handful of projects in that category which have been approved by the Board and are—should the—the planning and all should be prepared so that it can move into the budget. The Board's anticipation is that the Director and the Foundation management will, as rapidly as possible, move this project and other projects into funding in such a way that they can assure that once the funding commences that the project is moved along as rapidly as possible so that the taxpayer gets the most value for the funds.
Mr. GRUCCI. I appreciate your comments. And we just want to understand when that process is going to unfold. We don't have a quarrel with the process. We just want to be able to understand when it is happening. It is my understanding that Ice Tea—I am sorry—the Ice Cube has been—Ice Tea, the New York style—the Ice Cube has been earmarked in this year's budget and has—and is moving forward. And we certainly want to—you know, we want the—want our project——
Page 77 PREV PAGE TOP OF DOC
Chairman SMITH. The Chair would call on Representative Rivers.
Mr. GRUCCI. Thank you, Mr. Chairman.
Ms. RIVERS. Thank you, Mr. Chairman. I apologize to the Panel and to the Chairman that I missed the first few minutes of the hearing. So one of the things that I—as I listen to this and I look over the materials, is, I am not sure what problem it is that we are trying to solve. I understand that the Galileo Project went over. Certainly, this is not the first federal project that have ever exceeded budget. So the question I would ask to Dr. Boesz is, before this Galileo issue, did you see the process that was being used in the NSF as a problem before the current controversy erupted? Has this been a problem over time? And what are the contours of that problem if it has been?
Dr. BOESZ. I believe, Ms. Rivers, you are referring to the Gemini Telescope Project.
Ms. RIVERS. Uh-huh.
Dr. BOESZ. The—I have been at——
Ms. RIVERS. Gemini. Yeah.
Dr. BOESZ. I have been at the National Science Foundation of the Inspector General for about a year-and-a-half and the audit work was in process when I arrived. My office has looked at this as an opportunity, not as a huge problem, but that it is a way for NSF to continue to improve the way it has handled projects in the past. We were looking at the accounting systems, the way project management occurred, and we found some areas for improvement. So it wasn't cost overruns per se that peaked our interest as much as how we could move the entire process in a better streak.
Page 78 PREV PAGE TOP OF DOC
Ms. RIVERS. I understand project management and some other aspects, but I am particularly questioning this issue of whether there should or shouldn't be numerical priorities assigned to projects. And what I am asking is, in the past, has this been a problem? I have heard it over and over and over up here. But I am asking, in the actual management of the National Science Foundation and the Nation's science activities, has this been a problem and what have the contours of that problem been?
Dr. COLWELL. Perhaps, I ought to answer it——
Ms. RIVERS. Thank you.
Dr. COLWELL.—because I would like to say that, indeed, when the Inspector General arrived at the National Science Foundation , she and I had a very good discussion on—in fact, when we talked before she came to the Foundation, that we recognized that the National Science Foundation is a very important agency that was in a process of having much more responsibility—larger budget, more projects, more complicated, interdisciplinary science, which is at the forefront.
Ms. RIVERS. Uh-huh.
Dr. COLWELL. It is different. It requires new ways at NSF doing this business and that she and I should work together to provide the absolute best management for the—what I consider—forgive me, Mr. Chairman—the best agency in the world. So we wanted to ensure that this would continue. So it isn't so much a problem as planning for the future, improving. And we have been working together extremely well. So that is really the answer to the question.
Page 79 PREV PAGE TOP OF DOC
Ms. RIVERS. And I didn't mean to sound critical at all. What I was trying to understand is since there was this tremendous emphasis on this side of the discussion——
Dr. COLWELL. Uh-huh.
Ms. RIVERS.—to there must be priorities and why are there not priorities, is I was trying to understand that in the real world operation of the agency, what sort of negative consequences have come from that, if any.
Dr. COLWELL. Well, it is still—yeah.
Ms. RIVERS. Because in Washington, we are often searching for a solution without the problem. And I want to make sure that that is not the case here.
Dr. COLWELL. No. Historically, NSF has not had a problem with its priorities in the MRE account. And, the fiscal year 2002 budget has been a sort of an inhibition for our ability to go forward. And that has, I think, kind of created, if you will, in the—and created a bulge and so we kind of have a bulge in trying to digest it——
Ms. RIVERS. Thank you. Thank you, Mr. Chairman.
Chairman SMITH. Mr. Gutknecht.
Page 80 PREV PAGE TOP OF DOC
Mr. GUTKNECHT. Thank you, Mr. Chairman. I will be brief and probably the easiest question you will have to deal with today. I do empathize with Mr. Grucci and I empathize with you. As a member of the Budget Committee—Dr. Jones, you used—you described some of the groups that you work with as large, complex, and costly. Now, as a member of the Budget Committee, when I hear those terms, I think of almost every federal bureaucracy. And I think the reason we are here today, and I think it is important—I mean, we all ultimately are held accountable. And objective as you, you know, try to be, as you mentioned, all of the projects that come before you, in terms of requests, are important projects. And, you know, serious scientists around the world, around the country, at least, have submitted those for your consideration and, as objective as you try to be, at the end of the day, it seems to me there is a certain amount of subjectivity to it.
And the only thing I am going to admonish you to do is keep us informed. Because, you know, we do serve constituencies and, you know, we have different pressures from our constituents about these. And we don't expect perfection. I don't expect perfection. I do expect as much accountability as possible. And the other thing we expect is you do as best you can to communicate with us—you know, what you are doing, what the process is. Are the taxpayers getting a good bang for the buck?
We will do our best, at least I will, on this Committee, to defend you because you do have a tough job. It is not easy. And, you know, I always remember the story of a former Senator from Wisconsin, who used to give away these Golden Fleece awards, and I suspect more than one of them were at the expense of the National Science Foundation. But that is the way it is. I mean, that is the world we live in and you are going to have some failures. You are going to have some cost overruns. And so the only thing I would say is do your best to be objective. Do you best to hold the projects accountable to try and get the most for what we can for the taxpayers. But most importantly, from our perspective—keep us informed. Just let us know what is going on, and we will do our best to work with you and defend you when you need it. Thank you.
Page 81 PREV PAGE TOP OF DOC
Dr. COLWELL. Thank you very, very much. We appreciate that deeply. And actually, let me just say that so far we have some astounding successes. We don't want any failures. Thank you.
Mr. GUTKNECHT. Thank you.
Chairman SMITH. With that kind of congeniality from the Budget Committee, I am sure—and just to add, I think, to the discussion, your challenges are probably greater than most departments and agencies. You are looking at trying to mesh the ability to accomplish scientific achievements with the construction of those particular facilities. And so the challenge is more than just the challenge of brick and mortar.
Dr. Boesz, when do you think a brilliant research scientist would not be capable of being a good Project Manager?
Dr. BOESZ. What a question. I think a brilliant scientist can certainly learn what is required to be a good Project Manager. And with the appropriate training the NSF plans to do, I think that can be accomplished. But I think that the elements that are required to handle construction do take a certain understanding to move a project along, and that balance is what needs to be achieved.
Chairman SMITH. Well, I think it is a—I think that that is one of our challenges. And it concerns me a little, to some degree, in your management of large facilities projects, the organizational staff chart, you have the project management under the awardee project director under the NSF Program Manager. And sometimes these programs—do you plan on sometimes putting a temporary program manager coming in from their university in charge of—with that responsibility?
Page 82 PREV PAGE TOP OF DOC
Dr. COLWELL. It depends on the project. In some cases, that might be appropriate. In other cases, it would not be appropriate.
Chairman SMITH. Dr. Jones?
Dr. JONES. Yes, sir.
Chairman SMITH. When is it appropriate?
Dr. JONES. Let us see, I think Tina went to the fundamentalists. Whoever is in charge of that magnitude of a project ought to have the experience and the expertise to conduct their responsibilities.
Chairman SMITH. What would be the complications of this? What are—normally, what is this rotating person? Is that a two-year——
Dr. COLWELL. No. It is a two-year, three-year, up to a maximum of 6 years for rotators.
Chairman SMITH. So this person has some responsibility, if they are going to go back to a viable operation at their university to, to some extent, continue the maintenance of their lab, their facility, their efforts, back at their home university.
Dr. COLWELL. But we——
Page 83 PREV PAGE TOP OF DOC
Chairman SMITH. And so it gives me a little bit of concern——
Dr. COLWELL. No. We have the—we have the option of having a permanent staff, career staff, employee assigned to the project. And we do caution oversights. And, in fact, the large projects we site visit once a month. So it is not just as if it is just sort of left to drift. And it is not—it isn't just a transient who comes in and leaves.
Chairman SMITH. It would be my experience that sometimes that scientist, that researcher, is not capable of the kind of appropriate oversight that might be required in physical construction. And so somehow the balance in the appointment of a Deputy—whatever you call this person—a Deputy for Large Facility Project Management—seems to me to be extremely important and——
Dr. COLWELL. Well, that is the——
Chairman SMITH. I asked for a timeline.
Dr. COLWELL. Sure.
Chairman SMITH. Could you give me the timeline on how—on the selection process—when you expect to put this person in place?
Dr. COLWELL. Well, we are anticipating drafting the job description to make sure we get the right criteria and we hope to have someone in place by January.
Page 84 PREV PAGE TOP OF DOC
Chairman SMITH. By January.
Dr. COLWELL. Yes.
Chairman SMITH. And this would be——
Dr. COLWELL. But in the meantime, we are having workshops on best practices——
Chairman SMITH. This would be an executive grade person that you expect to review, enter, apply, interview, select, confirm, in the next, whatever, 2b months.
Dr. COLWELL. Well, we are hoping that within certainly January or February we could do that.
Chairman SMITH. Is that possible? When will your——
Dr. COLWELL. The constraints are the federal hiring system too.
Chairman SMITH. When will—have you written your requirements and——
Dr. COLWELL. Well, we have been really working very hard to determine just what the criteria should be.
Page 85 PREV PAGE TOP OF DOC
Chairman SMITH. Dr. Boesz, what kind of a person should this be with your experience in different agencies?
Dr. BOESZ. I believe the person should have requisite skills in understanding large facilities and what it takes to construct large facilities. Probably, if I were doing it, I would put an emphasis on the combination of engineering skills and financial management skills.
Chairman SMITH. Could I have the permission, Dr. Colwell, to ask your Finance Director Cooley a question?
Dr. COLWELL. Certainly. But let me also say, Mr. Chairman, that we will keep you informed as we move along in this process.
Chairman SMITH. I—in the letter that I sent to you and the colloquy that Representative Walsh and I had on the Floor, we were both very interested in the timeline and the process and the kind of individual. So I am interested in that kind of a timeline development. If we are going to do this within a reasonable time period, it seems to me that it is going to be difficult to accomplish this, at least my experience in previous Administrations, in less than four months, but maybe you can do it.
And, Mr. Cooley, I was going to ask you your thoughts as Financial Officer, which would—which, I think, has a tremendous responsibility under the Director for accomplishing some of these tasks. When—how do you decide on appropriate management skills and how do you make that decision when an NSF Program Manager is going to be the temporary and when they are going to be a full-time trained management project person?
Page 86 PREV PAGE TOP OF DOC
Mr. COOLEY. Start with your first question—last question first.
Chairman SMITH. Would you introduce yourself for the record?
Mr. COOLEY. Thank you. My name is Tom Cooley. I am the Chief Financial Officer and Director of the Office of Budget, Finance, and Award Administration at the National Science Foundation.
To start with your last question first, make no mistake about it, most of the staff that we have at the Foundation that oversee these projects are full-time, permanent employees. So that is the predominant requirement. Occasionally, you will find that we will need to bring in an expert on—for a short period of time, say, two or three years.
For example, in the Directorate for Geosciences, the Assistant Director for Geosciences, Dr. Margaret Leinen, has just hired someone with 15 years' experience in outfitting aircraft planes. That plane can be purchased and outfitted in a matter of two or three years. So you really only need someone there for a short period of time.
On the other hand, with South Pole Station Modernization, or with Gemini, or even the Atacama Large Millimeter Array, where you have a horizon of 8 or 9 years construction, you need someone that you are going to bring in full time, who will not only oversee that project full time, but that will have responsibility for seeing it into its operational phase and long-term life, because most of these things are alive for 30 to 40 years.
Page 87 PREV PAGE TOP OF DOC
To get to your first question last, you are correct, the federal policies for hiring people are probably going to take us four months. So I have got two parts to your question—two parts to the answer to your question. First of all, we are developing a position description right now, and we would be happy to share that with you once we have posted it. Under normal senior executive surface rules, we normally post that for four to six weeks. My preference, personally, is to do six weeks.
If you want the best people, you are going to do a nationwide search. You want it widely advertised and you want people to be able to come in. Looking for people, as Tina said, with excellent project management background, who know the system, will understand what it is, and what concrete is, what steel is, what the various grades are, who understands what it is to do a work breakdown structure and to cost that out appropriately. So you need someone with that kind of background and skills.
By the time you go through receiving the applications, doing the technical review panel, I prefer to do an interview panel after that, and select someone, you are quite right, it is going to be about four months before we select someone and bring them in.
So what do we do in the meantime? We have got four months between now and then. You take advantage of the experience of on-staff that we currently have on hand. You bring in consultants who have experience in these things and you ask them to help you formulate policies and procedures that we need to have in place so that when you hire this person and he hits the ground, he or she, hits the ground right.
Page 88 PREV PAGE TOP OF DOC
Dr. COLWELL. Four months is roughly January, February.
Chairman SMITH. Hopefully. That is going to take some—it seems to me that hopefully, the sooner the better, and the quicker you move ahead with it—but not to disadvantage the best possible selection because you try to limit it to four months. And it seems to me this person, because of—you need the experience in construction and you need the experience in management, but you also have to have some person that has the kind of appreciation for scientific skills. So it would seem to me——
Dr. COLWELL. That advice we are very happy to take. You are right.
Chairman SMITH. I am sorry?
Dr. COLWELL. I said that—your advice is—we are happy to take—you are right. We agree.
Chairman SMITH. Well, on this—in the political sector, we give lots of advice. Sort of like salt and pepper—you have to use it to suit your own taste, except for when we pass up Appropriation bills. Dr. Boesz, who do you report to?
Dr. BOESZ. I report to the National Science Board. And——
Chairman SMITH. The National Science Board——
Page 89 PREV PAGE TOP OF DOC
Dr. BOESZ. Yes.
Chairman SMITH.—hired you.
Dr. BOESZ. Yes. They did.
Chairman SMITH. And is the separation between you and NSF enough that it gives you the kind of flexibility to help assure that you are going to use your independent judgment without solving the problems before Congress sees your reports?
Dr. BOESZ. I believe that it does. I think that the relationship that we have—my office has with both the Board and with Dr. Colwell and her staff, is one that, while it is collegial, it is independent, and that independence is respected. And we are called upon to offer advice. Our reports are taken seriously and I am comfortable with that relationship.
Chairman SMITH. Dr. Jones, is the NSB concerned with cost overruns?
Dr. JONES. Oh. It certainly would be concerned with cost overruns wherever they might occur. As I said it is mandatory for the Director to come back to the Board if a cost overrun is anticipated that is over a certain amount. I think you are rightly focusing in on the independence of the IG. The Board, particularly through the Audit and Oversight Committee, works very closely with the IG and encourages the kind of independence that permits an independent evaluation of what is going on when the IG audits them.
Page 90 PREV PAGE TOP OF DOC
Dr. COLWELL. Let me also add, Mr. Chairman, that we have in place a mechanism whereby if a project goes into contingency, which is $10 million, or 20 percent of the original cost, whichever is less, the Director must be informed. And, in fact, when that is that is going to triggered, the Deputy Director is informed as well, for smaller amounts. So we have in place now an automatic system or understanding when a project is going to go over.
Chairman SMITH. I——
Dr. COLWELL. And out of 61 projects, we have gone into contingency on only six projects.
Chairman SMITH. I mean, it—traditionally, I suspect that you simply come back to the Appropriations Committee and say, here is a problem. But if the responsibility comes back, such as on the Green Bank Telescope, and I suspect the Court Decision of 4.9 million would end up coming back to NSF. And here again, to the extent that that liability comes back to NSF, it means that somehow you are going to have to reduce or modify priorities and it is going to have to come from someplace else in the budget. Is that the case, Dr. Colwell?
Dr. COLWELL. I think in the Judgment that was made, it indicates that, in fact, the claim was much, much larger and the settlement was very small.
Chairman SMITH. Well, 4.9 million. But——
Dr. COLWELL. Yeah. But I think that we are working that out. I am not quite sure whether that—how that is going to completely play out. It hasn't—that hasn't been decided.
Page 91 PREV PAGE TOP OF DOC
Chairman SMITH. Well, just to the extent of the possibility that cost overruns might come back to haunt our efforts and have to reduce some of our other goals in research and development, it seems to me to play back to the importance of the management criteria and how exactly NSF, and with the supervision of the Board, plans to oversee the awardees, whether it is a university or come to some other element, in terms of going out to that university and following up and managing, such as I see in your proposed organizational staff chart. Traditionally, my understanding is we have left a lot of that up to the individual awardee to accept that responsibility with not as much oversight as maybe we should have.
Dr. COLWELL. We actually have visits to the site once a month for the large projects. In addition, we have an automatic—now, and within the last year, we have an automatic triggering of when the costs are going to go over. So that—with notification to the Deputy and to me. So we have, I think with the new financial system that is in place—we have the ability to be tracking that much more closely. I would ask my Chief Financial Officer if he wants to—he can——
Mr. COOLEY. The only thing that I would add is that for most of our projects, the program manager in charge, who reports up the line through the Assistant Director to the Director, normally visits the site about once a month, to kick the tires, so to speak. That Project Manager at NSF has his or her own external advisory board that reports to him and helps him or her oversee that project. In addition, at the site, the Principal Investigator at the institution almost always has an external oversight advisory body that independently comes in about every 6 months, certainly at least once a year, to help the PI on site evaluate their progress towards meeting their milestones and their costs.
Page 92 PREV PAGE TOP OF DOC
Dr. COLWELL. If there are any disputes that arise, we also would have the opportunity to bring in a Blue Ribbon panel to have them look at the costs and——
Chairman SMITH. And what kind of a—what kind of a panel? How do you make balance between that needed scientific knowledge and the practical expertise in large facilities construction?
Dr. COLWELL. Well, let me tell you—the example probably that is best to provide is when the technology shift occurs. For example, computers, every two or three years, are outdated. And if, in the case of astronomy with adaptive optics that—and the technique that was developed, one has to make a modification in the project. And this is most likely where the overruns might occur. That is, increased costs that were unplanned. And at that point, then there has to be an evaluation of the technology that is going to be incorporated, whether it is going to be an appropriate investment to make, it will jump—catapult or make a solitative leap in the capacity of the instrument in its final completion. And does the cost that it would take to do that—is the balance appropriate?
And, in a case like that, you may have to bring in some scientific experts, as well as management experts, to make the decision to go or no-go, or to stick with the original plan. That is one example.
Chairman SMITH. It would be my—I would just suggest to you that I anticipate tighter budgets as we experience less revenues coming into the Federal Government than we have seen in the past. And so a greater responsibility maybe. And just for wrap-up, maybe with each one of you, starting with you, Dr. Boesz. Do you—under legislation, you are partially responsible to report to Congress and to give us your guidance and help in this area. Final comments on these areas that Congress might be looking at—your advice to us, in terms of management, in terms of the kind of people that are going to make these decisions. Any conflicts between science and experience in facility construction?
Page 93 PREV PAGE TOP OF DOC
Dr. BOESZ. My advice will be that we will share with you and hopefully we would participate with my office in analyzing the implementation of this Plan as it goes forward. I think we are going to all have to work very closely together to assure we have the kind of accountability and performance measures that we are all comfortable with. And so I think we need together to focus on performance measures of how this Plan is going to work.
Chairman SMITH. And, Dr. Jones, does the Science Board meet how often, about once a month?
Dr. JONES. It meets five times a year.
Chairman SMITH. Five times a year. And this would be a, what, a 2- or 3-day meeting?
Dr. JONES. Two to 3-day meeting. Yes.
Chairman SMITH. And is that—does that make you comfortable in terms of having—being abreast of the—of what is happening in the situation to—for the Board to give the kind of advice that is needed, or do you depend a great deal on staff's—full-time staff's oversight?
Dr. JONES. Let us see, I am comfortable with the frequency. In addition to that, there are numerous telecons. The Board is organized into committees who have specific chartered responsibilities so that in parallel there are numerous committees looking at different aspects and different issues that come in front of the Board.
Page 94 PREV PAGE TOP OF DOC
We also have a Board office and that is to help—is immensely valuable. And, of course, we get a lot of help from—we are never asked—from the Foundation management. And I am comfortable with that. Yes, sir.
Chairman SMITH. Any other comments for a wrap-up on the quality of management, the type of management person that should be involved, or the Board's input in making sure that it is not just politics that ends up in prioritizing?
Dr. JONES. Let us see, I think we have all arrived at a heightened concern about managing these kinds of projects. As you say, the budget is tighter. We are—there is more demand for these projects. They are more complex. I look forward to a sound implementation of this Plan. And I think it is a step forward for the Foundation with the Board's support to create a more formal plan and to put in place this Deputy for Large Facility Projects who needs to have the kind of expertise that has been described, in terms of actual construction management. It is not just a scientist who is a user that is appropriate for that kind of a job. Thank you.
Chairman SMITH. Dr. Colwell, NSF is the best research organization in the world, and, your wrap-up comments.
Dr. COLWELL. Well, my comment is that—NSF is scientific and engineering excellence. And that has to remain in balance and I appreciate your comments. We intend to be as open as we can with you and with our scientific communities towards a more transparent agency and to have in place the management structure that will allow us to move into—I would like to be have been more optimistic—to eventually a doubled NSF budget coordination. Thank you.
Page 95 PREV PAGE TOP OF DOC
Chairman SMITH. Thank you, Dr. Colwell. And thank all the witnesses. The record will remain open for five legislative days for any additional comments. And we ask the panelists, if we haven't asked all of the right questions and staff decides we would like to ask you a couple more questions, we hope you would respond to those questions. And with that, the Committee is adjourned.
[Whereupon, at 10:58 a.m., the Subcommittee was adjourned.]
Appendix 1:
Answers to Post-Hearing Questions
ANSWERS TO POST-HEARING QUESTIONS
Post-Hearing Questions Submitted to Dr. Rita R. Colwell, Director, National Science Foundation from Subcommittee Chairman Nick Smith
QUESTION: In your testimony you stated that the NSF Program Manager exercises primary responsibility for all aspects of project management, yet it appears the Deputy for Large Facilities Projects will also have overall responsibility for large project construction and management. Who has ultimate authority over large facility projects? Please describe the chain of command for Large Facilities Projects.
ANSWER: With few exceptions, NSF does not directly manage the facilities it supports. That responsibility is vested with the awardee institution. However, NSF does manage the oversight of facility awards during each of the stages of the facility's lifecycle. To do this, NSF employs a team-oriented approach, with its scientific and engineering staff working hand-in-hand with its business operations staff. The NSF Program Manager exercises primary responsibility within NSF for all aspects of the project, subject to normal oversight by his or her division director, Assistant Director or Office Head, the MREFC Panel (if applicable), the Director's Office and the National Science Board (NSB). As described in the Answer to Question 2, the Program Manager is supported by the Deputy, Large Facility Projects (LFP) and by Project Advisory Teams.
Page 96 PREV PAGE TOP OF DOC
QUESTION: Please describe the specific management responsibilities of both the Deputy for Large Facilities Projects and the Program Manager for a particular project. Please include in your description how the Program Manager and the Deputy will: (1) share authority and accountability with respect to large project management; (2) communicate effectively given that, according to the official agency management plan, there are no direct links between the Program Manager and Deputy; and (3) resolve conflicts or disagreements.
ANSWER: The Program Manager manages the NSF funding for a given project and, hence, has primary responsibility within NSF for all aspects of the project. The Program Manager is supported in this effort by the Deputy, Large Facility Projects, who provides expert assistance on oversight of non-technical aspects of project management, including planning, budgeting, construction and/or acquisition, and operations. He or she, in conjunction with business operations staff, will ensure that awards and awardees are in compliance with all NSF and Federal policies, guidelines and procedures. This individual will participate in the Project Advisory Teams (PAT) convened by the Program Manager. PATs are constituted to provide advice and assistance on planning, review and management of projects to assure the establishment of realistic cost, schedule and performance goals and to develop terms and conditions of awards for constructing, acquiring and/or operating a facility. Each PAT is comprised of professionals with critical expertise in the relevant science and engineering fields, as well as management, business operations, financial and legal requirements.
Supported by several NSF staff with a mix of skills and experience, the Deputy, LFP will conduct comprehensive post-award oversight of business operations, financial and internal control systems, and cost and schedule performance. The Deputy works very closely with the Program Manager to ensure that awardees are performing to the terms and conditions of their awards and that they are attaining cost and schedule goals. The Program Manager, with the advice of the Deputy, LFP, is responsible for recommending corrective action, if necessary. They are also responsible for informing their managers of significant problems. The Program Manager and the Deputy, LFP will attempt to resolve any disagreements or conflicts that arise. In the unlikely event that conflicts cannot be resolved at this level, the issues will be brought to successively higher levels (e.g., CFO and Assistant Director or Office Head) until a resolution is reached.
Page 97 PREV PAGE TOP OF DOC
QUESTION: Please clarify the guidelines for Intergovernmental Personnel Act (IPA) and temporary employment assignments as well as the process by which permanent employees are recruited and selected from the temporary staff. What impact will temporary employees and IPAs have on the management of long-term MRE projects? How will non-permanent employees be held responsible for problems in MRE project oversight and how will temporary employees be actively engaged in the evolution of a particular project from conception to completion?
ANSWER: The Intergovernmental Personnel Act (IPA) allows the temporary assignment of personnel between NSF and other governmental, academic, tribal, and eligible non-profit organizations. IPA assignments can strengthen management, assist in the transfer and implementation of new technology, involve officials of other organizations in developing and implementing Federal policies and programs, and enhance the professional abilities of the participants. IPA participants assigned to NSF are subject to all laws and regulations applicable to Federal employees, to any state or local government regulations, and to any regulations of the home institution specified in their agreements. Initial IPA appointments can be made for up to two years and may be extended to three years. Under unusual circumstances, assignments may be extended for up to four years with the approval of the Deputy Director of NSF.
The NSF Visiting Scientist, Engineer and Educator (VSEE) Program enables the Foundation to augment its permanent staff with individuals on leave of absence from research and/or educational institutions. The VSEE Program is designed to benefit both the Foundation and scientific, engineering, and educational organizations. It provides the Foundation with professionals who are active in scientific and engineering fields, and provides these individuals and their institutions with active exposure to the philosophy and mechanisms of Federal support for research. Visiting scientists are given temporary, excepted appointments under NSF's Statutory Excepted Appointment and Compensation Authority. Such appointments are made for one year, but can be extended for an additional year. IPAs and VSEEs are evaluated in the same manner as permanent staff. If there are performance problems, appropriate steps are taken.
Page 98 PREV PAGE TOP OF DOC
It is unlikely that an IPA, VSEE or other temporary personnel would be assigned as Program Manager for a long-term large facility project. Were this to occur, NSF would ensure that there would be no negative impact on the management and oversight of the project. In such instances, a permanent NSF employee would be assigned to assist the IPA/VSEE Program Manager, and provide continuity when that official departs NSF.
QUESTION: How does the NSF determine travel budgets for program managers so that they can provide adequate oversight, including site visits, for all awards including centers, MRE, and large infrastructure awards? How does this compare with the travel budget for a program manager overseeing a greater number of smaller awards?
ANSWER: Program Managers individually estimate the cost of all oversight activities associated with the awards for which they are responsible. Within each Directorate/Office, these individual estimates are combined and, where necessary, adjustments are made. NSF Program Managers who are required to oversee large facility projects and centers are usually given additional travel funds, over and above what they would receive if they had only small awards in their portfolios.
QUESTION: In your testimony you stated that the NSF conducts monthly site visits for each large facility project. Who conducts these visits? How long does each visit last, and which appropriated budget category provides funding for this oversight function? Do the monthly visits occur during both the construction and implementation phase of the project?
ANSWER: The frequency, duration and format of site visits vary with the size and complexity of the project. While some of the more complex projects are visited on a monthly basis, others are visited on a quarterly basis. Site visits normally last between three and five days. Site visits occur throughout the facility's lifecycle but are more frequent during construction and implementation.
Page 99 PREV PAGE TOP OF DOC
Barring any indication that there are issues or problems, operational facilities would typically be visited once during the award period. The Program Manager coordinates site visits to facilities. In addition to the Program Manager, other internal NSF and external personnel will participate depending upon the type of review being conducted.
All travel costs for permanent NSF staff must be paid through the Salaries and Expenses appropriation account, including travel associated with projects funded through the MREFC appropriation account.
QUESTION: Please provide us with an updated hiring timeline for the Deputy, Large Facilities Management Project position based on where you are today in the hiring process.
ANSWER: NSF developed a position description for the Deputy, Large Facility Projects, and advertised the Senior Executive Service vacancy in early January. Advertisements were placed in the Washington Post, Wall Street Journal and the Chronicle of Higher Education. The advertisement was also listed on the following web sites: Project Management Institute, National Council for University Research Administration, Society for Research Administrators, Federal Demonstration Project and internal NSF pages. The vacancy closed on February 8, 2002. According to NSF's Division of Human Resource Management, over 150 applications have been received in response to the advertisement. The CFO has begun to solicit peer reviewers to serve on the panel to evaluate the applications received and to interview the most qualified candidates. NSF expects to have an individual in place by the end of April.
QUESTION: You stated in your testimony that six out of 61 projects have gone into ''contingency'' meaning that spending for those projects exceeded the approved budget by 20 percent or $10 million. Please provide us with a summary of the six contingency projects mentioned in your testimony including the amount of the overrun, the reason for the overrun and the resolution to the problem.
Page 100 PREV PAGE TOP OF DOC
ANSWER: We conducted an analysis of all awards requiring NSB approval for the period 1995 through May 2001. During this period, there were 52 awards totaling $3.38 billion that required NSB approval. The six awards—four of which were facilities, but only one of which received funding through the MREFC appropriation account—totaled $226.9 million and exceeded the NSB-approved amount by $10.8 million. This overrun represents 4.8 percent of the cost of the six awards, and 0.3 percent of the total cost of all NSB-approved awards.
75811c.eps
QUESTION: You indicated that NSF has adopted a system that automatically notifies you when a project is going to go over budget. What triggers this automatic notification and what strategies do you employ either to eliminate or minimize the cost over-run?
ANSWER: The system generates an automatic reminder of the approved amount along with the requirement for notifying the Director's Office and the NSB whenever the Program Manager seeks to commit funds for the project. In the event that the award will exceed the approved amount, the Program Manager begins discussions with his or her division director, Assistant Director/Office Head, and MREFC Panel (if applicable) and the Director's Office prior to the expenditure of additional funding beyond the NSB-approved amount.
There are several strategies that are employed when it becomes clear that a project could go into an overrun situation. The Program Manager will, for example, work with the awardee and the science and engineering community to de-scope the project, or to conserve funds by extending the scheduled completion date. More importantly, however, are the risk management strategies used at the beginning of the process to identify potential cost, schedule and performance risks, and to implement methods of addressing the risk. NSF is currently adapting risk management guidelines and procedures, with an extensive section on potential risks and how to identify and manage risks.
Page 101 PREV PAGE TOP OF DOC
QUESTION: At the NSF-sponsored ''Large Facility Projects Management & Oversight Best Practices Workshop,'' held on September 21, 2001, an NSF program manager of a large MRE project stated that once construction begins, changes to the original construction plan are supported by operating funds and not construction funds, even if these changes occur during the original construction phase. This statement seems to contradict NSB Bulletin 01–15, which states that ''once an award for an MRE project is established with MRE funds, no funds from other appropriations may be obligated under the award.'' What is NSF policy regarding this issue and what steps are being taken to ensure that program managers understand and comply with this policy?
ANSWER: The statement made by the NSF program manager requires some additional clarification. With few exceptions, NSF supports planning, design and development from the R&RA Account, as well as operations and maintenance, once a facility is completed. Acquisition, construction and commissioning are funded from the MREFC Account. With respect to significant additional costs imposed by changes to the scope of an MREFC project or by other unforeseen circumstances, they will normally be funded through the MREFC Account. Such costs are very closely controlled and tracked by NSF. All changes are reviewed within the affected Directorate/Office, and the Office of the Director is to be notified prior to the expenditure of additional funding beyond the NSB approved amount. Significant changes (costing an additional 20 percent of the project budget or $10 million, whichever is less) require the approval of the National Science Board. NSF will notify the OMB and the Congress of any significant changes to the cost of an MREFC project.
In reference to NSF Bulletin 01–15, the policy was instituted to prevent the mixing of funds from different appropriations under a single NSF award. The Bulletin does not prohibit a separate award, funded through a different appropriation account, from being issued to the same awardee.
Page 102 PREV PAGE TOP OF DOC
NSF is in the process of clarifying its policies and guidance with respect to the use of the MREFC and R&RA Accounts in funding the various project phases, as well as tracking the total life cycle costs of these projects.
ANSWERS TO POST-HEARING QUESTIONS
Post-Hearing Questions Submitted to Dr. Rita R. Colwell, Director, National Science Foundation from Subcommittee Ranking Minority Member Eddie Bernice Johnson
QUESTION: The IG, in her testimony, indicated that the NSF management plan for large facility projects should clarify who will have final oversight and accountability for each large facility project. The plan seems to create two lines of authority—one for scientific project management and one for administrative oversight.
ANSWER: Inherent in NSF's management structure is the responsibility for scientific and engineering management and the responsibility for business operations management. NSF manages the oversight of facility awards during each of the stages of the facility's lifecycle through a team-oriented approach, with its scientific and engineering staff working hand-in-hand with its business operations staff. The Program Manager, representing the programmatic side, exercises primary responsibility within NSF for all aspects of the project, subject to normal oversight by his or her division director, Assistant Director or Office Head, the MREFC Panel (if applicable), the Director's Office and the National Science Board (NSB). As described in the Answer to Question 2, the Program Manager is supported by the Deputy, Large Facility Projects (LFP) and by Project Advisory Teams.
Page 103 PREV PAGE TOP OF DOC
QUESTION: What is the relationship between the NSF Program Manager for a project and the new Deputy for Large Facility Projects?
ANSWER: The Program Manager manages the NSF funding for a given project and, hence, has primary responsibility within NSF for all aspects of the project. The Program Manager is supported in this effort by the Deputy, LFP, who provides expert assistance to the Program Manager on non-technical aspects of project management, including planning, budgeting, construction and/or acquisition, and operations. He or she, in conjunction with business operations staff, will ensure that awards and awardees are in compliance with all NSF and Federal policies, guidelines and procedures. This individual will participate in the Project Advisory Teams (PAT) convened by the Program Manager. PATS are constituted to provide advice and assistance on planning, review and management of projects to assure the establishment of realistic cost, schedule and performance goals and to develop terms and conditions of awards for constructing, acquiring and/or operating a facility. Each PAT is comprised of professionals with critical expertise in the relevant science and engineering fields, as well as management, business operations, financial and legal requirements.
Supported by several NSF staff with a mix of skills and experience, the Deputy, LFP will conduct comprehensive post-award oversight of business operations, financial and internal control systems, and cost and schedule performance. The Deputy works very closely with the Program Manager to ensure that awardees are performing to the terms and conditions of their awards and that they are attaining cost and schedule goals. The Program Manager, with the advice of the Deputy, LFP, is responsible for recommending corrective action, if necessary. They are also each responsible for informing their managers of significant problems. It is expected that the Program Manager and the Deputy, LFP will be able to resolve any disagreements or conflicts that arise. In the unlikely event that conflicts cannot be resolved at this level, the issues will be brought to successively higher levels (e.g., CFO and Assistant Director or Office Head; Director's Office) until a resolution is reached.
Page 104 PREV PAGE TOP OF DOC
QUESTION: Where does the ultimate authority for a project reside, and who is responsible for ensuring that senior NSF management and the National Science Board are informed about schedule, cost, or technical problems with a project?
ANSWER: Ultimate authority for any NSF project rests with the NSF Director and the National Science Board. This responsibility is delegated to the NSF Program Manager, who exercises control over the funding available to a project and is responsible for project management and oversight. The Program Manager is assisted by, among others, the LFP Deputy and the PAT. The Program Manager is responsible for informing senior management and the NSB of significant problems with a project. In addition, the LFP Deputy, who reports to the NSF Chief Financial Officer, and the OIG bear some responsibility for independently informing senior management and the NSB.
QUESTION: The IG points out that in some cases conflicts may arise between the scientific capabilities of a new facility and financial and engineering concerns. How will such conflicts be resolved under the proposed management plan?
ANSWER: The Program Manager has the responsibility to weigh the costs and benefits and risks associated with scientific capabilities and financial and engineering concerns. Based on advice and information gathered from the awardee, the science and engineering community and other experts, he or she makes the decision that is in the best interest of the community to be served given the resources available. If the decision involves a significant change to the cost, schedule and/or scope of the project, it is reviewed by NSF senior management, and when appropriate, by the National Science Board.
Page 105 PREV PAGE TOP OF DOC
QUESTION: Why is the comprehensive management plan that will be required for every facility construction project not evaluated as part of the proposal evaluation process, rather than as a post-award task?
ANSWER: For all large facility projects, NSF requires a comprehensive project management plan to be submitted and evaluated before any constructions awards are made.
QUESTION: Describe how the post-award reviews of construction projects will be conducted in terms of content and frequency, and how the results of the reviews will be used to monitor construction progress and, potentially, to revise the project management plan.
ANSWER: The awardee is expected to provide appropriate internal oversight of its own activities. In addition, however, reporting and reviews are an important part of the oversight process that allows the Program Manager to monitor performance and compliance with project goals. Due to the complex nature of facilities, the level of oversight will be more extensive than for a standard NSF grant. The frequency, duration and format of site visits vary with the size and complexity of the project. While some of the more complex projects are visited on a monthly basis, others are visited on a quarterly basis. Site visits normally last between three and five days. Site visits occur throughout the facility's lifecycle but are more frequent during construction and implementation. Barring any indication that there are issues or problems, operational facilities would typically be visited once during the award period.
In some cases, such as large construction activities or projects in which NSF reviews and approves technical changes, it may be necessary to have personnel—either NSF staff or contractor staff—on-site to participate in the project. Oversight, reviews, and reporting requirements change as the project moves from construction/acquisition to operations. In general, operations are reviewed on a once-per-award period basis. The Program Manager defines the reporting and review requirements of the awardee in the award instrument, and these deliverables are noted as milestones on the project master schedule. Reports are generally provided and reviews conducted on a monthly and/or quarterly basis, with a comprehensive annual report provided by a pre-determined date. Some projects, particularly those with construction activities or those subject to frequent changes in design, may need more frequent reporting and review intervals. For construction projects, reports and reviews typically include: project tasks completed, milestones achieved, percent of project completed, cost and schedule variances, remaining contingency, emerging technical issues, management issues, etc. For operating facilities, reporting and review shifts to operational performance and efficiency, user demand versus supply, research developments, instrument development activities, users involved (faculty, postdocs, students), degrees resulting, research progress and achievements, analysis of down time, maintenance activities, facility improvements, etc.
Page 106 PREV PAGE TOP OF DOC
The Program Manager can require the awardee to organize regular reviews during which the Program Manager, along with the PAT or other review personnel (e.g., external experts) chosen by the Program Manager, assesses project progress and performance. Such a review may, for example, address:
Project cost
Project schedule
Project performance, including technical design
Systems integration
Complex procurement issues
Safety and Environmental issues
Plans for Operations & Maintenance
The LFP Deputy and his/her staff will conduct reviews of all business operations aspects of facilities. At least annually for construction projects, the awardee's systems and transactions will be tested and cost and schedule performance will be reviewed independently from any reviews the Program Manager conducts.
Page 107 PREV PAGE TOP OF DOC
NSF has developed GPRA goals to measure the project performance for the construction/acquisition and operations stages of a facility. Information related to these goals is collected from awardee facilities managers via the ''Performance Data Reports'' module of the FastLane Project Reports system. Reports cover the period October 1 through September 30 of each fiscal year. As each report is submitted, it is routed to the cognizant Program Manager for acceptance.
The results of all these reports and reviews will be collected and analyzed periodically so that trends—either best practices or recurring problems—can be identified and solutions employed across all of NSF's facilities.
QUESTION: What happens to the priority of a new research facility project that is included in the NSF budget request but is not provided an appropriation by Congress? Does it simply go back into the pool with other facility projects judged to be ready for construction or does it have priority for inclusion in the next year's budget? If it does not have priority for the next year, please explain why.
ANSWER: Projects that are in the NSF budget request, but not funded by Congress, do have priority over projects that have not been previously requested. But this priority is not absolute, and must be modified by the current budget and political environment, new scientific opportunities, and the expressed wishes of the Congress.
QUESTION: Through what process are priorities established and funding balance achieved among the competing demands of new facility construction projects, operations and maintenance of existing facilities, and support for research grants that would make use of major research facilities? Is consideration being given to budgeting funds specifically for support of research that makes use of a new research facility?
Page 108 PREV PAGE TOP OF DOC
ANSWER: NSF's budget and planning process includes a comprehensive review and evaluation of all of these competing demands, the end result of which is a set of budget priorities approved by the NSF Director and the National Science Board. At present, NSF does take into consideration the need to provide for support of research that makes use of a new research facility.
ANSWERS TO POST-HEARING QUESTIONS
Post-Hearing Questions Submitted to Dr. Anita K. Jones, Vice Chair, National Science Board from Subcommittee Chairman Nick Smith
QUESTION 1: In your testimony you stated that the National Science Board approves funding for large facilities projects for a ''limited time.'' What are the time limits on NSB approval—in other words, when does this approval ''run out,'' and how are potential investigators notified when their NSB approval has lapsed? Is it appropriate to issue solicitations for new MRE proposals or award new NSB approvals if other already approved projects have not yet been funded? If so, how do you justify supporting new projects when previously approved projects are still awaiting funding?
ANSWER: In my testimony I noted that for all major projects, including facilities, the NSB approves funding for a limited time. This statement was in reference to the common NSF practice of making award recommendations for a fixed and predetermined multi-year period (an award of limited duration). When the Board exercises its responsibility to review and approve those award recommendations that meet thresholds requiring NSB approval, the Board typically approves the amount and duration of the award as recommended by the cognizant NSF program official. When recipients learn that a proposal was funded, they are fully aware of the size of the award as well as its anticipated ending date.
Page 109 PREV PAGE TOP OF DOC
In the case of awards made under the Major Research Equipment Account, a multi-step process is employed. First, the NSF Director brings before the Board a project for its consideration. In this stage, the Board is asked, in effect, to authorize the Foundation to pursue inclusion of the project in future budget requests. The second stage takes place when and if the project is incorporated into a future budget. In that eventuality, the project comes before the Board again as a part of the overall budget request. When the Board approves the Foundation's budget submission, it reaffirms its support for the MRE projects that are included in the budget request. In the final stage before an award is made the specific MRE project comes to the Board again for approval of funding at a specific level. Additional, subsequent reviews are conducted by the NSB Committee on Programs and Plans at the midpoint of construction and whenever significant issues arise.
The point you raise about approving new projects while other previously approved projects await funding was a critical consideration that led to the development of the Guidelines recently approved by the Board. The provision that ''projects will be authorized close to the time that funding requests are expected to be made'' specifically addresses this concern. It is our expectation that as we gain further experience with the Guidelines we will be able to refine the process to ensure the orderly implementation of these facilities that are so important to the Nation's science and engineering enterprise.
QUESTION 2. In your testimony you defined ''major projects'' as those whose ''costs exceed one percent of the Budget of the sponsoring directorate or office,'' but later you also said that the Board approves ''major programs whose budget exceeds three percent of the budget of the sponsoring directorate or office.'' Please clarify this issue.
Page 110 PREV PAGE TOP OF DOC
ANSWER: The National Science Foundation distinguishes between projects and programs. A ''major project'' as noted in my testimony refers to a single award action typically made in response to a proposal received by the Foundation. Consistent with the Board's delegation of authority to the Director, if the amount of the award for a single project exceeds one percent of the total budget of the directorate or office that sponsors the award, that project is brought before the Board for approval. A ''program'' is the description the Foundation uses to characterize the type of work for which the agency seeks proposals. Multiple awards may be made in one program. For example, theoretical physics is a program and many proposals are considered for funding in that overall subject area. On occasion, special activities, because of their size and complexity, are also treated as programs, as in the case of the Atacama Large Millimeter Array (ALMA). If the budget for a new program [the total amount for all the awards anticipated in that area] exceeds three percent of the budget for the sponsoring directorate or office, the program comes before the Board for approval before proposals are sought.
QUESTION 3: What dialogue has the National Science Board had with the scientific community to determine how best to plan for long-term investments in major infrastructure projects and how does the NSB keep the community informed of its year-to-year funding priorities? What can the National Science Board do to lead the National Science Foundation in developing long-term plans for major investments in scientific infrastructure? What impact do National Academy of Sciences decadal reports have on the planning of long-term infrastructure investments at NSF?
ANSWER: The NSF process leading to the establishment of major infrastructure projects is highly interactive, engaging broad segments of the science and engineering community. Typically, needs for these major facilities are articulated by the scientists working in the fields. They bring their ideas to the Foundation, and, in some cases, to other agencies and organizations that may wish to join a partnership to develop the facility. By virtue of their own close ties to the science and engineering communities, the members of the National Science Board bring an additional perspective to these deliberations. The NSB provides strategic budget guidance to the Foundation that is critical as the agency develops its long-range plans for major investments in scientific infrastructure. These plans are typically the product of intensive discussions between the NSB and NSF's Director. The Board is confident that actions brought before them are based on extended discussion and that they incorporate community views expressed in special studies, such as those produced by Academy committees and other interested parties.
Page 111 PREV PAGE TOP OF DOC
Reports produced by the National Academy of Sciences and other organizations with an interest in ensuring the adequacy of the science and engineering infrastructure are carefully considered as decisions are made about new facilities projects. For example, a recent NAS report addressing issues in astronomy was taken into account as the Foundation considered plans for the ALMA project.
Through its oversight of the NSF budget process and its written Guidelines for Setting Priority for Major Research Facilities (Attachment 1, p. 54) the Board actively communicates its views about which projects should be funded from year to year. These views are clearly and routinely communicated to the Foundation through its regularly scheduled meetings, as well as through the normal discourse among scientific colleagues. Funding priorities for major research facilities, as expressed in the NSF budget, are made public when the President delivers his budget to Congress. This information is made available to the science and engineering communities, the media, and the general public.
QUESTION 4. In your testimony you stated that the NSB performs ''prioritization by group'' of large facility projects but does not order them by rank. The NSB's November 15, 2001 Guidelines for Setting Priority for Major Research Facilities document states, ''Due to cost, not all facilities can be built at the time that their need is determined and plans are in order for construction. Some priority order on facility construction projects must be set'' (emphasis added). How has the Board's policy changed since your testimony? How does it address the need for priority setting in situations in which there are more Board-approved large facility projects than can be accommodated in a given fiscal year?
Page 112 PREV PAGE TOP OF DOC
ANSWER: Since my appearance before your committee on September 6, 2001 a number of developments have occurred that affect the Board's priority setting process for major research facilities. You note one of these developments in your question. That is, the Board's new Guidelines for Setting Priority for Major Research Facilities (Attachment 1, p. 54). These guidelines clearly articulate a number of considerations that the Board examines as it evaluates and prioritizes facilities. They are intended to support an orderly Board process that will enable the accommodation of projects that are approved in a given year by limiting the size of the pool to a level that is consistent with budget realities. However, it is important to recognize that the number of facilities awaiting support reflects the real need felt by the science and engineering community to preserve the Nation's leadership in research. In addition to promulgation of Guidelines, the Board has clearly expressed its priorities for specific projects. On October 11, 2001 it passed a resolution (Attachment 2, p. 55) that reads as follows: ''The Board's highest MRE priorities presented in the budget are ALMA Phase II, EarthScope, and NEON.''
QUESTION 5: As stated in the Charter for the NSB's Committee on Strategy and Budget, of which you are the Chair, ''One of the Board's most important responsibilities is the provision of strategic budget guidance.'' What role does the Strategy and Budget Committee play in reviewing and/or establishing priorities for investment in large facilities, particularly in situations in which there are more Board-approved large facility projects than can be accommodated in that years' budget?
ANSWER: The Board's Committee on Strategy and Budget is its newest standing committee, formally chartered late in 2001. It is charged with framing, examining, and making recommendations to the Board on strategic, typically long-term issues across all NSF programs, and with analyzing the Foundation's budget to ensure progress and consistency against strategic directions for the Foundation. Among its central responsibilities is the assessment of strategic, long-term budget issues—such as major research facilities, placing them in context with other resource demands, and making recommendations to the full Board. As noted in my responses to previous questions, the Board (and the Committee on Strategy and Budget) takes very seriously its role in examining and suggesting priorities for large facilities. The Guidelines document, as well as the NSB resolution stating the highest MREFC priorities presented in the budget (NSB–01–180), attached, illustrate the role played by the committee so far, and I believe illuminate the path its work will take in the future. In addition to these responsibilities more directly related to major research facilities, the committee works closely with the Board's Committee on Programs and Plans, which has primary responsibility for reviewing awards and programs.
Page 113 PREV PAGE TOP OF DOC
75811d.eps
75811e.eps
ANSWERS TO POST-HEARING QUESTIONS
Post-Hearing Questions Submitted to Dr. Christine C. Boesz, Inspector General, National Science Foundation from Subcommittee Chairman Nick Smith
QUESTION 1. In your testimony you expressed concern over the lack of clarity in the Large Facilities Management Plan with respect to chain-of-command and conflict resolution between the Deputy, LFP, and the Program Manager. Please explain this concern more fully and recommend changes that would improve the current plan.
ANSWER: As it is written, the Large Facilities Management Plan (the Plan) describes two organizational structures with respect to large facility projects: one for scientific project management and the other for administrative oversight. For each of these structures, the Plan describes a clear chain of command. For programmatic oversight, the Plan shows lines of ''authority, responsibility and communication from the NSF Director to the NSF Program Manager to the awardee Project Director.'' For administrative or business oversight, the Plan shows authority flowing from the NSF Director to the NSF Chief Financial Officer, to the proposed Deputy, LFP, through the Business Oversight Team and on to the Awardee Institution.
My concern goes to authority for the project as a whole. For many issues that arise during the life of these projects, one or the other organizational component should be able to address them. However, there may be times when these two functional structures may differ on how to proceed; for example, when proposed scientific enhancements conflict with cost and schedule considerations. Who resolves this potential conflict? Will the project be completed on time and on budget, or will it further advance the frontier of science and technology?
Page 114 PREV PAGE TOP OF DOC
Although the Plan does establish Project Advisory Teams to integrate these two viewpoints, these teams are advisory in nature and will be led by the cognizant Program Manager. It is important that NSF clearly identify the ultimate authority for these projects so that it can strike the appropriate balance between good science and good project management. I am not necessarily advocating that a single person or group be designated as having ultimate authority for all projects. While that may be appropriate in some cases, because of the scientific and managerial complexity of these awards, it may not be appropriate in all cases.
My recommendation is simply that NSF include in the Plan a clear method of establishing ultimate authority over each project in order to strike a balance between scientific expertise and practical project management. By addressing this issue upfront, NSF may avoid some potential problems as these projects progress. Staff involved will know from the outset where to turn if conflict should arise.
QUESTION 2. Please elaborate on the training programs that you feel must be implemented for both NSF personnel and MRE awardees and contractors involved in the management of large facilities projects. Will the NSF Academy provide adequate training for the various personnel involved in management and oversight of large infrastructure projects and, if not, what additional training measures would you recommend?
ANSWER: All NSF staff engaged in managing large facility projects should be fully trained to meet the requirements of their respective positions. As recommended in my office's audit of the Gemini Project, NSF should at least ''provide training to all NSF staff engaged in MRE projects on the fund control and project management procedures necessary to effectively manage these programs, including compliance oversight procedures.'' In response to this recommendation, NSF is in the process of ensuring proper training of its staff. While it is not practical or realistic to expect every employee to become experts in the total management of these projects, it is realistic to expect each to become aware of and sensitive to the particular issues facing each discipline, be it financial or scientific. Although it has not created any mandatory training requirements, NSF is beginning to address this need through the creation of the NSF Academy for in-house training. While it is too soon to evaluate whether the Academy is fully meeting this need, NSF is certainly capable of using the Academy to provide the necessary types of training, mandatory or otherwise, to its staff. In addition, NSF staff should still have access to a wealth of external training programs.
Page 115 PREV PAGE TOP OF DOC
Ensuring that MRE awardees and contractors have received necessary training could be accomplished through the use of the management plan that the Plan requires to be submitted with every large facility proposal. This management plan could be a very useful tool for assessing the management capability of prospective awardees, including their level of training and expertise, and for continuing to monitor awardees' progress once the projects are underway. NSF can also use this pre-award stage of a project's life to ensure that potential awardees have met any mandatory training requirements that NSF may wish to impose. In addition, NSF is using its fundamental business process, grant making, to help ensure that the scientific community has access to project management training. For example, NSF is supporting a series of workshops on managing large scientific projects at the California Institute of Technology, the first of which took place on January 18–19, 2002. This series is intended to expose scientists to the practices, techniques, and culture of large scientific construction projects.
QUESTION 3. The management plan is unclear about the parameters and methods of oversight, but appears to rely heavily on reports generated by investigators and their internal management team. How often should NSF personnel actually visit the site or location of an MRE project and who should be involved in such site visits? Are there additional oversight activities that should be considered for future MRE projects?
ANSWER: I believe that it is NSF's responsibility to manage and oversee its awards, including making site visits when necessary. Because each project is different and will have different oversight needs and risks, it is incumbent upon NSF to develop a strategy for determining the frequency of site visits necessary for each project. NSF may find that the frequency of site visits changes with the phase of the project. Also, some projects may have unique issues that require a different frequency of visits. It may not be necessary for NSF to establish a rigid site-visit schedule that is applicable to every project. However, NSF should address the issue of site-visit frequency at the beginning of each new project and assess its value throughout the life of the project.
Page 116 PREV PAGE TOP OF DOC
In addition to site visits and reports generated by awardees, NSF can engage in many oversight activities. For example, prior to making an award, NSF can determine whether proposed project costs are realistic, and whether an entity has proper accounting systems in place to adequately account for Federal funds. NSF can develop budgets at the outset for each phase of the project and then track actual costs against those budgets throughout the life of the project. Also during the life of the project, NSF can ensure that awardees are exercising proper project management. As I stated earlier in response to Question 2, NSF can use the management plan required of the awardees to track progress. This should be a key component of the oversight process and if monitored closely could prevent issues from occurring.
QUESTION 4. It appears that much of the NSF Large Facilities Management Plan is focused on improved financial management of MRE awards. Have there been any problems with the scientific or technical accomplishments of any MRE projects, and does the Large Project Facilities Management Plan include adequate strategies to safeguard against scientific or technical failures?
ANSWER: Assessing the scientific or technical accomplishments of MRE and other large facility projects should be an integral and important part of NSF's oversight process. Establishing internal controls over the programmatic aspects of these projects is as important as establishing controls over the business and administrative functions.
To date, my office has not had a large role in evaluating the scientific or technical accomplishments of any MRE projects. This is partly because the MRE account is relatively new and the majority of the projects funded through it are not yet ready for such an evaluation. Also, in our role as evaluators of NSF's processes, we evaluate NSF's systems and methods for providing scientific oversight of its projects, rather than making that determination ourselves. However, to the extent that our evaluation of NSF's oversight processes identify issues regarding the scientific or technical accomplishments of MRE projects, we report these issues. For example, as part of my office's review of the Gemini Project, we looked into the reasons for the additional costs incurred by this Project. In so doing, we learned of the delay of two or more years in the deployment of the facility-class instruments for the telescopes, causing the Project to rely on borrowed instruments. This delay affected the intended technical proficiency of the telescopes, but the Project has implemented a new management plan that should ensure timely delivery of future instrumentation. Consequently, we made no recommendations. I have attached a copy of this audit report for your review.
Page 117 PREV PAGE TOP OF DOC
The Large Project Facilities Management Plan requires each prospective awardee to submit a management plan with their proposal for funding. These plans can be very useful in ensuring scientific and technical accomplishments as well as monitoring project management. As we saw with the Gemini Project, it was through the use of a rigorous management plan that the Project was able to address the issue of delayed instrumentation, which could have a significant impact on Gemini's operating efficiency and scientific competitiveness. As part of its own internal control process, NSF could use these management plans to ensure that satisfactory technical and scientific progress is being made.
QUESTION 5. It is not uncommon for large construction projects to undergo physical or technical modifications once construction actually begins. At what point in the life cycle of the project should these changes be financed with operating funds rather than construction funds? Have you found examples other than the Gemini telescope project in which the National Science Foundation has authorized the use of operations funds for construction activity in a way you believe to be inappropriate? What policies and guidelines could be implemented in the future to ensure that operations funds are not utilized prematurely or inappropriately for construction activities?
ANSWER: Senator Christopher Bond, the former Chairman of the Subcommittee on VA, HUD and Independent Agencies of the Senate Appropriations Committee, in a letter dated May 22, 2001, stated his concerns about the use of RRA funds to supplement the costs of major research equipment and facilities and requested that my office perform further work to determine if other large facility projects have financial management issues similar to those found with the Gemini project. We have completed the fieldwork for this audit and have issued a discussion draft report to NSF for comment. We expect to issue a final report by March 31, 2002.
Page 118 PREV PAGE TOP OF DOC
In general, we found that NSF does not track the total costs of its large facility projects and is inconsistent in the types of costs it funds through the Major Research Equipment appropriation account. As a result, NSF cannot ensure that projects stay within authorized funding limits. Additionally, NSF cannot ensure it provides accurate and complete information to decision-makers by properly stating total costs for large facility projects. Although NSF has developed policies and procedures to request the initial approval of funding for a large facility project, the lack of comprehensive policies and procedures for managing the complete life cycle of the project has contributed to these financial management issues.
We are recommending that NSF ensure that its financial and management systems and policies provide a total project management focus to identify, record, track and report the full costs associated with all phases of large facility projects from inception to completion.
NSF also needs to ensure that its systems and policies clearly define the types of costs to be funded from the MRE account, as well as from its other appropriations. In many of these large facility projects, the line between construction and operations is not clearly identifiable. For example, while one school of thought may include software development necessary for the operation of a detector to be a construction cost, another may consider this type of cost operational. NSF will need to develop a process to ensure that similar costs are funded consistently. In the fiscal year 2002 House Appropriations Committee Conference Report, NSF received clear guidance on which costs should be charged to the MRE account. Therefore, to comply with Congressional directives and my office's recommendations, and to continue to improve its financial management, NSF will have to adjust its existing financial accounting and management policies, procedures, and systems to support a total project focus and correctly assign costs to its various appropriation accounts.
Page 119 PREV PAGE TOP OF DOC
ANSWERS TO POST-HEARING QUESTIONS
Post-Hearing Questions Submitted to Dr. Christine C. Boesz, Inspector General, National Science Foundation from Ranking Minority Member Eddie Bernice Johnson
QUESTION 1. You indicated in your testimony that, among its oversight activities on facility projects, the IG's office will assess whether proposed project costs are realistic. From your reviews of facility projects, how well has NSF done in the past in estimating project construction costs? Have you developed recommendations on ways for NSF to improve the accuracy of such estimates?
ANSWER: I believe that it is NSF's responsibility to manage and oversee its awards, including determining whether proposed project costs are realistic, ensuring that awardees are exercising proper project management, determining whether an entity has proper accounting systems in place to adequately account for Federal funds, and ensuring that the scientific goals of the project are achieved. These are the types of internal control processes that NSF should have in place to ensure proper use of Government funds.
In its role of ensuring that NSF has these control processes in place and that they are working properly, my office will periodically evaluate NSF's award oversight process, including its process for evaluating proposed project costs, and, where necessary, make recommendations for improved efficiency and effectiveness. My office will also continue to be available to NSF, upon request, to assist in specific areas for which we have particular expertise. For example, we have recently conducted, at NSF's request, a review of an awardee's proposed indirect cost rate to be applied to a large facility project. We have also in the past conducted proposal reviews and will continue to be available to NSF, resources permitting, for these types of consultations.
Page 120 PREV PAGE TOP OF DOC
While my office has not evaluated NSF's overall ability to estimate project construction costs, we have reviewed proposed versus actual project costs as we look at individual projects. For example, in our audit of the Financial Management of the Gemini Project, we found that one of the primary causes for why actual and projected construction and commissioning costs exceeded budgeted costs was that the original budget was unrealistic to meet the scientific needs of the project. Estimating project costs is a process that we will continue to evaluate as we continue to audit NSF's systems and projects.
QUESTION 2. What do you see as the appropriate role for the Deputy for Large Facility Projects, and how should this position relate to that of the Program Manager for a specific facility project?
ANSWER: I believe that it is NSF's responsibility to clearly define the role of the Deputy for Large Facility Projects as well as the role of each project's Program Manager. My concern is simply that NSF communicate these clearly-defined roles and ensure that a method for overall project accountability exists so that the appropriate balance between good science and good project management can be struck. I am not necessarily advocating that this authority be vested in a particular individual or group. However, NSF should include in the Large Facility Projects Management & Oversight Plan a clear method of establishing ultimate authority over each project in order to strike a balance between scientific expertise and practical project management. By addressing this issue upfront, NSF may avoid some potential problems as these projects progress. Staff involved will know from the outset where to turn if conflict should arise.
Page 121 PREV PAGE TOP OF DOC
QUESTION 3. How would you recommend structuring the training program for NSF personnel responsible for management of large facility projects?
ANSWER: All NSF staff engaged in managing large facility projects should be fully trained to meet the requirements of their respective positions. As recommended in my office's audit of the Gemini Project, NSF should at least ''provide training to all NSF staff engaged in MRE projects on the fund control and project management procedures necessary to effectively manage these programs, including compliance oversight procedures.'' While it is not practical or realistic to expect every employee to become experts in the total management of these projects, it is realistic to expect each to become aware of and sensitive to the particular issues facing each discipline, be it financial or scientific. Although it has not created any new mandatory training requirements, NSF is beginning to address this need through the creation of the NSF Academy for in-house training. While it is too soon to evaluate whether the Academy is fully meeting this need, NSF is certainly capable of using the Academy to provide the necessary types of training, mandatory or otherwise, to its staff. In addition, NSF staff should still have access to a wealth of external training programs.
Ensuring that MRE awardees and contractors have received necessary training could be accomplished through the use of the management plan that the Plan requires to be submitted with every large facility proposal. This management plan could be a very useful tool for assessing the management capability of prospective awardees, including their level of training and expertise, and for continuing to monitor awardees' progress once the projects are underway. NSF can also use this pre-award stage of a project's life to ensure that potential awardees have met any mandatory training requirements that NSF may wish to impose. In addition, NSF is using its fundamental business process, grant making, to help ensure that the scientific community has access to project management training. For example, NSF is supporting a series of workshops on managing large scientific projects at the California Institute of Technology, the first of which took place on January 18–19, 2002. This series is intended to expose scientists to the practices, techniques, and culture of large scientific construction projects.
Page 122 PREV PAGE TOP OF DOC
(Footnote 1 return)
NSF's strategic goals include investments in People, Ideas, and Tools. Investments in Tools ''provide state-of-the-art infrastructure for research and education, such as instrumentation and equipment, multi-user facilities. . .Internet-based and distributed user facilities, advanced computing resources, [and databases].'' Ideas are ''discover[ies] across the frontier of science and engineering, connected to learning, innovation, and service to society.'' National Science Foundation, Government Performance and Results Act Strategic Plan FY 2001–2006 6–8 (Oct. 3, 2000) (http://www.nsf.gov/pubs/2001/nsf0104/nsf0104.pdf).
(Footnote 2 return)
Marcia Bartusiak, Einstein's Unfinished Symphony: Listening to the Sounds of Space-Time 141 (2000).
(Footnote 3 return)
Accounting and Information Management Division, U.S. General Accounting Office, Executive Guide: Leading Practices in Capital Decisionmaking (1998).
(Footnote 4 return)
Office of Management and Budget, Circular No. A–11: Capital Programming Guide (Ver. 1.0 1997).
(Footnote 5 return)
National Aeronautics and Space Administration Program and Project Management Processes and Requirements, NPG 7120.SA (1998).
(Footnote 6 return)
Members of my office attended a presentation given last year by the Department of Energy on their project management structure and techniques.
(Footnote 7 return)
Office of Inspector General, National Science Foundation, Audit of the Financial Management of the Gemini Project (Dec. 15, 2000); Office of Inspector General, National Science Foundation, Semi-annual Report to the Congress 6–7 (Mar. 2001).
(Footnote 8 return)
Executive Office of the President of the United States, A Blueprint for New Beginnings: A Responsible Budget for America's Priorities 161 (Feb. 2001).
(Footnote 9 return)
Letter from Christine C. Boesz, Inspector General, National Science Foundation, to Senator Fred Thompson, Chairman, Senate Committee on Governmental Affairs (Nov. 30, 2000) (on file with the National Science Foundation Office of Inspector General).
(Footnote 10 return)
National Science Foundation, Large Facility Projects Management & Oversight Plan NSB–01–153 (Draft, Sept. 2001).