SPEAKERS CONTENTS INSERTS
Page 1 TOP OF DOC
80–811PS
2002
SATELLITE DATA MANAGEMENT
AT NOAA
HEARING
BEFORE THE
SUBCOMMITTEE ON ENVIRONMENT, TECHNOLOGY,
AND STANDARDS
COMMITTEE ON SCIENCE
HOUSE OF REPRESENTATIVES
ONE HUNDRED SEVENTH CONGRESS
SECOND SESSION
JULY 24, 2002
Serial No. 107–80
Printed for the use of the Committee on Science
Available via the World Wide Web: http://www.house.gov/science
Page 2 PREV PAGE TOP OF DOC
COMMITTEE ON SCIENCE
HON. SHERWOOD L. BOEHLERT, New York, Chairman
LAMAR S. SMITH, Texas
CONSTANCE A. MORELLA, Maryland
CHRISTOPHER SHAYS, Connecticut
CURT WELDON, Pennsylvania
DANA ROHRABACHER, California
JOE BARTON, Texas
KEN CALVERT, California
NICK SMITH, Michigan
ROSCOE G. BARTLETT, Maryland
VERNON J. EHLERS, Michigan
DAVE WELDON, Florida
GIL GUTKNECHT, Minnesota
CHRIS CANNON, Utah
GEORGE R. NETHERCUTT, JR., Washington
FRANK D. LUCAS, Oklahoma
GARY G. MILLER, California
JUDY BIGGERT, Illinois
WAYNE T. GILCHREST, Maryland
W. TODD AKIN, Missouri
TIMOTHY V. JOHNSON, Illinois
Page 3 PREV PAGE TOP OF DOC
MIKE PENCE, Indiana
FELIX J. GRUCCI, JR., New York
MELISSA A. HART, Pennsylvania
J. RANDY FORBES, Virginia
RALPH M. HALL, Texas
BART GORDON, Tennessee
JERRY F. COSTELLO, Illinois
JAMES A. BARCIA, Michigan
EDDIE BERNICE JOHNSON, Texas
LYNN C. WOOLSEY, California
LYNN N. RIVERS, Michigan
ZOE LOFGREN, California
SHEILA JACKSON LEE, Texas
BOB ETHERIDGE, North Carolina
NICK LAMPSON, Texas
JOHN B. LARSON, Connecticut
MARK UDALL, Colorado
DAVID WU, Oregon
ANTHONY D. WEINER, New York
BRIAN BAIRD, Washington
JOSEPH M. HOEFFEL, Pennsylvania
JOE BACA, California
JIM MATHESON, Utah
STEVE ISRAEL, New York
Page 4 PREV PAGE TOP OF DOC
DENNIS MOORE, Kansas
MICHAEL M. HONDA, California
Subcommittee on Environment, Technology, and Standards
VERNON J. EHLERS, Michigan, Chairman
CONSTANCE A. MORELLA, Maryland
CHRISTOPHER SHAYS, Connecticut
CURT WELDON, Pennsylvania
NICK SMITH, Michigan
GIL GUTKNECHT, Minnesota
CHRIS CANNON, Utah
FELIX J. GRUCCI, JR., New York
MELISSA A. HART, Pennsylvania
WAYNE T. GILCHREST, Maryland
J. RANDY FORBES, Virginia
SHERWOOD L. BOEHLERT, New York
JAMES A. BARCIA, Michigan
LYNN N. RIVERS, Michigan
ZOE LOFGREN, California
MARK UDALL, Colorado
ANTHONY D. WEINER, New York
BRIAN BAIRD, Washington
JOSEPH M. HOEFFEL, Pennsylvania
JOE BACA, California
Page 5 PREV PAGE TOP OF DOC
JIM MATHESON, Utah
RALPH M. HALL, Texas
PETER ROONEY Subcommittee Staff Director
MIKE QUEAR Democratic Professional Staff Member
ERIC WEBSTER Professional Staff Member
CAMERON WILSON Professional Staff Member/Chairman's Designee
MARTY SPITZER Professional Staff Member
SUSANNAH FOSTER Professional Staff Member
ELYSE STRATTON Majority Staff Assistant
MARTY RALSTON Democratic Staff Assistant
C O N T E N T S
July 24, 2002
Witness List
Hearing Charter
Opening Statements
Statement by Representative Vernon J. Ehlers, Chairman, Subcommittee on Environment, Technology, and Standards, Committee on Science, U.S. House of Representatives
Written Statement
Page 6 PREV PAGE TOP OF DOC
Statement by Representative James A. Barcia, Ranking Minority Member, Subcommittee on Environment, Technology, and Standards, Committee on Science, U.S. House of Representatives
Written Statement
Panel
Vice Admiral Conrad C. Lautenbacher, Jr., Under Secretary of Commerce for Oceans and Atmosphere, National Oceanic and Atmospheric Administration
Oral Statement
Written Statement
Biography
Ms. Linda D. Koontz, Director, Information Management Issues, General Accounting Office
Oral Statement
Written Statement
Biography
Dr. Mark Abbott, Dean, College of Oceanic and Atmospheric Sciences, Oregon State University; Chair, Committee on Earth Studies, National Research Council
Oral Statement
Written Statement
Biography
Financial Disclosure
Page 7 PREV PAGE TOP OF DOC
Discussion
Preparing for NPOESS
Funding Issues for Current Data Products
Metadata Collection and Storage
Archiving Data Products
Meeting Requests for Data Products
Funding Requests
Technology Needs and Planning
Oversight Panel for NPOESS
Modernizing Information Technology
Role of the Private Sector
Aerosol Sensors on NPOESS
Improvements in Predicting Weather
Appendix 1: Additional Material for the Record
Vice Admiral Conrad C. Lautenbacher, Jr., Under Secretary of Commerce for Oceans and Atmosphere, National Oceanic and Atmospheric Administration
Response to Mr. Baca's Hearing Question
Response to Chairman Ehler's Hearing Question
Article for the Record Submitted by Mr. Gutknecht, ''California in the Clouds,'' The Wall Street Journal, July 24, 2002
SATELLITE DATA MANAGEMENT AT NOAA
Page 8 PREV PAGE TOP OF DOC
WEDNESDAY, JULY 24, 2002
House of Representatives,
Subcommittee on Environment, Technology,
and Standards,
Committee on Science,
Washington, DC.
The Subcommittee met, pursuant to call, at 10:23 a.m., in Room 2318 of the Rayburn House Office Building, Hon. Vernon J. Ehlers [Chairman of the Subcommittee] presiding.
80811a.eps
HEARING CHARTER
SUBCOMMITTEE ON ENVIRONMENT, TECHNOLOGY, AND STANDARDS
COMMITTEE ON SCIENCE
U.S. HOUSE OF REPRESENTATIVES
Page 9 PREV PAGE TOP OF DOC
Satellite Data Management at NOAA
WEDNESDAY, JULY 24, 2002
10:00 A.M.–12:00 P.M.
2318 RAYBURN HOUSE OFFICE BUILDING
I. Purpose
On Wednesday, July 24, 2002 at 10:00 a.m., the House Science Committee's Subcommittee on Environment, Technology, and Standards will hold a hearing to examine satellite data management at the National Oceanic and Atmospheric Administration (NOAA). NOAA, through its line office, the National Environmental Satellite, Data and Information Service (NESDIS), procures and operates the Nation's environmental monitoring satellites, which provide raw data and processed data products to the National Weather Service and the Department of Defense for weather forecasting and prediction. NESDIS is also primarily responsible for the long-term archiving and managing of environmental satellite data from all NOAA satellites and for many of the research satellites flown by the National Aeronautical and Space Administration (NASA) for use by researchers and others.
The hearing will focus on three major concerns: (1) NESDIS is not delivering all the weather satellite data products requested by the National Weather Service and the Department of Defense in a timely manner; (2) NESDIS is having great difficulty in maintaining, archiving, and distributing satellite data and data products for researchers primarily because of the tremendous increase in both the volume of data produced by currently deployed satellites and the demand for archived data during the past few years; and (3) NOAA is in the final planning stages for the new National Polar Orbiting Environmental Satellite System (NPOESS), which will cost $6.5 billion and produce hundreds of times more data and subsequent information than today's satellites. The $6.5 billion NPOESS budget plan does not include funding or specific upgrades of NESDIS' satellite data management capabilities either for producing products used in real-time weather operations or for long-term archiving of data for retrieval by researchers.
Page 10 PREV PAGE TOP OF DOC
The Subcommittee plans to explore several overarching questions:
1. Why is NESDIS unable to provide all its satellite data products to the National Weather Service and the Department of Defense for weather forecasting and prediction in a timely manner, and is there a specific plan to correct this situation?
2. What are NESDIS' short- and long-term plans to ensure the maximum use of raw data and data products produced by NPOESS and other new satellites in weather forecasting and other real-time operations?
3. What current challenges is NESDIS facing in archiving today's environmental data, and how is it planning for the dramatic increase in data that will come from future NOAA and NASA satellites?
4. How is NOAA working with the research community to ensure access and usability of current and future environmental satellite data, especially for climate change research?
2. Witnesses
Vice Admiral Conrad C. Lautenbacher, Jr., Under Secretary for Oceans and Atmosphere, NOAA, Department of Commerce.
Ms. Linda D. Koontz, Director, Information Management Issues, General Accounting Office.
Page 11 PREV PAGE TOP OF DOC
Dr. Mark Abbott, Dean, College of Oceanic and Atmospheric Sciences, Oregon State University, and Chair, Committee on Earth Studies, National Research Council.
3. Summary of Issues
According to NOAA's fiscal year 2003 budget summary, NESDIS currently cannot deliver 65 percent of new satellite data products requested by the National Weather Service and the Department of Defense in a timely manner. NESDIS processes raw satellite data into products such as maps and charts of atmospheric ozone measurements, snow and ice cover, and sea surface temperatures. In 1995, NESDIS supported the development and operation of about 100 satellite data products. Today, the backlog of products that cannot be delivered in a timely manner is approximately 325—more than three times the total supported just seven years ago. It is not clear if the problem is strictly one of funding or if there are technological hurdles to overcome. However, more importantly, NOAA has not indicated how this problem affects the ability of the National Weather Service to forecast and predict severe weather. It remains to be seen what improvements in weather forecasting would result from increased distribution of data products. In addition, while NESDIS has requested small increases in its budget for improved product distribution, it continues to spend about the same amount on new product development.
Currently only about 16 percent of NESDIS' raw satellite data is used directly in weather forecasting models. Models play a critical role in real-time weather forecasting. NESDIS sends raw satellite data directly to the computers of the National Center for Environmental Prediction (of the National Weather Service) for incorporation into the Nation's weather forecasting models. However, most of the raw satellite data cannot be translated by the models into usable information because of the need to develop better mathematical algorithms. This problem will be exacerbated by the huge increase in data volume expected to come from NPOESS. NOAA and NASA have developed a Joint Center for Satellite Data Assimilation for the specific purpose of increasing the amount of satellite data used in models. Also, because of the difficulty in developing these algorithms, it has taken on average two years from the launch of a new satellite until any data can be successfully incorporated into the models. Because the average satellite lasts only five years, much of the investment is lost before it can perform this critical function. NOAA and NASA have yet to provide a comprehensive plan, including a projection of required funding to ensure the maximum use of NPOESS and other new NASA satellites in weather forecasting models. While increasing the ability to use data in models is not specifically included in the $6.5 billion NPOESS price tag, NOAA has been touting improved weather forecasting based on models as one of the main benefits to justify the NPOESS project.
Page 12 PREV PAGE TOP OF DOC
A 2001 NOAA report entitled, ''The Nation's Environmental Data: Treasures at Risk,'' states that the massive influx in volume of satellite and other environmental data will far exceed NESDIS' archive and access capabilities. While satellite data and data products are vital to real-time weather operations, archived data and products are also important for climate change and other environmental research. The size of NESDIS' archives more than quadrupled in volume during the 1990s to more than 760 terabytes. By 2005, NESDIS' holdings will be nearly five times this amount, and are expected to grow to 14000 terabytes by 2015, almost 20 times the 1999 level. Much of this increase will come from the new NPOESS project. In addition to the massive increase in data, the number of requests for information is expanding at nearly the same rate. The solution is not just an issue of larger and faster computers. While technology will be the backbone of a new system, new policies, procedures, and standards must be developed to ensure the wide variety of data coming from all sources, not just satellites, can be integrated, accessed, and used by the research community. The NOAA report also contains a chart (attached at the end) indicating that most of NESDIS' data management functions require substantial additional resources. However, NOAA has yet to provide a specific plan how it will develop a new system or how much money it will cost to fulfill its current and expected future obligations.
According to National Research Council report entitled, ''Ensuring the Climate Record,'' by the National Research Council, there are currently no specific funds to archive NPOESS data, and no implementation strategy to archive even the raw data records. Researchers studying the issues surrounding global climate change have a particular need for the kind of repetitive, long-term, high-quality measurements that satellites provide. NOAA weather satellites have provided the best means for securing these measurements. While NOAA has worked with the research community to determine what sensors and measurements are important for climate research, it is equally important to ensure the data systems will meet researcher needs. NOAA has not provided a comprehensive plan to ensure even the basic needs of the research community for a data system will be met.
Page 13 PREV PAGE TOP OF DOC
4. Background
What is NESDIS?
The National Environmental Satellite, Data, and Information Service (NESDIS) acquires and operates NOAA's satellites and manages the processing, distribution, and archiving of their data and other environmental data through its National Data Centers. NOAA satellites are used for ''operational'' purposes, mostly for providing real-time data and products to the National Weather Service, whereas NASA satellites are used mostly for research purposes. NOAA's mission requires at least two geostationary and two polar-orbiting satellites be deployed in orbit at the same time to ensure no gap in coverage. NESDIS also operates three National Data Centers, which together are the largest collection of atmospheric, geophysical, and oceanographic data in the world.
The FY 2003 budget request for NESDIS is $765 million of which $152 million is for regular operations, research and facilities and $613 million is for procurement, acquisition and construction of satellites. NESDIS received $702 million in FY 2002, with a similar ratio between for operations and procurement. Obviously, there is a huge disparity in the cost to build and launch a satellite versus the amount needed to ensure the data is processed into products, distributed, and archived. However, a lack of investment in the latter significantly reduces the usefulness and cost-effectiveness of the satellites. For FY 2003 NOAA requested an increase of $7 million ($28 million total) for satellite data product processing and distribution. NOAA also requested an increase of $2.5 million (total of $26 million) for product development. Oddly, despite its acknowledged deficiencies, NOAA requested a $4.3 million cut ($60 million total) for FY 2003 for its Data Centers and Information Services and stated in its budget summary that the decreased level of funding will increase NOAA's abilities in these areas. It remains to be seen at what levels Congress will fund these activities for FY 2003.
Page 14 PREV PAGE TOP OF DOC
What is NPOESS?
The National Polar-orbiting Operational Environmental Satellite System (NPOESS) is a tri-agency effort between NOAA, NASA, and the Department of Defense (DOD) to combine and integrate the polar satellite needs and capabilities of all three agencies. As with NOAA, the Department of Defense currently operates two polar-orbiting satellites mostly for weather forecasting. NPOESS will replace the four NOAA/DOD satellites with just two that the agencies will share at a total cost of $6.5 billion, split evenly between NOAA and DOD. The estimated savings from this collaboration is $1.8 billion. However, none, if any, of the cost to ensure the maximum use of the data (which will be hundreds of times current levels) is included in the $6.5 billion cost. The first NPOESS satellite must be ready by 2008, to cover the possible failure of the last of the older generation of polar satellites, but it is more likely that the first NPOESS satellite will not be launched until 2010.
NASA is providing technical help and will fly many of the NPOESS sensors on a NASA satellite or airplane starting in 2005 to ensure the sensors work and allow NOAA time a chance to view the data to ensure it can be incorporated into its models and made into products.
NOAA's FY 2003 request for NPOESS is $237 million, an increase of $80 million over last year. DOD is requesting the same amount. The NOAA FY 2004 request for NPOESS is expected to be more than $300 million and remain at that level for several years before declining. Again, this amount does not include any funds for NESDIS to ensure that its satellite data management capabilities are up to the challenge. However, this amount is a significant portion of NOAA's overall budget, and will probably be more than NOAA spends on all oceans and atmospheric research.
Page 15 PREV PAGE TOP OF DOC
5. Questions
The witnesses were asked to address the following questions in their written testimony to the Subcommittee.
Questions for Admiral Lautenbacher, Under Secretary for Oceans and Atmosphere, NOAA, Department of Commerce.
1) NOAA's fiscal year 2003 budget book states that NESDIS currently cannot deliver 65 percent of the new satellite products requested by the National Weather Service and the Department of Defense. Please provide an explanation of why NESDIS is unable to fulfill these requests and what NOAA is doing to correct the problem.
2) What are NOAA's short- and long-term plans to ensure the maximum use of data and products produced by the new Polar Orbiting Environmental Satellite System (NPOESS) and other new satellites in weather forecasting and operations? How much will it cost to implement these plans and is the necessary funding part of the $6.5 billion planned for the NPOESS project? If not, why not?
3) In August of 2001, NOAA released a report entitled, ''The Nation's Environmental Data: Treasures at Risk.'' The report contains a chart that shows NOAA needs ''substantial additional resources'' to fulfill its requirements for end-to-end environmental data management functions. What specific levels of funding are required to fill out the chart? Are NOAA's difficulties limited to funding issues or are there technology problems that need to be addressed as well? To what extent is the increased funding and the technology improvements needed to address these problems already built into the NPOESS project?
Page 16 PREV PAGE TOP OF DOC
Questions for Ms. Linda Koontz, Director, Information Management Issues, General Accounting Office.
1) Please provide a brief overview of NOAA's current Polar-Orbiting Environmental Satellite (POES) program, and how the National Weather Service and the Department of Defense use the data for weather forecasting.
2) Please provide a brief overview of the $6.5 billion new National Polar-Orbiting Environmental Satellite System (NPOESS), and give a status report for the project. How would you rate the overall progress of the project thus far? What are the major differences between the current polar satellite program and the new program proposed by NOAA?
3) What are the key satellite data management challenges that NOAA faces with NPOESS? How is NOAA preparing to meet those challenges, and how much will it cost? Are these costs included in the $6.5 billion price tag for NPOESS? If not, should they be?
Questions for Dr. Mark Abbott, Dean, College of Oceanic and Atmospheric Sciences, Oregon State University, and Chair, Committee on Earth Studies, National Research Council.
1) How and in what ways do researchers use satellite data? Are there differences in how data or products are used for operations, such as by the National Weather Service, and data used for research purposes?
Page 17 PREV PAGE TOP OF DOC
2) What are the current challenges researchers face using satellite data, especially from NOAA?
3) What specific climate change research capabilities will NPOESS provide? What concerns does the research community have about the NPOESS project? How is NOAA working to address these concerns? What recommendations would you give NOAA regarding NPOESS and satellite data management in general?
80811b.eps
Chairman EHLERS. I now call the Subcommittee on Environment, Technology, and Standards to order. We do not yet have a minority member, and that is always a courtesy I try to observe, but with the permission of the minority staff, we will proceed based on information that minority members are on their way.
It is my pleasure to welcome everyone to today's oversight hearing on satellite data management at the National Oceanic and Atmospheric Administration. While the topic may not seem as fascinating as deep-sea exploration or chasing tornadoes, it may actually be more important. NOAA satellites provide the Nation and the world with the critical information that will make much of our forecasting and research on weather, climate, and the whole Earth system possible.
A very important aspect of these satellite systems that seems to get lost is the tremendous investment. Satellites are extremely expensive. NOAA's new National Polar Orbiting Environmental Satellite System, better known as NPOESS, will cost the Federal Government a total of $6.5 billion, that is billion with a ''B.'' This is a staggering sum when you consider that NOAA's yearly budget for all its programs and activities is only about $3.3 billion. Given these huge investments and the importance of satellites to so many aspects of our lives, it is the Science Committee's duty to ensure that the taxpayers are getting their money's worth.
Page 18 PREV PAGE TOP OF DOC
But getting our money's worth is not simply contingent on a satellite being successfully launched and data being beamed down; the key factor is using the data. We must be able to fully collect, process, use, achieve, and retrieve the data from the satellite or the satellite is of minimal use. According to NOAA, it takes, on average, two years from the launch of a new satellite until the data can be used directly in weather forecasting models. Because the average satellite lasts only five years, much of the investment is lost before it can perform this critical function. This is an astounding statement and one I hope our panel can address specifically. The procurement and building of the satellite should go hand in hand with our ability to utilize the data, but unfortunately, as we will hear today, that is usually not the case.
Let me provide some basic background. NOAA's satellite program, the National Environmental Satellite Data and Information Service, known as NESDIS, procures and operates the Nation's environmental modeling—monitoring satellites. It also is responsible for providing raw data and processed data products to the National Weather Service and the Department of Defense as well as the private sector for weather forecasting and prediction. The satellite program is also primarily responsible for the long-term archiving and managing of environmental satellite data from all NOAA satellites and for many of the research satellites flown by the National Aeronautic and Space Administration.
This hearing will focus on three major concerns that NOAA must address for these programs to run more effectively. First, according to NOAA's own documents, its satellite program can only deliver 35 percent of the weather satellite data products requested by the National Weather Service and the Department of Defense. We must know what is preventing them from delivering this information in a timely manner, and how NOAA plans to correct this problem. Second, NOAA's satellite program is having great difficulty in maintaining, archiving, and distributing satellite data and data products for researchers primarily because of the tremendous increase in both the volume of data produced by currently deployed satellites and the demand for archive data during the last few years. We must know what NOAA's long-term plan is for archiving and delivering raw data and process products considering that the new satellites will produce substantially more data over the next 10 years. Third, NOAA is in the final planning stages for the new National Polar Orbiting Environmental Satellite System, which will cost a total of $6.5 billion. However, this figure does not include funding or specific upgrades of satellite data management capabilities either for producing products used in real-time weather operations or for long-term archiving of data for retrieval by researchers. We must know what NOAA's long term plan is for ensuring that the data from the new satellites are fully utilized.
Page 19 PREV PAGE TOP OF DOC
And finally, we need to know what it is going to cost to fix all of the problems of the current data acquisition, processing, and storage systems and what the costs are going to be in the future so that these problems do not recur when the new satellites are launched. To fix these problems, it is going to take bigger and faster computers, knowledgeable programmers, new technology, and new ways of thinking on data processing, storage, and retrieval. However, it would be unfair to the taxpayers to doubt the benefits of a new $6.5 billion satellite system, and then when it has been launched into space, come back and tell us you need another $500 million or so to actually process the data the satellite is sending.
We hope this panel can help us to better understand the problems, the needs, the solutions, and the costs. I thank our distinguished panel for being here today, and I look forward to their testimony. I am now pleased to recognize Congressman James Barcia, the Ranking Minority Member of the Environment, Technology, and Standards Subcommittee for his opening statement. Mr. Barcia.
[The prepared statement of Chairman Ehlers follows:]
PREPARED STATEMENT OF CHAIRMAN VERNON J. EHLERS
I welcome everyone to today's oversight hearing on satellite data management at the National Oceanic and Atmospheric Administration (NOAA). While the topic may not seem as fascinating as deep-sea exploration or chasing tornadoes, it may actually be more important; NOAA satellites provide the Nation and the world with the critical information that make much of our forecasting and research on weather, climate and the whole earth system possible.
Page 20 PREV PAGE TOP OF DOC
Another important aspect of these satellite systems that seems to get lost is the tremendous investment. Satellites are extremely expensive. NOAA's new National Polar Orbiting Environmental Satellite System (NPOESS) will cost the Federal Government a total 6.5 billion (that is Billion with a ''B'') dollars. This is a staggering sum when you realize that NOAA's yearly budget for all its programs and activities is only about $3.3 billion. Given these huge investments and the importance of satellites to so many aspects of our lives, it the Science Committee's duty to ensure that the taxpayers are getting their money's worth.
But getting our money's worth is not simply contingent on a satellite being successfully launched and data being beamed down. The key factor is using the data. We must be able to fully collect, process, use, archive and retrieve the data from the satellite or the satellite is of minimal use. According to NOAA, it takes on average two years from the launch of a new satellite until the data can be used directly in weather forecasting models. Because the average satellite lasts only five years, much of the investment is lost before it can perform this critical function. This is an astounding statement, one I hope our panel can address specifically. The procurement and building of the satellite should go hand-in-hand with our ability to utilize the data, but unfortunately, as we will hear today, that is usually not the case.
Let me provide some basic background. NOAA's satellite program, the National Environmental Satellite Data and Information Service (NESDIS), procures and operates the Nation's environmental monitoring satellites. It also is responsible for providing raw data and processed data products to the National Weather Service and the Department of Defense, as well as the private sector, for weather forecasting and prediction.
Page 21 PREV PAGE TOP OF DOC
NESDIS is also primarily responsible for the long-term archiving and managing of environmental satellite data from all NOAA satellites and for many of the research satellites flown by the National Aeronautical and Space Administration (NASA).
This hearing will focus on three major concerns that NOAA must address for these programs to run more effectively. First, according to NOAA's own documents, NESDIS can only deliver 35 percent of the weather satellite data products requested by the National Weather Service and the Department of Defense. We must know what is preventing them from delivering this information in a timely manner, and what's NOAA's plan to correct this problem.
Second, NESDIS is having great difficulty in maintaining, archiving, and distributing satellite data and data products for researchers primarily because of the tremendous increase in both the volume of data produced by currently deployed satellites and the demand for archived data during the past few years. We must know what is NOAA's long-term plan for archiving and delivering raw data and processed products considering that the new satellites will produce substantially more data over the next ten years.
Third, NOAA is in the final planning stages for the new National Polar Orbiting Environmental Satellite System (NPOESS), which will cost a total of $6.5 billion. However, this figure does not include funding or specific upgrades of NESDIS' satellite data management capabilities either for producing products used in real-time weather operations or for long-term archiving of data for retrieval by researchers. We must know what NOAA's long-term plan is for ensuring that the data from the new satellites are fully utilized.
And finally, we need to know what it is going to cost to fix all the problems of the current data acquisition, processing, and storage systems, and what the costs are going to be in the future, so that these problems do not recur when the new satellites are launched. To fix these problems it is going to take bigger and faster computers, knowledgeable programmers, new technology, and new ways of thinking on data processing, storage, and retrieval. However, it would be unfair to the taxpayers to tout the benefits of a new $6.5 billion satellite system and then when it has been launched into space, come back and tell us you need another $500 million or so to actually process the data the satellite is sending. We hope this panel can help us to better understand the problems, the needs, the solutions, and the costs.
Page 22 PREV PAGE TOP OF DOC
I thank our distinguished panel for being here today, and I look forward to their testimony.
Mr. BARCIA. Thank you, Mr. Chairman, and I want to welcome all of the panel guests to today's hearing and thank the Chair for scheduling this hearing on this very important and timely topic.
Today's hearing seems to be about an abstract topic at NOAA. However, nothing could be further from the truth. Our ability to forecast weather is only as good as the information gathered by the National Weather Service's satellite system and the National Weather Service's ability to process the data. Today's hearing focuses on the vital link between NOAA's ability to process the weather-related data it gathers from its satellite system. We are now several years into the new Polar Satellite Program. The focus of this program is improved data-gathering capabilities.
We want to insure that NOAA is developing a plan to utilize and store the additional information we will obtain from these satellites. Data management, which includes the formatting, processing, archiving, and storage of data received from satellites and other monitoring efforts, is a topic that rarely receives the attention that the building and deployment of new satellite receives. Nor do cost estimates for the deployment of new satellite systems include the increased costs to effectively utilize the data in weather forecasting and research. However, management of the information we receive is essential if we are to continue to improve our ability to forecast the weather and produce timely, accurate information to serve national security needs.
Page 23 PREV PAGE TOP OF DOC
Current and future generations of researchers will want to access and utilize the information we collect today to examine trends in weather and climate. This will not be possible without storing data and its supporting information in an accessible archive. If we do a good job of planning and preparation before these new satellites are launched, we will protect the substantial investment we are making in the development of these satellites.
I am concerned with GAO's findings that NOAA does not have a coordinated and comprehensive plan to manage the data that will be gathered by its newest satellite system. NOAA's past is littered with a disconnect between its procurement activities and data management. I would have hoped that we would have learned from past mistakes. It doesn't matter if we invest billions of dollars in advanced equipment if we don't have the capabilities to effectively utilize the data it is designed to gather.
That is all I have, Mr. Chairman. Thank you.
[The prepared statement of Mr. Barcia follows:]
PREPARED STATEMENT OF REPRESENTATIVE JAMES A. BARCIA
Good morning and welcome to today's hearing.
Today's hearing seems to be about an abstract topic at NOAA. Nothing could be further from the truth. Our ability to forecast weather is only as good as the information gathered by the National Weather Service's satellite system and the National Weather Service's ability to process the data. Today's hearing focuses on this vital link—NOAA's ability to process the weather related data it gathers from its satellite system.
Page 24 PREV PAGE TOP OF DOC
We are now several years into the new Polar Satellite Program. The focus of this program is improved data gathering capabilities. We want to ensure that NOAA is developing a plan to utilize and store the additional information we will obtain from these satellites.
Data management—the formatting, processing, archiving and storage of data received from satellites and other monitoring efforts is a topic that rarely receives the attention that the building and deployment of a new satellite receives. Nor do cost estimates for the deployment of new satellite systems include the increased costs to effectively utilize the data in weather forecasting and research.
However, management of the information we receive is essential if we are to continue to improve our ability to forecast the weather and produce timely, accurate information to serve national security needs.
Current and future generations of researchers will want to access and utilize the information we collect today to examine trends in weather and climate. This will not be possible without storing data and its supporting information in an accessible archive.
If we do a good job of planning and preparation before these new satellites are launched, we will protect the substantial investment we are making in the development of these satellites. I am concerned with GAO's findings that NOAA does not have a coordinated and comprehensive plan to manage the data that will be gathered by its newest satellite system. NOAA's past is littered with a disconnect between its procurement activities and data management. I would have hoped that we would have learned from past mistakes. It doesn't matter if we invest billions of dollars in advanced equipment if we don't have the capabilities to effectively utilize the data it is designed to gather.
Page 25 PREV PAGE TOP OF DOC
As a Member of the authorizing Committee for NOAA's program, I want to know what investments in additional computer capacity, personnel, and research that may be needed to enable us to benefit fully from the investment we are making in these new polar satellites.
I look forward to hearing from our witnesses today and to working with the Administration to ensure the success of this program.
Chairman EHLERS. Thank you, Mr. Barcia. If there is no objection, all additional opening statements submitted by the Subcommittee members will be added to the record. Without objection, so ordered.
Chairman EHLERS. At this time, I would like to introduce our witnesses. First, we have Vice Admiral Conrad C. Lautenbacher, Junior, who is the Undersecretary of Commerce for Oceans and Atmospheres at the National Oceanic and Atmospheric Administration. Next, we are pleased to have Ms. Linda Koontz, who is Director of Information Management Issues at the General Accounting Office, the much-feared governmental agency. And finally, we have Dr. Mark Abbott, who is Dean of the College of Oceanic and Atmospheric Sciences at Oregon State University and Chair of the Committee on Earth Studies of the National Research Council.
As I'm sure our witnesses know or have been told, spoken testimony is limited to five minutes each, after which the members of the Committee will take turns having five minutes to ask you questions. My staff has chastised me for constantly referring to the trapdoor that is going to fall if you don't finish in five minutes, so I have gone high-tech. And instead of that, you will be transported by Star Trek technology, if you exceed your time limits.
Page 26 PREV PAGE TOP OF DOC
We will start with Admiral Lautenbacher.
STATEMENT OF VICE ADMIRAL CONRAD C. LAUTENBACHER, JR., UNDER SECRETARY OF COMMERCE FOR OCEANS AND ATMOSPHERE, NATIONAL OCEANIC AND ATMOSPHERIC ADMINISTRATION
Admiral LAUTENBACHER. Thank you, Mr. Chairman, Congressman Barcia, distinguished members of the staff. It is indeed a great pleasure to be here with you this morning and to discuss this topic, which we believe is very critical to our program and to the needs of our country in terms of the future of satellite, environmental satellite data and usage for the products. I have brought a couple of slides along with me to give you an overview, a quick overview.
I might mention that, first of all, it is very important that—it is a good topic to discuss right now because of the mark-ups that are going on both in the Senate and in the House in order to ensure that we have a balanced program. I will mention a little bit more about that in a minute. But I agree with the opening statements. Data management, data simulation, utilization of data is not a prominent topic. It is a difficult one both for us to push through the Administration as well as it is to justify to Congress. So having this opportunity is really a privilege, and I appreciate that.
Could we go to the next slide, please? Okay. NOAA takes on, in this case, end-to-end responsibility. We look forward to not only, you know, building the requirements, having all of the sexy instruments, looking at all of the wonderful products, but we look at the acquisition, the launch, what you see here, command and control, real-time product development as well as archiving access. And in the end, you see just a sample of the product that you get from the archiving side of this, not a real-time product, but a product that you need from the archiving side. So there are a number of things that we have to deal with here.
Page 27 PREV PAGE TOP OF DOC
Next slide. This slide just gives you an idea of the products in addition to weather. Weather is very important. You mentioned that utilization in weather, but there are a number of other products that come from our imagery that go directly to other users and serve a number of purposes. And you can see some of them on here. I want to mention volcanic ash monitoring. It is very important for safety of our airplanes and aircraft. We have had difficulties in the past without having that data available. Fire detection right now and providing the right data to our firefighters for where the smoke is going. These things are very important, and they come directly out of these products. And you can see some of the aerosols, of course, are very important to our climate, examinations for the future and for a global observing system that we are going to need to do the kinds of things we need to in determining the future for climate in the world, for that matter, not just our country.
Next slide. Okay. This gives you an idea of the size of NOAA's budget and the piece of which satellites is—you have got about $765 million out of $3.3 billion. You can see that most of the money that comes for our data centers and for the data handling that we are talking about today is in the NOAA data centers and the environmental satellite observing services. That is not all data assimilation and data handling, but that is the place where it fits in. So it is a piece of that, and it is not a big piece. It is a relatively small piece.
Go ahead. Next slide, please. This gives you a feel for the parts of the budget that—in that big pie chart that directly affect the subject that we have today. And as you can see, we have asked for substantial help and improvement in this area from '02 to '03, $26.6 million in product processing distribution. This is the big chunk that provides the types of data we need now, weather system models and our real-time utilization, data assimilation, that sort of thing. And we have a joint center. We are asking for another increment to build the joint center for satellite data assimilation. I can explain that later in the questioning.
Page 28 PREV PAGE TOP OF DOC
We are continuing to invest in our large scale archiving system called CLASS. And we are also looking for an increment to improve our archiving and use of the satellite data and research satellites that are in the Earth observing system from NASA. So the significant request in here for improvements to solve some of the issues that you have brought up there, and we, as I have in my original testimony here, ask for continued support on this and would be delighted to explain more of that.
Next slide. I just want to mention NPOESS is critical to our future. It is, itself, a system, which is saving money, because it is a combination of a separate system that the Department of Defense used to have and a separate system that NOAA has. This is a great step forward in terms of being economical with taxpayers' dollars. We are on track, on scale. We have the money in our budget to do this. It has been supported by the Administration, and we ask for your help with this next great leap forward. We need this satellite to come on place in time, because the ones that are up there will run out in '08, '09. It is tightly scheduled.
Next slide. I see, and my time is up here. These are the instruments. We have a whole group of new instruments. Next slide, please. They cover a wide variety of issues from atmospheric, oceanic, terrestrial space, climate. They will—about 20 times more data will be coming out of this satellite as we have previously thought.(see footnote 1)
Next slide. I want to mention that this is the standard way of doing business for funding. What you see in green basically is system funding. The archiving, access, data usage, individual agency usage is covered in separate budgets. That is the traditional way of doing business, because each agency in the civilian world has different uses for the data. NOAA is covering our part of this.
Page 29 PREV PAGE TOP OF DOC
Next slide. We have a plan to use this data. We are planning to use NPOESS data on day one of its availability. We have done a number of things to ensure that. I would be happy to talk about some of these during the questioning.
Next slide. This is our status of environmental data management. You need to look at the green and the red checks. And you can see that as of last year, we are not doing so well. There are lots of things that need to be improved. And particularly, we talk about the future satellites.
Next slide. We believe that if we can get the budget request that we have asked for, it will improve the situation to the chart that you see here. A number of areas will be improved. We still have work to do, but I believe the size of the increments we have asked for are prudent in terms of executability and the natural progression of building the capability to handle data.
And with that, let me stop. Thank you very much. I am sorry for using up too much time, but I would be delighted to answer any questions on this program.
[The prepared statement of Admiral Lautenbacher follows:]
PREPARED STATEMENT OF CONRAD C. LAUTENBACHER, JR.
Thank you, Mr. Chairman, and members of the Subcommittee for providing us the opportunity to testify on the President's FY 2003 Budget Request for the National Oceanic and Atmospheric Administration (NOAA) and specifically NOAA' Satellite Data Utilization and Management activities.
Page 30 PREV PAGE TOP OF DOC
The FY 2003 Budget Request supports and enhances the goals of the President and the Department of Commerce's activities to provide satellite, data, and information products that support public and private services and contribute to the quality of life of all Americans. In addition to supporting the Nation's economic growth, satellite, data and information products contribute to National and Homeland Security efforts, as well.
Today, I will discuss with you how NOAA is addressing the challenges, significant opportunities, and initiatives underway to exploit the current and future streams of satellite data. I will highlight the following:
Value of NOAA Satellite Data;
Overview of NOAA's Satellite Programs;
FY 2003 Funding Request;
Satellite Data Utilization and Data Management;
Deficiencies in FY 2002 Satellite Data Processing and Distribution;
NOAA's Plans to Ensure Maximum Use of NPOESS Data; and
Archiving challenges From NPOESS and NASA EOS Satellites.
Page 31 PREV PAGE TOP OF DOC
A. Value of NOAA's Satellite Data
NOAA's services promote, protect, and enhance the Nation's economy, security, environment, and quality of life. NOAA's satellite observations are major contributors to these services. Since the launch of the first weather satellite, Television Infrared Observation Satellite (TIROS–1), on April 1, 1960, NOAA, including its predecessor organizations, has been providing the Nation with continuous global environmental observations.
The data and products derived from these satellite observing platforms have not only revolutionized our ability to forecast the weather, but have also contributed to our understanding of climate, the oceans, and other aspects of the Earth's environment. Satellite imagery and derived products have become a critical data source for severe weather forecasting and are essential for the tracking and prediction of hurricanes and typhoons. To quantify the impact of satellite data, I offer the following example: Today, our skill for 5-day forecasts equals our skill for 3-day forecasts in 1985. This improvement can be attributed to improved modeling and global observations—of which 97 percent is derived from satellites. As part of our future plans, our goal is to make accurate forecasts to seven days and beyond. This goal can only be realized with advances in modeling and new satellite observations.
Virtually every sector of the Nation's economy relies upon rapid and reliable access to environmental data and information. NOAA environmental data form the basis for making decisions that have far-reaching economic consequences at local, regional, and global levels. These data are distributed to, and used by government, commerce, industry, science, engineering, and national defense. NOAA contributes to the national economy by providing environmental data for energy distribution, the development of global food supplies, and management of natural resources. Our environmental satellite observations are an important contribution to U.S. national security by providing military users real-time and near real-time information for aircraft, ships, and facilities around the world.
Page 32 PREV PAGE TOP OF DOC
For climate observations, the President has called for programs which can reduce the uncertainty of the climate. While models play a role, calibrated global observations are key to our understanding of climate. Satellites are the only global observing platform for much of the atmosphere, oceans, and land surface.
B. Overview of NOAA's Satellite Programs
NOAA's National Environmental Satellite, Data, and Information Service (NESDIS), the Nation's only civil operational space organization, operates the United States' geostationary and polar-orbiting environmental satellites. NESDIS includes three National Data Centers, which manage the largest collection of atmospheric, geophysical, and oceanographic data in the world.
Geostationary Operational Environmental Satellites (GOES)
NOAA operates two environmental satellites in geostationary orbit above the Equator, known as the Geostationary Operational Environmental Satellites (GOES). They monitor North and South America and most of the Atlantic and Pacific Oceans. The two GOES satellites operate day and night to provide satellite images and critical data to users throughout the Western Hemisphere.
To keep pace with the growing needs for GOES data and products, NOAA is evolving its geostationary remote-sensing capabilities with the development of GOES–R, scheduled for launch in 2012. Upgrades to NOAA's environmental observation requirements in the GOES–R series focus on greater temporal and horizontal resolutions, improved product accuracies, and extended geographical coverage.
Page 33 PREV PAGE TOP OF DOC
Polar-orbiting Operational Environmental Satellites (POES)
Complementing the geostationary satellites are the polar-orbiting satellites, known as Polar-orbiting Operational Environmental Satellites (POES). Continuously circling Earth in sun-synchronous orbit, these satellites support global weather and marine forecasts. Operating as a pair, these satellites ensure that observations for any region of Earth are no more than six hours old. The METOP satellite partnership between NOAA and its European partners is designed to continue satellite data availability through joint partnership in our polar satellite program.
The Nation's responsibility for continuity of operational polar-orbiting services lies with the National Polar-orbiting Operational Environmental Satellite System (NPOESS) program. This is a joint Department of Commerce (DOC)/NOAA, Department of Defense (DOD), and National Aeronautics and Space Administration (NASA) program merging the current POES and Defense Meteorological Satellite Program (DMSP) systems into a common system of polar satellites. The goal is to provide meteorological, oceanographic, terrestrial, climate, space environment and other environmental data products for both civilian uses and military operations.
The GOES, POES and NPOESS programs are critical building blocks to the development of an integrated observation strategy for the 21st Century.
II. FY 2003 Funding Request
The President's FY 2003 Budget Request for NOAA is $3,330.5 million in total budget authority. Of that amount, the FY 2003 request for NOAA's Satellite and Information Service is $764.7 million. The funding requested in the FY 2003 President's Budget Request will allow NOAA to ensure that our mission for environmental stewardship, assessment, and prediction of the Nation's resources is fulfilled, and continues to excel in our science and services to the American people.
Page 34 PREV PAGE TOP OF DOC
Of the total $764.7 million in the NESDIS budget request, $360.2 million (47.1 percent) is allocated to polar orbiting satellites, $227.4 million (29.7 percent) to geostationary satellites, $91.8 million (12 percent) to environmental satellite observing services, and $60.1 million (7.9 percent) to NOAA data centers and information services. The remaining $25.2 million (3.3 percent) goes for construction and other costs. Later in my testimony, I will cover funding for specific initiatives in FY 2003 for satellite data utilization.
80811c.eps
III. Satellite Data Utilization and Data Management
In support of NOAA's mission, it is critical to ensure that the data and products from our satellites are responsive to users' needs in terms of timeliness, quality, and delivery. To address this challenge, NESDIS has been re-engineering its planning processes to fulfill our end-to-end responsibility. Our improved planning approach looks systematically at the full program cycle: from requirements and planning, to acquisition, launch, command and control, real-time product development, archive and access, assessments, and user services.
A major part of the end-to-end approach includes a comprehensive requirements process. NOAA's satellites are just one of the major observing tools we use to meet our overall mission of environmental monitoring and marine resource stewardship. This requirements process will lead us to defining a single civilian space architecture capable of serving NOAA's diverse requirements for space-based environmental observations. Our program planning and funding requests will be developed in the context of this end-to-end approach.
Page 35 PREV PAGE TOP OF DOC
With improvements in satellite instrumentation, and the growth of a more sophisticated and knowledgeable user community, NOAA faces major challenges to provide an ever increasing number of satellite products to a broader and more demanding user base. Explosive growth is occurring in both the number of research and operational environmental satellite missions in orbit.
NESDIS presently acquires and processes data from NOAA's two GOES and two POES satellites, as well as from DOD's two DMSP polar orbiting satellites. Additionally, NESDIS acquires and processes data from a number of non-NOAA, non-DOD spacecraft. The satellite data acquired each day by NESDIS total around 130 gigabytes, and enable NOAA to provide urgently needed information to our customers.
Data from non-NOAA research and operational satellites provide information capabilities not available from NOAA spacecraft that can be used to allow us to improve support for our diverse user community. In addition, our scientists, analysts, and future systems designers can conduct risk reduction assessments in preparation for the introduction of a future NOAA operational capability.
Some of the non-NOAA instruments for which NOAA processes data include:
improved imagery, surface temperature data, and advanced atmospheric soundings from the NASA Earth Observing System (EOS) Terra and Aqua missions;
Synthetic Aperture Radar (SAR) providing all weather, day and night images from the Canadian RADARSAT satellite;
Page 36 PREV PAGE TOP OF DOC
Meteosat winds and imagery from Europe;
Japan's GMS winds and imagery;
ocean surface wind measurements from NASA's QuikSCAT and Europe's ERS; and
ocean color data from the commercial ORBVIEW–2 satellite.
Currently, nearly 300 POES and about 200 GOES products are distributed to a variety of Federal, international, State, university, private, and public users. Many of these products are created many times throughout the day, such as the GOES high density winds, which are created 48 times in a 24-hour period. As a result of an increasing number of satellite products and applications, NESDIS serves and interacts with a growing number of user communities.
These communities include:
Weather forecast offices throughout the Nation and the Western Hemisphere;
NOAA's Environmental Prediction Centers specializing in hurricane, severe storm, aviation, space environment, hydrological, ocean, and climate forecasts and environmental modeling;
NOAA Research Laboratory; the national and international climate community concerned with climate change and ozone levels;
Page 37 PREV PAGE TOP OF DOC
Federal, State, and local resource managers responsible for coastal environmental monitoring; officials reacting to environmental hazards, including firefighting authorities, including but not limited to:
Federal Aviation Administration (FAA) and the aviation community
U.S. Forest Service (USFS)
Federal Emergency Management Agency (FEMA)
National Interagency Fire Center and all sectors of the wildland firefighting community
U.S. Coast Guard, including search and rescue efforts, and support to shipping and fishing industries
U.S. Department of Agriculture (USDA) for agricultural and drought monitoring
NASA
U.S. Geological Survey
U.S. Department of State and U.S. Agency for International Development in support of U.S. foreign policy directives
U.S. Department of Energy
Page 38 PREV PAGE TOP OF DOC
U.S. Department of Defense, all services (Navy, Army, Air Force, Marines) to support national security interests via the NOAA/DOD Shared Processing Program.
NESDIS' data is also used by the Government and academic research community, such as Rutgers University, the Universities of Colorado, Miami, Wisconsin; Massachusetts Institute of Technology; the University Corporation for Atmospheric Research; and the Office of Naval Research.
NESDIS' data is also used by many international users, such as meteorological and climate data centers in Great Britain, Germany, India, Switzerland, Canada, and Australia; shipping and fishing interests, and media outlets. The DOD also receives acquired NOAA POES data.
In addition to other operational numerical weather prediction centers, including DOD and international customers, the National Weather Service (NWS) and its National Centers for Environmental Prediction (NCEP) are primary consumers of satellite data. NWS Weather Forecast Offices (WFOs) use much of the full resolution satellite stream to produce the higher resolution, interactive products required to enhance public safety.
NESDIS has strong partnerships with the NCEP Environmental Modeling Center (EMC), Tropical Prediction Center (TPC), and Climate Prediction Center (CPC). NESDIS and EMC are conducting a number of joint projects to accelerate the use of satellite data in numerical weather prediction. NESDIS and TPC scientists are exploring the optimum utilization of satellite-derived temperatures and winds in hurricane intensity investigations. NESDIS ozone scientists work very closely with CPC scientists responsible for monitoring the stratosphere, and are collaborating with TPC on several tropical storm research projects.
Page 39 PREV PAGE TOP OF DOC
NOAA also established the CoastWatch program, which is a national network of eight regional offices (North Carolina, Michigan, California, Florida, Hawaii, Mississippi, Rhode Island and Alaska) that provide assistance and satellite data to coastal managers, forecasters, and researchers. Each office is located within an appropriate NOAA line office activity in the region (for example, a National Marine Fisheries Laboratory or NWS Regional Office). By virtue of the CoastWatch regional offices being located on-site along the U.S. coast, we have intimate contact with our users and their resource management issues. CoastWatch helps users to become more aware and able to make use of satellite data in local and regional coastal and ocean resource management activities. Many users rely on NOAA's satellite-derived coral bleaching products to support critical natural resource management issues. Other examples of users within NOAA are provided in Appendix 1.
NOAA is constantly looking for ways to improve the efficiencies of its data management functions. Through programs over the past decade such as the National Virtual Data System (NVDS), NOAA was able to manage a 10-fold increase in data archived, a 50-fold increase in the number of users, with just a 40 percent increase in budget (when adjusted for inflation) and a 30 percent reduction of federal FTE.
A. Satellite Data Processing and Distribution
The NESDIS Office of Satellite Data Processing and Distribution (OSDPD) manages NOAA's central ground facilities, which process and distribute satellite data and products, derived from multiple sources. These data and products support U.S. Government time-critical mission requirements, such as NWS numerical weather prediction models.
Page 40 PREV PAGE TOP OF DOC
Since the late 1980's, the number of satellite products that NESDIS produces to support NWS and DOD requirements has grown from 40 to 500, and the number of satellites used to create these products have increased from 6 to 18.
80811d.eps
80811e.eps
NESDIS has continually focused on maximum exploitation of its available resources; nonetheless, a backlog of new satellite products required by NWS, DOD, and other operational users has accumulated. These products and services, which include valuable flood, fire and volcano products, could improve warnings and forecasts if implemented, helping to save lives and property in many parts of the United States. The reliability of NESDIS' satellite data processing operations has also degraded, resulting in instances of delay in delivery of satellite-based information required for severe weather support.
To help correct these problems, the President's FY 2003 President's Budget Request includes four new initiatives to ensure our Nation's ability to best utilize satellite data by addressing the following issues.
Reducing the Risk to Continuity of Critical Operations ($3.05M). This effort provides support to ensure the continuity of critical satellite product processing and distribution capabilities that supply 97 percent of the data used in NWS numerical weather prediction models.
Page 41 PREV PAGE TOP OF DOC
Environmental Algorithm Development for Climate Monitoring & Hazards ($0.5M). NOAA will use the funds to develop advanced algorithms and methods to handle higher resolution data and techniques to maximize the information content of current and near-future satellite data.
Improved Support for Weather and Hazards Products ($2.0M). These funds will accelerate the deployment of satellite products into operations and provide for their continued support.
Joint Center for Satellite Data Assimilation ($2.6M). These funds will be used to in a partnership program with NASA to more fully utilize satellite data to improve weather forecasts and warnings, and improve accuracy and extend the time range of weather and climate forecasts.
B. NOAA's Plans to Ensure Maximum Use of NPOESS Data
Driven by user requirements, the increasing numbers of satellite missions are placing ever-increasing demands on NOAA's data processing, distribution, archiving and access systems. No previous decade has ever had the magnitude of changes in the volume of data coming into NOAA for processing and archiving that were experienced in the 1990s. However, that explosive growth will be surpassed by what will take place between the year 2000 and 2015. Even as current observing systems continue to provide data, new satellite systems, such as NASA's EOS, the NPOESS Preparatory Program (NPP), and NPOESS itself are, or will be, going into operations. These systems will provide massive amounts of new data that will present major opportunities for NOAA.
80811f.eps
Page 42 PREV PAGE TOP OF DOC
80811g.eps
1. What does NPOESS funding include?
The tri-agency effort by DOC/NOAA, DOD, and NASA established the NPOESS Integrated Program Office (IPO) to develop and acquire the Nation's future polar-orbiting system. The NPOESS operational suite of instruments will provide continuity to the NOAA POES series and DOD series of polar-orbiting, operational environmental satellite systems incorporating new technologies from NOAA, DOD, and NASA. The first NPOESS satellite is scheduled to be launched in April 2009. Since NPOESS is a joint program, it serves both military and civilian users. It was decided early on that NPOESS funding should include requirements definitions; mission planning; systems definition; systems acquisition; and systems operations and maintenance. These functions include satellite system acquisition and launch, ground systems, command, control and acquisition, and near real-time environmental data record distribution to DOD and NOAA centers. The funding does not include some data management functions such as metadata, archiving, non real-time access, and archive migration to new media. These long-term management functions were determined to be the responsibility of NOAA. NOAA has responded to these NPOESS data archiving challenges, as well as to requirements to provide long-term management of other large data sets such as radar, by defining and planning a data modernization program called the Comprehensive Large Array-data Stewardship System (CLASS), wherein all these data could be handled the same way, and therefore used together in building applications. The FY 2003 budget includes $3.6 million for this purpose. It also has a separate budget line of $3 million to begin archiving EOS data into CLASS. By 2010, the total archived data forecast to be managed will be about eight petabytes. The resources required for this volume of data, given today's projection of efficiencies, would be in the range of a few tens of millions of dollars per year. Another way to estimate the data management requirements would be approximately 5–10 percent of the systems budget.
Page 43 PREV PAGE TOP OF DOC
2. Validated Requirements
NOAA will be able to use the data from NPOESS to meet the requirements established in the validated and approved NPOESS Integrated Operational Requirements Document (IORD–II). The improved atmospheric, oceanic, terrestrial, space environmental, and climatic data that will be delivered from the more capable imaging and sounding instruments on NPOESS will be used to improve NOAA's short-term weather warnings and forecast services for protection of life and property, longer-term climate change assessment and prediction, and a variety of other real-time applications. These data will not only be utilized directly in global, regional, and local data assimilation models to support numerical atmospheric and oceanic prediction, but the imagery, soundings, and other NPOESS data will support the generation of over 500 distinct products that will be delivered daily to users in NOAA, DOD, other civilian agencies, and universities. To prepare for these new data, NOAA has begun several efforts to use new data faster, to ameliorate the risks of satellite failures, and to train and educate the community on the applications possible with NPOESS data.
3. Use of Satellite Data in Numerical Models
Much progress has been made in the utilization of satellite data since the first meteorological satellite was launched in 1960. During the 1960s and 1970s, the advances in satellite instruments raced ahead of computing capacity and analysis techniques needed to use these data effectively in weather forecasts. Satellite imagery analysis by local weather forecasters was an immediate success, but the use of quantitative satellite data and products in computerized weather forecasting lagged behind. By the early 1990s, however, very fast computers and sophisticated methods of merging satellite data with numerical forecast models were becoming available.
Page 44 PREV PAGE TOP OF DOC
This enabled, for the first time, direct assimilation of satellite radiance data into U.S. operational weather forecast models, which led to significant improvements in weather forecast accuracy. Data assimilation is the process by which weather observations and a short-term (e.g., six hours) forecast of the weather variables are merged to obtain the initial conditions needed to make a numerical weather forecast. The important scientific advances that made direct data assimilation possible were: 1) the development of fast radiative transfer models that allow transformation of weather forecast model variables, such as atmospheric temperature and humidity, into radiance, the quantity measured by weather satellites; and, 2) techniques by which the model variables are modified so that the radiance derived from the models matches the satellite observations. With these advances, the temperature, moisture and wind information inherent within the large volume of satellite data can be used by the models to improve operational weather and climate forecasts. Increased user demand and evolution of satellite capabilities are now imposing a requirement for significant scientific effort to fully exploit the new data.
4. Joint Center for Satellite Data Assimilation
To improve the exploitation of current satellite data and to prepare for quicker use of future data in operational weather forecasts, NASA and NOAA have formed a collaborative Joint Center for Satellite Data Assimilation (JCSDA). It addresses technical issues discussed above, and development of an end-to-end process for the operational utilization of satellite observations. JCSDA will be a center distributed among several centers of expertise. Within NOAA, those centers will include NESDIS, NWS, NCEP, and the Office of Atmospheric Research (OAR). The NASA Data Assimilation Office (DAO) is also a full partner in JCSDA. Each will bring its own area of expertise to the joint effort. This collaboration will make efficient and rapid advances in the use of satellite data in weather forecast models.
Page 45 PREV PAGE TOP OF DOC
JCSDA will promote the development of common weather forecast models for research and operations. Now, each U.S. forecasting center runs its own models, and data assimilation advances made at one center are not easy to transport to the others. Common models will make this process efficient. Components required by data assimilation will be developed for community use. This will include community radiative transfer models, surface emissivity models, and surface physics models. Recent accomplishments of the JCSDA include the operational implementation of QuikSCAT wind and TRMM precipitation data into the NWS/NCEP operational models.
This is the first time NOAA has used data from research satellites controlled by NASA within the operational data stream used by NWS. There have also been major strides in preparation for NASA's Aqua launch this spring so that an accelerated assessment and use of its advanced atmospheric sounding data can occur. These activities are projected to be completed within 10 months, as opposed to the two years it has taken in the past to use new satellite data operationally.
80811h.eps
Much of the early NPOESS data utilization/demonstration work that NOAA has begun will form the core of the research to operations transition efforts that will be conducted under JCSDA. To support the ongoing JCSDA projects, NOAA, NASA, and IPO invested approximately $2.5 million in FY 2002. Sustained investment in JCSDA will be required to attack existing satellite data assimilation challenges, and to ensure that NPOESS data will be fully utilized when the system becomes operational. DOD has recently been invited to become a partner in JCSDA. Participation of the DOD Operational Processing Centers in JCSDA will help ensure that satellite data processing and assimilation methods developed for NPOESS will be applicable to the NOAA, Navy, and Air Force processing centers that will be the primary recipients of the NPOESS data.
Page 46 PREV PAGE TOP OF DOC
To assist in this effort, in FY 2003 NOAA has requested an additional $2.6 million for the Joint Center for Satellite Data Assimilation. Using these funds, JCSDA will develop new and powerful mathematical techniques to assimilate current and NPOESS-era satellite data into numerical weather prediction models thus improving the accuracy and extending the time range of weather and climate forecasts.
5. Risk Reduction
The utilization of research satellites by operational agencies is also very important in risk reduction for future operational systems and technology transfer. The groundwork for use of NPOESS data will be laid through a series of systematic risk reduction projects leading from early use of new data from current satellite missions being flown by NASA, to the joint IPO/NASA NPP, which will be launched in 2006, to the first operational NPOESS spacecraft that will be available for launch in 2008 and is expected to be launched in early 2009. To date, NOAA, with support from IPO and NASA (NPP program support) has invested approximately $6.6 million on a series of pre-operational demonstrations and utilization of data from NPOESS-like instruments to ensure that NOAA will be ready to use NPOESS data when the new system becomes operational. Over the next six years, NOAA and IPO are planning to directly invest approximately $10 million to continue the development of the processing capabilities for key sensors.
Presently, NOAA is upgrading and enhancing current processing capabilities to begin acquiring and exploiting in near-real time data from the Moderate Resolution Imaging Spectroradiometer (MODIS) and the Advanced Infrared Sounder (AIRS) on the NASA EOS Terra and Aqua missions. The MODIS instrument is very similar to the Visible/Infrared Imager Radiometer Suite (VIIRS), and the AIRS instrument is similar to the Cross-track Infrared Sounder (CrIS) that will be flown on the NPP mission and on the operational NPOESS spacecraft. NOAA efforts with data from these instruments are critical to reduce the risk and gain experience with data handling, processing, storage, and communication of high volume data sets from similar instruments; and to allow the users to gain early, pseudo-operational experience with NPP and NPOESS-like data sets, well before the first operational NPOESS spacecraft is launched.
Page 47 PREV PAGE TOP OF DOC
Similar efforts are being pursued to build the capability to handle and process data from the future Conical-scanning Microwave Imager/Sounder (CMIS) that will be flown on NPOESS to measure, among other parameters, the ocean surface vector wind field. Current efforts at NOAA (and the Navy) address the operational/tactical use of ocean surface vector winds from active scatterometer missions (e.g., SeaWINDS). Beginning in January 2003 with the launch of the joint Navy/IPO/Air Force Coriolis/WindSat mission as risk reduction for the CMIS instrument scheduled to fly on the first operational NPOESS, the NOAA processing capabilities for SeaWINDS will be redirected to processing and utilizing data from the WindSat mission.
Beginning with its launch in 2006, NPP will supply data on atmospheric and sea surface temperatures, humidity soundings, land and ocean biological productivity, and cloud and aerosol properties. NPP will contribute to instrument risk reduction by offering early instrument and system level testing, lessons learned for design modifications in time to ensure NPOESS launch readiness, ground system risk reduction, early user evaluation of NPOESS data products, such as algorithms and instrument verification, and opportunities for instrument calibration. The IPO will deliver an Integrated Data Processing System (IDPS) to NOAA in 2004 to support the processing of Environmental Data Records (EDRs) from NPP. A prototype NOAA system to further process the NPP data into derived products and real-time applications, distribute the products to users, and archive these data will be developed, evaluated, and perfected with data from the NPP satellite two to three years in advance of the launch of the first operational NPOESS spacecraft.
NOAA is also involved in significant risk reduction for the next generation of advanced geostationary satellites, the GOES–R series. The requirements for the GOES–R series translate into a significantly increased volume of data from the preceding GOES–N series. The impacts of this increase in data must be identified and addressed in the areas of data acquisition, product processing, distribution to real-time users, access and application by users, and retrospective data access and archive. NESDIS has established a user-wide, end-to-end system requirements and implementation process by which impact and development in each of these areas is addressed. Most notably, studies are already underway to consider alternative communication capabilities, which extend from expanded on-board processing to data compression and increased bandwidth. Interface planning documents are currently being developed to identify needs of the NESDIS Operational Data Acquisition, Product Processing and Archive centers, and most importantly, the user agencies, in order to conduct comprehensive and integrated budget and program planning for each system to achieve full readiness by the first GOES–R series launch in 2012.
Page 48 PREV PAGE TOP OF DOC
A major risk reduction program for the future operational GOES–R satellites is the Geosynchronous Imaging Fourier Transform Spectrometer (GIFTS) project. GIFTS is a tri-agency joint mission with NASA's New Millennium Program (NMP), the U.S. Navy's Office of Naval Research, and NOAA. GIFTS is a sounder with high spatial resolution and very high spectral resolution and will significantly improve the accuracy of temperature and moisture observations and our observations of severe weather. GIFTS, to be launched in the 2005 time frame, will significantly improve our ability to observe, analyze, and predict weather, and will enable scientists and meteorologists to forecast weather with a higher level of accuracy. This capability is especially important in forecasting severe weather, such as thunderstorms and tornadoes. GIFTS will also greatly improve the diagnostic and prediction of tropical storm intensity. NESDIS and NWS will use GIFTS data to improve short-term weather forecasting and to improve hazard monitoring (volcanic ash, wildland fires, and pollutants). However, sufficient resources are needed to ensure that products are delivered to NWS and utilized. GIFTS will offer NOAA early access to advanced sounding products expected from their future GOES–R advanced baseline sounder.
6. Training and Education
A major challenge is training and educating NOAA's large and varied user community on the nature and use of satellite data and products. Improved data utilization is planned by increasing training, user interactions and data access through websites. Currently users can access data and many of our operational products through the Internet. Examples include imagery from GOES, ocean color from commercial systems, coral reef bleaching, ocean winds and altimetry, atmospheric temperatures, surface temperature, snow cover and ice, aerosols, fires, clouds and vegetation index.
Page 49 PREV PAGE TOP OF DOC
NOAA invests in classroom and computer-based training through the Cooperative Program for Operational Meteorology, Education and Training (COMET) and a joint NESDIS/NWS and Cooperative Institute [Cooperative Institute for Research in the Atmosphere (CIRA) and Cooperative Institute for Meteorological Satellite Studies (CIMSS)] program called Virtual Institute for Satellite Integration Training (VISIT). Attendees of COMET are primarily NWS personnel, although representatives from other agencies (for example, DOD forecasters, university faculty and students) also participate. The VISIT program uses Internet technology to provide distance learning that allows interaction between students and instructors similar to a classroom situation. VISIT provides concurrent instruction to multiple sites. Between April of 1999 and 2001 there were 245 training sessions with a total of 4,585 students. In addition to the VISIT interactive classroom, NESDIS maintains, through its web pages, a Virtual Institute with satellite tutorials, and a wide assortment of case studies, and technical information and documents for users to peruse and use.
This concept of remote training is expanding well beyond the U.S. Through the auspices of the World Meteorological Organization, NESDIS scientists are bringing real-time geostationary data and interpretative assistance to Central and South America and the Caribbean Regional and Meteorological Training Centers. On the horizon are plans to expand collaboration and training programs to Europe, Africa, and Asia.
In addition to classroom efforts, NESDIS scientists participate in traditional workshops, professional conferences and publish in scientific literature.
C. Archiving Challenges from NPOESS and NASA's EOS Satellites
Page 50 PREV PAGE TOP OF DOC
The use of satellite data for climate studies has progressed from being experimental to mandatory. These data sets have proven to be of high value for climate studies. They have been used in regional and global temperature and upper tropospheric humidity trend studies, in studies of the ozone hole, and in studies of clouds and rainfall. These products have, in turn, been used in assessments by the Intergovernmental Panel on Climate Change (IPCC) and in various World Climate Research Programs (WCRP). Further applications of satellite data to climate studies, particularly for retrieval of column CO, are currently under development and appear promising. Several recent National Academy of Science Reports raise concerns about the Nation's ability to monitor climate variations and emphasize the need for a program of long-term, sustained observations of the Earth's changing climate. A major challenge for NOAA and NESDIS will be the development of the science capabilities needed to take on the mission of sustained long-term monitoring of Earth's climate from space. These include:
Ensuring the required accuracy, continuity, calibration, stability, and documentation that are essential for climate change detection and attribution.
Transitioning NASA developed technology, in the form of satellite climate instruments, instrument characterization and calibration science, and data processing systems, to NOAA. This will include measurements never made before by NOAA, but crucial for monitoring and understanding climate change.
Establishing partnerships with the climate user community to update observational requirements, provide feedback on products, and analyze data.
Page 51 PREV PAGE TOP OF DOC
Answering these challenges will take a concerted effort by instrument scientists, climate scientists and computer scientists. There must be extensive collaboration between the research and operational climate communities. Computer scientists, climatologists, and archivists will need to provide a sound and effective means to ensure that all necessary data are preserved and remain accessible in easy-to-use formats. It will also take a long-term commitment to provide resources to enable preservation of the climate archive from the first generation satellite systems of the 1980s and 1990s, through the transition satellite systems of the EOS era, to the second generation of operational systems in the future NPOESS, which will begin operation in 2009.
1. Building the Climate Record
The NPOESS system will have new weather and climate monitoring instruments as well as new instruments for monitoring ozone. A prototype of the NPOESS spacecraft, NPP, will be launched in 2006. A substantial research effort will be needed to insure continuity in the satellite climate record between the current operational system that has been in operation for the last 20 years and NPOESS. The proper time and data sets for this research effort will be provided by NPP, as a bridge to NPOESS. NPP will allow coincident climate observations between the old satellite instruments and the new ones that will begin operation with NPOESS and continue for many years. Construction of a seamless climate record between the current satellites and NPOESS is a very important and difficult challenge.
As a result of this challenge, NESDIS is implementing the concept of scientific stewardship within NESDIS. Scientific stewardship means providing the data and information services to answer the global change scientific questions of highest priority to the Nation, both now and in the future.
Page 52 PREV PAGE TOP OF DOC
The NESDIS scientific stewardship program has five goals. These goals are:
To provide real-time monitoring of climate-scale biases in the global suite of satellite observing systems. Since subtle spatial and temporal biases can create serious problems in future use of the data, we must develop the tracking tools necessary for detection of biases in the climate record. These biases can then be minimized or eliminated through efficient communication and coordination of information related to network performance using both in situ and satellite observations.
To document Earth system variability and change on global, regional, and local scales. This will be accomplished by building and maintaining a high quality base of data and information and establishing the best possible historical perspective critical to effective analysis and prediction. The creation of long-term, consistent records requires a long-term commitment of resources to accomplish these tasks.
To provide the necessary algorithms to ensure that understanding of key climate processes can be derived from space-based systems and the combination of space-based and in situ systems. The best possible scientific understanding of critical climate and global change issues can only be reached when all opinions and ideas can be explored. Thus, an active program engaging the research community, partnerships with industry, and increased interactions with local and regional governments is envisioned.
To optimize data and information services in order to make research easier and more effective by ensuring those services are simple, straight forward, direct, and responsive. This will be achieved by establishing end-to-end accountability for establishing long-term, scientifically valid, and consistent records for global change studies. This will ensure that our data and information are available to the maximum amount of users.
Page 53 PREV PAGE TOP OF DOC
To enable and facilitate future research. This aspect of stewardship involves providing the basic information technology, hardware, telecommunications, and software support to guarantee that data can be safeguarded and communicated both within NOAA and to outside users for generations in the future.
We believe that achieving these goals will result in a long-term archive that is flexible and innovative, that appropriately focuses responsibility on NESDIS for preservation of optimal data character, that provides for open access to the data by the scientific community and the public, and that will rapidly track technological developments.
In order to responsibly perform stewardship of incoming data sets, in FY 2003 NOAA has a new initiative for a Solar X-ray Imager Archive ($0.3M). The Solar X-ray Imager (SXI) represents a new observing capability that will be operational with the activation of GOES–12, currently on standby after successful demonstrations. NOAA will use the SXI archive to derive new products to help reduce the effects of extreme space weather events on telecommunications satellites, and on electrical power services, which cause estimated business losses of $1 billion per year.
2. Comprehensive Large Array-data Stewardship System (CLASS)
There is reason to expect that the information technology advances we have seen in the last ten years will continue in the future. With these advances, NOAA has made significant progress in its ability to archive and provide access, and will continue to leverage these advancing technologies through effective stewardship of current resources. Management of the increased volume of data can be accomplished
Page 54 PREV PAGE TOP OF DOC
80811i.eps
only through a rapid expansion in storage capacity, increased communications bandwidth, and automation of the means of data ingest, quality control, and access. The CLASS program will act as the connection in NOAA's effort to meet these challenges and pave the way to accommodate the additional massive data volumes expected over the next several years.
The ability to ensure on-going scientific stewardship for NOAA's environmental data and information will only be possible through extensive enhancement of NOAA's current data ingest, quality assurance, storage, retrieval, access, and migration capabilities. This goal will be met through the development and implementation of a standardized archive management system, which will be integrated with a robust, large-volume, rapid-access storage and retrieval system that is capable of storing the incoming large array environmental data, in situ data, and operational products as well as receiving a user's on-line data request, automatically processing the request, and providing the requested data on the most appropriate media. This system will provide standardization in media, interfaces, formats, and processes for the very large datasets produced by satellites and radars. Additionally, the system will facilitate ongoing migration, preservation, and validation to new technology and media. This system is modular in design, built to integrate with automated real-time or near-real-time systems that deliver data. Transaction processing will be implemented to enable an essentially ''hands-off'' operation and, where appropriate, allow users to pay for data or services through credit card or automated billing.
80811j.eps
Page 55 PREV PAGE TOP OF DOC
Placing data on-line for access via the Internet is a high priority in accordance with the Federal Government's eGov initiative. Data storage and retrieval systems will continue to be upgraded to support effective and efficient access with special focus on Internet interfaces, emerging telephony technologies, and on-line data that support the objectives of the CLASS concept of operations and ensure that the Nation has access (including Section 508 compliance) to their data and information.
The system will be able to handle the data flow from current satellite-based (e.g., GOES, DMSP, and POES) and ground-based (e.g., NEXRAD) observing systems, and be structured to handle the large increases in data that will come from planned satellite launches, including the METOP, NPOESS, NPP, and some EOS missions.
The target architecture goal will, through life cycle replacements and upgrades, bring the current NOAA National Data Centers under a single archive and access architecture that will be under formal configuration management control. This architecture will eliminate duplication of effort, minimize stand-alone systems, build the infrastructure to accommodate the large array data sets, and reduce the overall operational and system maintenance costs. The foundation system that is being used is the highly successful Satellite Active Archive (SAA). Recognized as a stable, modular, well-built system, the SAA approach provides maximum flexibility while minimizing development work and costs. The heart of the development centers on upgrading communications capabilities, increasing computer storage and power, exploiting commercially available modular hardware and software, and expanding Internet access to the data and information through new or enhanced database management, search, order, browse, and sub-setting techniques.
Page 56 PREV PAGE TOP OF DOC
The FY 2003 NOAA initiative for EOS Data Archive and Access System ($3.0 M) will ensure that NOAA possesses the ability to fully exploit the vast amounts of new NASA EOS satellite data becoming available, process and distribute that data, provide stewardship, and make the data accessible through CLASS to users in the private, research, government, and public sectors.
IV. Conclusion
In conclusion, Mr. Chairman and members of the Subcommittee, I have been pleased to be able to describe the challenges and significant opportunities facing NOAA in optimizing the use of satellite data. We are only scratching the surface on what satellites can do to support the Nation's requirements for environmental data and information. I have described the numerous initiatives underway, and those requested in FY 2003 by the Administration. These initiatives allow us to do the most that we can to continue to provide operational products and services to our current users, to create some of the most important new products requested, and to effectively plan for our future using an end-to-end approach. A key element to our strategy is partnering with other agencies, such as NASA and DOD, international partners such as Europe and Japan, and academia. These partnerships have proven to be wise investments for NOAA and the Nation. We have also greatly appreciated the support and interest expressed by this Subcommittee.
Mr. Chairman and Subcommittee Members, this concludes my testimony. I would be happy to answer any questions.
Appendix 1
Page 57 PREV PAGE TOP OF DOC
Examples of NOAA-wide Utilization of Satellite Data
National Weather Service (NWS)
Much of the full resolution satellite stream is used to produce the higher resolution, interactive products required by NWS Weather Forecast Offices (WFOs) to enhance public safety. The number of satellite products that NESDIS produces to support NWS, Department of Defense, and other users' requirements has grown since the 1980s from 40 to 500. For numerical weather prediction modeling applications, data derived from the POES, GOES, and Japan and Europe's geostationary satellites, are sent to the NWS National Centers for Environmental Prediction (NCEP) where these data are ingested into NWP models as radiance information.
Significant improvements in weather forecast accuracy came in the last half of the 1990s when, for the first time, satellite sounder radiance data were directly assimilated into U.S. operational weather forecast models. Over the period 1990–2010, there will be a five orders of magnitude increase of satellite data with potential for use in weather forecast models. Even after the data are thinned by an ''intelligent process'' to select the best observations, balance inputs, and minimize correlation errors, NOAA satellites provide 97 percent of the observational data assimilated into NWS NWP models.
Some of the satellite data not currently used by NCEP await scientific advances to allow us to properly model all physical processes that affect numerical predictions. As we improve our understanding of the complex geophysical processes through improved science and model techniques, the NCEP will more efficiently exploit the satellite data currently received and planned for future use.
Page 58 PREV PAGE TOP OF DOC
Office of Oceanic and Atmospheric Research (OAR)
NOAA's OAR is a co-investigator with NESDIS in a number of GOES model impact studies and field collaborations. Model impacts include studies with high spectral resolution infrared sounders and Global Positioning Satellite (GPS) meteorological profilers. Field collaborations include the Nauru Island ocean and atmospheric experiment. In addition, a scientist from the National Severe Storms Laboratory is stationed half time at the University of Oklahoma's Cooperative Institute for Mesoscale Meteorological Studies (CIMMS), where joint studies on using GOES Imager and Sounder data in field nowcasting and forecasting are being pursued.
National Marine Fisheries Service (NMFS)
NMFS is a key partner in our effort to apply remote sensing technologies to fisheries management. NMFS is host to four of the eight CoastWatch regional offices. NMFS also makes use of NOAA satellite-based data collection to track the migration and movement of fish such as great white sharks, bluefin tuna, and marlins. These satellite data are extremely valuable in modeling fish behavior and in determining stock limits. NMFS also uses satellite-derived data collection system to monitor fishing vessel movement and activity in specific fisheries. The fishery industry benefits from the satellite-assisted search and rescue program (COSPAS/SARSAT) which aids U.S. Coast Guard to rescue mariners in distress.
National Ocean Service (NOS)
NESDIS is collaborating with NOS in efforts to develop valid data processing algorithms for observations of ocean color (chlorophyll) in coastal waters. If successful, these algorithms will allow the quantitative use of satellite-observed chlorophyll in sediment-rich, coastal waters where current open-ocean algorithms fail. NESDIS also sponsors four full-time positions at the NOS's Coastal Service Center in Charleston, SC, in developing new coastal applications and information technologies in remote sensing.
Page 59 PREV PAGE TOP OF DOC
Office of Marine and Aviation Operations (OMAO)
OMAO benefits from a number of satellite-derived marine and aviation products and services developed using NOAA satellite data and data collection systems.
80811k.eps
80811l.eps
Chairman EHLERS. Thank you very much.
Ms. KOONTZ. Mr. Chairman.
Chairman EHLERS. Ms. Koontz.
STATEMENT OF MS. LINDA D. KOONTZ, DIRECTOR, INFORMATION MANAGEMENT ISSUES, GENERAL ACCOUNTING OFFICE
Ms. KOONTZ. Yes. Good morning. We appreciate the opportunity to be here today to talk about our work that we have done on the new Polar Orbiting Satellite System. Our written statement discusses the current satellite system, plans for the new system, and the key challenges that are faced by the four data processing centers in managing the vast amounts of data that are going to be generated by the new system. It is this last issue that I would like to particularly focus on now.
Page 60 PREV PAGE TOP OF DOC
As you know, the new satellite system is going to merge the two existing satellite systems at significant cost savings. This new system is expected to produce about ten times the data that is produced by the current system. This huge increase in data represents a number of significant challenges for the data processing centers. First, all four of the processing centers told us that they will have to make changes in their current computing infrastructures in order to support the new data streams. They need to increase the computational power of their supercomputers to process the data. They need to upgrade communication systems that transmit the data, and they need to increase data storage capacity.
All four centers have adopted a strategic approach and have done some high-level planning. And they have also begun to upgrade their infrastructures to put them on the path to meeting these new data streams.
However, it is really too soon to do the more detailed planning at this point, because until the contract is awarded and the design is finalized, it is difficult. They won't know all of the characteristics of the new system. And in addition, it is very difficult to predict what technology is going to be like six or seven years from now. But after the contract is awarded and the system design is more fully understood, the centers are going to need to start to do the detailed planning for addressing this issue. And all four centers have expressed their intention to do so.
There is also going to be a challenge in terms of incorporating this increased data into operational weather models and products, as we have heard earlier. Experts in the weather modeling community have told us that satellite data are not always effectively used in weather models, because the science needed to understand how to use the data are not yet mature. For example, it generally takes two to five years of study before a satellite launch to effectively incorporate data from new sensors into weather models. We asked each of the four centers their views on their ability to use the data, the 55 data records that will be generated by the new satellite system. We found that the centers did not always agree on what needed to be done in order to use these data records. Specifically, their responses differed for 30 of the data records in terms of whether a major advance in science was needed in order to assimilate the data.
Page 61 PREV PAGE TOP OF DOC
While there could be some valid reasons for these differences, for example, different uses of the data among the centers, the centers had not yet compared their views and they hadn't identified opportunities to leverage the expertise of the various centers. And further, they haven't really determined the actions that are needed to use the new data or establish specific plans. Agency officials, however, do recognize the need for coordination in planning so that data from the new satellite system can be effectively used and stated that they would likely accelerate their efforts after the contract award, which is scheduled for late next month.
In summary, there is time to meet the data challenges associated with the new satellite system. Each of the processing centers is planning activities to build its capacity to deal with the increased volume, but more can be done to coordinate their efforts and determine what needs to be done to use the data and address those that require major scientific advances. Beginning this process early will make sure that the Government does not experience any delays in its ability to use the new data.
And before I conclude, I should also mention, in terms of context, that NOAA faces a tremendous challenge in terms of archiving the data that comes from not only the satellites but from other sources as well and in making these data available to the research community and other users. This issue was addressed at some detail in a NOAA report to Congress in 2001. This area wasn't a specific focus of the work that we did, but we do recognize it is a very important issue, a very big challenge, and we know that NOAA has a number of efforts, ongoing and planned, to address this issue. That concludes my statement.
Page 62 PREV PAGE TOP OF DOC
[The prepared statement of Ms. Koontz follows:]
PREPARED STATEMENT OF LINDA D. KOONTZ
Polar-orbiting Environmental Satellites
Status, Plans, and Future Data Management Challenges
Mr. Chairman and Members of the Subcommittee:
We appreciate the opportunity to join in today's hearing to discuss our work on the planned National Polar-orbiting Operational Environmental Satellite System (NPOESS). At your request, we will discuss our nation's current polar-orbiting weather satellite program, plans for NPOESS, and key challenges in managing future NPOESS data volumes.
In brief, today's polar-orbiting environmental satellite program is a complex infrastructure encompassing two satellite systems, supporting ground stations, and four central data processing centers that provide general weather information and specialized environmental products to a variety of users, including weather forecasters, military strategists, and the public. NPOESS is planned to merge the two satellite systems into a single state-of-the-art environment monitoring satellite system, at a significant cost savings. This new satellite system is expected to provide vast streams of data, far more than are currently handled by the four processing centers and weather information users.
To handle this increased volume of satellite data, the four processing centers will need to build up their respective infrastructures, and they will need to work to efficiently incorporate new data into their weather products and models. Because the NPOESS launch is several years in the future, agencies have time to plan for expanding their infrastructures and models so that they can handle the increased data volumes that NPOESS will provide. Each of the processing centers is planning activities to build its capacity to handle increased volumes of data, but more can be done to coordinate and focus these plans.
Page 63 PREV PAGE TOP OF DOC
The approach we used to perform this work—our objectives, scope, and methodology—is provided in appendix I.
Polar Satellite Program Overview
Since the 1960s, the United States has operated two separate polar-orbiting meteorological satellite systems. These systems are known as the Polar-orbiting Operational Environmental Satellites (POES), managed by the National Oceanic and Atmospheric Administration's (NOAA) National Environmental Satellite, Data, and Information Service (NESDIS), and the Defense Meteorological Satellite Program (DMSP), managed by the Department of Defense (DOD). These satellites obtain environmental data that are processed to provide graphical weather images and specialized weather products, and that are the predominant input to numerical weather prediction models—all used by weather forecasters, the military, and the public. Polar satellites also provide data used to motitor environmental phenomena, such as ozone depletion and drought conditions, as well as data sets that are used by researchers for a variety of studies, such as climate monitoring.
Polar Satellite Infrastructure
Unlike geostationary satellites which maintain a fixed position above the earth, polar-orbiting satellites constantly circle the Earth in an almost north-south orbit, providing global coverage of conditions that affect the weather and climate. Each satellite makes about 14 orbits a day. As the earth rotates beneath it, each satellite views the entire earth's surface twice a day. Today, there are two operational POES satellites and two operational DMSP satellites that are positioned so that they can observe the earth in early morning, morning, and afternoon polar orbits. Together, they ensure that for any region of the earth, the data are generally no more than six hours old. Figure 1 illustrates the current operational polar satellite configuration. Besides the four operational satellites, there are five older satellites in orbit that still collect some data and are available to provide some limited backup to the operational satellites should they degrade or fail. In the future, both NOAA and DOD plan to continue to launch additional POES and DMSP satellites every few years, with final launches scheduled for 2008 and 2009, respectively.
Page 64 PREV PAGE TOP OF DOC
80811p.eps
Each of the polar satellites carries a suite of sensors designed to detect environmental data either reflected or emitted from the earth, the atmosphere, and space. The satellites store these data and then transmit the data to NOAA and Air Force ground stations when the satellites pass overhead. The ground stations then relay the data via communications satellites to the appropriate meteorological centers for processing.
Under a shared processing agreement among the four processing centers—NESDIS,(see footnote 2) the Air Force Weather Agency, Navy's Fleet Numerical Meteorology and Oceanography Center, and the Naval Oceanographic Office—different centers are responsible for producing and distributing different environmental data sets, specialized weather and oceanographic products, and weather prediction model outputs via a shared network. Each of the four processing centers is also responsible for distributing the data to its respective users. For the DOD centers, the users include regional meteorology and oceanography centers as well as meteorology and oceanography staff on military bases. NESDIS forwards the data to the National Weather Service for distribution and use by forecasters. The processing centers also use the Internet to distribute data to the general public. NESDIS is responsible for the long-term archiving of data and derived products from POES and DMSP.
In addition to the infrastructure supporting satellite data processing noted above, properly equipped field terminals that are within a direct line of sight of the satellites can receive real-time data directly from the polar-orbiting satellites. There are an estimated 150 such field terminals operated by the U.S. Government, many by DOD. Field terminals can be taken into areas with little or no data communications infrastructure—such as on a battlefield or ship—and enable the receipt of weather data directly from the polar-orbiting satellites. These terminals have their own software and processing capability to decode and display a subset of the satellite data to the user. Figure 2 depicts a generic data relay pattern from the polar-orbiting satellites to the data processing centers and field terminals.
Page 65 PREV PAGE TOP OF DOC
80811q.eps
Polar Satellite Data, Products, and Uses
Polar satellites gather a broad range of data that are transformed into a variety of products for many different uses. Satellite sensors observe different bands of radiation wavelengths, called channels, which are used for remotely determining information about the earth's atmosphere, land surface, oceans, and the space environment. When first received, satellite data are considered raw data.(see footnote 3) To make them usable, the processing centers format the data so that they are time-sequenced and include earth location and calibration information. After formatting, these data are called raw data records. The centers further process these raw data records into channel-specific data sets, called sensor data records and temperature data records. These data records are then used to derive weather products called environmental data records (EDR). EDRs range from atmospheric products detailing cloud coverage, temperature, humidity, and ozone distribution; to land surface products showing snow cover, vegetation, and land use; to ocean products depicting sea surface temperatures, sea ice, and wave height; to characterizations of the space environment. Combinations of these data records (raw, sensor, temperature, and environmental data records) are also used to derive more sophisticated products, including outputs from numerical weather models and assessments of climate trends. Figure 3 is a simplified depiction of the various stages of data processing.
80811r.eps
Page 66 PREV PAGE TOP OF DOC
EDRs can be either images or quantitative data products. Image EDRs provide graphical depictions of the weather and are used to observe meteorological and oceanographic phenomena to track operationally significant events (such as tropical storms, volcanic ash,(see footnote 4) and icebergs), and to provide quality assurance for weather prediction models.
The following figures present some polar-orbiting satellite images. Figure 4 is an image from a DMSP satellite showing an infrared picture taken over the west Atlantic Ocean. Figure 5 is a POES image of Hurricane Floyd, which struck the southern Atlantic coastline in 1999. Figure 6 is a polar-satellite image used to detect volcanic ash clouds, in particular the ash cloud resulting from the eruption of Mount Etna in 2001. Figure 7 shows the location of icebergs near Antarctica in February 2002.
80811s.eps
80811t.eps
80811u.eps
80811v.eps
Quantitative EDRs are specialized weather products that can be used to assess the environment and climate or to derive other products. These EDRs can also be depicted graphically. Figures 8 and 9 are graphic depictions of quantitative data on sea surface temperature and ozone measurements, respectively. An example of a product that was derived from EDRs is provided in figure 10. This product shows how long a person could survive in the ocean—information used in military as well as search and rescue operations—and was based on sea surface temperature EDRs from polar-orbiting satellites.
Page 67 PREV PAGE TOP OF DOC
80811w.eps
80811x.eps
80811y.eps
Another use of quantitative satellite data is in numerical weather prediction models. Based predominantly on observations from polar-orbiting satellites and supplemented by data from other sources such as geostationary satellites, radar, weather balloons, and surface observing systems, numerical weather prediction models are used to help forecast atmospheric, land, and ocean conditions hours, days, weeks, and months into the future. These models require quantitative satellite data to update their analysis of weather and to produce new forecasts. Table 1 contains examples of models run by the processing centers. Figure 11 depicts the output of one common model.
80811aa.eps
80811z.eps
All this information—satellite data, imagery, derived products, and model output—is used in mapping and monitoring changes in weather, climate, the ocean, and the environment. These data and products are provided to weather forecasters for use in issuing weather forecasts and warnings to the public and to support our nation's aviation, agriculture, and maritime communities. Also, weather data and products are used by climatologists and meteorologists to monitor the environment. Within the military, these data and products allow military planners and tactical users to focus on anticipating and exploiting atmospheric and space environmental conditions. For example, Air Force Weather Agency officials told us that accurate wind and temperature forecasts are critical to any decision to launch an aircraft that will need mid-flight refueling. In addition to these operational uses of satellite data, there is also a substantial need for polar satellite data for research. According to experts in climate research, the research community requires long-term, consistent sets of satellite data collected sequentially, usually at fixed intervals of time, in order to study many critical climate processes. Some examples of research topics include long-term trends in temperature, precipitation, and snow cover.
Page 68 PREV PAGE TOP OF DOC
The National Polar-orbiting Operational Environmental Satellite System
Given the expectation that converging the POES and DMSP programs would reduce duplication and result in sizable cost savings, a May 1994 presidential decision directive required NOAA and DOD to converge the two satellite programs into a single satellite program capable of satisfying both civilian and military requirements. The converged program is called the National Polar-orbiting Operational Environmental Satellite System (NPOESS), and it is considered critical to the United States' ability to maintain the continuity of data required for weather forecasting and global climate monitoring. To manage this program, DOD, NOAA, and the National Aeronautics and Space Administration (NASA) have formed a tri-agency integrated program office, located within NOAA.
Within the program office, each agency has the lead on certain activities. NOAA has overall responsibility for the converged system, as well as satellite operations; DOD has the lead on the acquisition; and NASA has primary responsibility for facilitating the development and incorporation of new technologies into the converged system. NOAA and DOD share the costs of funding NPOESS, while NASA funds specific technology projects and studies.
NPOESS Overview
NPOESS is a major system acquisition estimated to cost $6.5 billion over the 24-year period from the inception of the program in 1995 through 2018. The program is to provide satellite development, satellite launch and operation, and integrated data processing. These deliverables are grouped into four main categories: (1) the launch segment, which includes the launch vehicle and supporting equipment, (2) the space segment, which includes the satellites and sensors, (3) the interface data processing segment, which includes the data processing system to be located at the four processing centers, and (4) the command, control, and communications segment, which includes the equipment and services needed to support satellite operations.
Page 69 PREV PAGE TOP OF DOC
NPOESS will be a launch-on-demand system, and satellites must be available to back up the planned launches of the final POES and DMSP satellites. The first NPOESS satellite—designated C1—is scheduled for delivery in 2008 and is to be available to back up the planned launch of the final POES satellite in 2008. If C1 is not needed to back up the final POES, it will be launched in April 2009. The second NPOESS satellite is to be available to back up the planned launch of the final DMSP satellite in late 2009, or if not needed as a backup, it is to be launched in 2011. Subsequent launches are expected to occur approximately every two years through 2018.
Program acquisition plans call for the procurement and launch of six NPOESS satellites over the life of the program and the integration of 13 instruments, including 11 environmental sensors and 2 subsystems. Together, the sensors are to receive and transmit data on atmospheric, cloud cover, environmental, climate, oceanographic, and solar-geophysical observations. The subsystems are to support nonenvironmental search and rescue efforts, as well as environmental data collection activities. According to the integrated program office, 8 of NPOESS's 13 instruments involve new technology development, whereas 5 others are based on existing technologies. The planned instruments and the state of technology on each are listed in Table 2.
80811bb.eps
Unlike the current polar satellite program, in which the four centers use different approaches to process raw data into the environmental data records that they are responsible for, NPOESS' integrated data processing system—to be located at the four centers—is expected to provide a standard system to produce these data sets and products. The four processing centers will continue to use these data sets to produce other derived products, as well as for input to their numerical prediction models.
Page 70 PREV PAGE TOP OF DOC
NPOESS is planned to produce 55 environmental data records (EDRs), including atmospheric vertical temperature profile, sea surface temperature, cloud base height, ocean wave characteristics, and ozone profile. Some of these EDRs are comparable to existing products, whereas others are new. The user community designated six of these data products—supported by four sensors(see footnote 5)—as key EDRs, and noted that failure to provide them would cause the system to be reevaluated or the program to be terminated.
Acquisition Strategy
The NPOESS acquisition program consists of three key phases: the concept and technology development phase, which lasted from roughly 1995 to early 1997; the program definition and risk reduction phase, which began in early 1997 and is ongoing now; and the engineering and manufacturing development and production phase, which is expected to begin next month and continue through the life of the program. The concept and technology development phase began with the decision to converge the POES and DMSP satellites and included early planning for the NPOESS acquisition. This phase included the successful convergence of the command and control of existing DMSP and POES satellites at NOAA's satellite operations center.
The program definition and risk reduction phase involves both system-level and sensor-level initiatives. At the system level, the program office awarded contracts to two competing prime contractors—Lockheed Martin and TRW—to prepare for NPOESS system performance responsibility. These contractors are developing unique approaches to meeting requirements, designing system architectures, and developing initiatives to reduce sensor development and integration risks. These contractors will compete for the development and production contract. At the sensor level, the program office awarded contracts to develop five sensors.(see footnote 6) These sensors are in varying stages of development. This phase will end when the development and production contract is awarded. At that point, the winning contractor will assume responsibility for managing continued sensor development.
Page 71 PREV PAGE TOP OF DOC
The final phase, engineering and manufacturing development and production, is expected to begin next month when the development and production contract is awarded. The program office issued a request for proposals for the contract in February 2002 and is currently evaluating proposals, with an expectation of awarding the contract by the end of August 2002. The winning contractor will assume system performance responsibility for the overall program. This responsibility includes all aspects of design, development, integration, assembly, test and evaluation, operations, and on-orbit support.
Risk Reduction Activities
In May 1997, the integrated program office assessed the technical, schedule, and cost risks of key elements of the NPOESS program, including (1) the launch segment, (2) the space segment, (3) the interface data processing segment, (4) the command, control, and communications segment, and (5) the overall system integration. As a result of this assessment, the program office determined that three elements had high risk components: the interface data processing segment, the space segment, and the overall system integration segment. Specifically, the interface data processing segment and overall system integration were assessed as high risk in all three areas (technical, cost, and schedule), whereas the space segment was assessed to be high risk in the technical and cost areas, and moderate risk in the schedule area. The launch segment and the command, control, and communications segment were determined to present low or moderate risks. The program office expects to reduce its high risk components to low and moderate risks by the time the development and production contract is awarded, and to have all risk levels reduced to low before the first launch. Table 3 displays the results of the 1997 risk assessment as well as the program office's projections for those risks by August 2002 and by first launch.
Page 72 PREV PAGE TOP OF DOC
80811cc.eps
In order to meet its goals of reducing program risks, the program office developed and implemented an integrated risk reduction program that includes nine initiatives. While individual initiatives may address one or more identified risks, the program office anticipated that the combination of these nine projects would address the risk to overall system integration. The nine projects are as follows:
Deferred development: To reduce program risk, the program office deferred development of 21 EDR requirements either because the technology needed to implement the requirements did not exist or because the requirement was too costly. For example, the requirement for measuring ocean salinity was deferred until the technology needed to take these measurements has been demonstrated in space. If feasible, the program office plans to implement these requirements later as program enhancements.
Early sensor development: Because environmental sensors have historically taken eight years to develop, development of six of the eight sensors with more advanced technologies was initiated early. In the late 1990s, the program office awarded contracts for the development, analysis, simulation, and prototype fabrication of five of these sensors.(see footnote 7) In addition, NASA awarded a contract for the early development of one other sensor.(see footnote 8) Responsibility for delivering these sensors will be transferred from the program office and NASA to the winning development and production contractor. According to program office officials, these sensors should be delivered at least two years before the earliest expected NPOESS launch because of these early development efforts.
Page 73 PREV PAGE TOP OF DOC
Building on existing sensor technologies: In order to minimize risks, the program office used existing sensor technologies as a starting point from which to build new sensors and also plans to use some existing sensors on NPOESS. For example, the new cross-track infrared sounder sensor grew from technology used on the POES high-resolution infrared sounder and on the atmospheric infrared sounder carried on NASA's Earth Observing System/Aqua satellite. Also, NPOESS' data collection system is based on the data collection system already flying on another satellite and, according to program officials, will likely be available largely ''off the shelf.'' Program office officials reported that building on existing sensors should enable them to obtain half of the NPOESS sensors and almost half of the required 55 EDRs while reducing the risk of integrating new technology into the program.
Ground demonstrations: To reduce the risk to the data processing segment, the program office had both of the program definition and risk reduction contractors conduct four ground-based demonstrations of hardware and software components of the data processing system. Because of work done during the program definition and risk reduction contract phase, the program office expects the interface data processing segment to be relatively mature before contract award.
Internal government studies: To reduce the risks in integrating the NPOESS space and interface data processing segments, over the past five years, the integrated program office has overseen risk reduction studies performed by over 30 major scientific organizations, including government laboratories, major universities, and institutes. These studies include observing system simulation experiments and data assimilation studies, which involve simulating a future sensor and then identifying ways to incorporate the new data into products and models. For example, the studies were used to assess the impact of advanced sounders similar to those on NPOESS and the impact of NPOESS-like data on forecasts and end user products.
Page 74 PREV PAGE TOP OF DOC
Aircraft flights: Since 1997, the integrated program office has used aircraft flights to demonstrate satellite sensors and to deliver early data to its users so that they can begin to work with the data. For example, in 2001, the NPOESS airborne sounder testbed project began using NASA aircraft to provide an environment in which instruments could be tested under conditions that simulate space.
Operational algorithm teams: The integrated program office established five operational algorithm teams to serve as scientific advisory groups. The teams, made up of representatives from government and federally funded research and development centers, worked with the program office for five years to oversee the development and refinement of various algorithms that NPOESS will use. They will continue to work with the development and production contractor to refine the data processing algorithms.
WindSat/Coriolis demonstration: WindSat/Coriolis is a demonstration satellite, planned for launch in 2003, to test critical new ocean surface wind-observing science and technology that will be used in the NPOESS conical microwave imager/sounder sensor. This demonstration project will also help validate the technology needed to support various EDRs.
NPOESS preparatory project: This is a planned demonstration satellite to be launched in early 2006, two to three years before the first NPOESS satellite launch. It is scheduled to host three critical NPOESS sensors (the visible/infrared imager radiometer suite, the cross-track infrared sounder, and the advanced technology microwave sounder), and it will provide the program office and processing centers an early opportunity to work with the sensors, ground control, and data processing systems. This satellite is expected to demonstrate about half of the NPOESS EDRs and about 80 percent of its data processing load.
Page 75 PREV PAGE TOP OF DOC
Data Management Challenges and Plans to Address Them
NPOESS is expected to produce a massive increase in the volume of data sent to the four processing centers, which presents considerable data management challenges. Whereas current polar satellites produce approximately 10 gigabytes of data per day, NPOESS is expected to provide 10 times that amount. When combined with increased data from other sources—other satellites, radar, and ground sensors—this increase in satellite data presents immense challenges to the centers' infrastructures for processing the data and to their scientific capability to use these additional data effectively in weather products and models.
The four processing centers and the integrated program office are well aware of these data management challenges and are planning to address them. Specifically, each of the four centers is planning to build its capacity to handle increased data volumes, and both the centers and the program office are working to improve their ability to assimilate new satellite data in their products. Because the NPOESS launch is several years in the future, agencies have time to build up their respective infrastructures and models so that they can handle increased data volumes. However, more can be done to coordinate and further define these efforts.
Infrastructure Challenges and Plans to Address Them
The expected increase in satellite data from NPOESS presents a considerable challenge to the processing centers' infrastructures for obtaining, processing, distributing, and storing satellite data. All four of the central processing centers reported that their current infrastructures would require changes in order to support expected NPOESS data streams. In fact, two centers reported that their current infrastructures could not support any of the NPOESS EDRs that they expect to use; another center reported that its infrastructure could not support 82 percent of the EDRs it expects to use; and the fourth center reported that its infrastructure could not support 27 percent of the EDRs that it will use.
Page 76 PREV PAGE TOP OF DOC
As for specific shortcomings, officials at the processing centers reported that they need to increase the computational power of the supercomputers that will process the data records, upgrade the communication systems used to transmit the data, and/or increase the storage capacity of the systems used to archive the data. For example, National Weather Service officials told us that current supercomputers could not process the vast amount of satellite data NPOESS will generate within required timeframes to produce forecasts, because even today they are encountering computer capacity constraints. Specifically, the target usage rate for effectively processing modeling data is 50 percent of computing capacity. Officials told us that the average current usage rate is 70 percent of capacity, and usage often peaks well above this rate. As another example of an infrastructure challenge, officials at the Navy's Fleet Numerical Meteorology and Oceanography Center reported that even with recent upgrades to their local data storage capacity, their current infrastructure could not likely support NPOESS increased data volumes.
To handle these increased data volumes, the four processing centers have begun high-level planning to transform their respective satellite data processing infrastructures. Understandably, the centers have not yet begun detailed planning for operational and technology change in the 2008–2009 timeframe because there are too many unknowns for them to do so reliably. For example, the architectural characteristics of the NPOESS system will not be known until sometime after the development and production contract is awarded later this year. Also, as stated by center officials, technology changes so quickly that it is difficult to predict technology options six to seven years from now.
Although the centers are not yet building their infrastructures specifically to support NPOESS, officials told us that they are currently working to upgrade their infrastructures to support current and future data streams. For example, NOAA plans to increase the processing capacity of its supercomputers to handle the increased volume of satellite data expected over the next several years. In addition, the Air Force Weather Agency is in the process of upgrading its information technology infrastructure to increase the capacity of its computer and communications systems.
Page 77 PREV PAGE TOP OF DOC
The processing centers recognize the infrastructure challenges they face, and each is planning or initiating upgrades to improve its data management capacity to meet immediate challenges. Once the NPOESS development and production contract is awarded and the system design is determined, it is imperative that the four processing centers adjust and further define their future architectures to address this design, and identify the steps they need to take to reach that future goal. All of the centers have expressed their intentions to do so.
Data Utilization Challenges and Plans to Address Them
The increased data volumes from NPOESS pose a challenge to those seeking to use these data in operational weather models and products. These models and products are heavily. dependent on satellite data, but experts in the weather modeling community acknowledge that satellite data are not always used effectively because the science needed to understand and use the data is sometimes immature. For example, forecasters do not yet know how to use microwave data from areas covered in ice or under heavy precipitation in their weather prediction models. Experts reported that it often takes years of study and scientific advances to effectively assimilate new satellite data into weather models and to derive new weather products. While there is some debate as to how long it takes to develop the science to put new data in models, in 2000, the National Research Council reported that it generally takes two to five years of simulations and analyses before a satellite launch for data from new sensors to be effectively incorporated into weather models.(see footnote 9) They noted that if this work does not occur, there is a gap of several years during which data are collected but not used efficiently in models. Defense and civilian modeling officials reiterated the value of advance assimilation studies by citing an example in which such studies performed before a new sensor was launched allowed modelers to use the data only 10 months after launch.
Page 78 PREV PAGE TOP OF DOC
The processing centers acknowledge that much needs to be done for them to be able to incorporate NPOESS data into operational products. Officials at the processing centers reported that they should be able to use some EDRs after only minor changes to their data processing algorithms and models, because these products are expected to be comparable to current products. Other EDRs, however, involve new data and will require major scientific advances in order to be used. That is, the centers will not be able to use these data until they conduct new scientific investigations and determine how to best use the data in their derived products and models. In fact, the three centers that are the heaviest planned users of NPOESS EDRs reported that about 45 percent of the EDRs they plan to use would require major advances in science in order to be utilized. For example, NESDIS stated that it would take major science changes to be able to utilize all six of the key EDRs, including atmospheric vertical temperature profile, soil moisture, and sea surface winds. Table 4 lists the number of EDRs each of the processing centers plans to use and each center's views of how many of those EDRs require major science changes. Appendix II identifies the EDRs that the centers reported as requiring major scientific advancements.
80811dd.eps
Effective and efficient use of satellite data in weather products, warnings, and forecasts is critical to maximizing our national investment in new satellites. A committee representing the four processing centers noted that expedited incorporation of new satellite data into weather models is a key metric for measuring NPOESS' success. Given that understanding, the processing centers and the integrated program office have various efforts under way and planned to address challenges in effectively using new NPOESS data. Key initiatives include the following:
Page 79 PREV PAGE TOP OF DOC
Joint Center for Satellite Data Assimilation: In July 2001, NOAA and NASA formed a joint organization to accelerate the rate at which satellite data are put into operational use. While the center is currently focused on assimilating data from existing satellites, joint center scientists plan to undertake projects to accelerate the assimilation of future satellite data, including NPOESS data, into weather prediction models. The joint center received $750,000 in its fiscal year 2002 budget and requested $3.4 million for fiscal year 2003. In a November 2001 letter to the processing centers, the integrated program office offered to help fund the joint center efforts to assimilate NPOESS data if the DOD processing centers were to join the joint center. The processing centers have discussed this option, but DOD has not yet made a final decision.
Processing centers' assimilation projects: Two of the three military processing centers, the Air Force Weather Agency and the Navy Fleet Numerical Meteorology and Oceanography Center, have developed programs to improve assimilation of high-resolution satellite data into their models. They have also developed a program that is designed to improve their models so that they will be able to use data from the NPOESS preparatory project, when they become available.
Other government-sponsored studies: As noted in its risk reduction efforts, the integrated program office has funded studies—both simulations and data assimilation studies—to prepare for the NPOESS data. Since fiscal year 1995, the program office has reportedly spent more than $3 million on satellite data assimilation experiments and projects to develop techniques for processing satellite data. For example, the program office funded NOAA to develop methods to begin processing and assimilating sounding data from the advanced infrared sounder on a NASA satellite. This effort was expected to pave the way for processing and assimilating data from two sensors that will fly on the NPOESS preparatory project in early 2006 and on NPOESS in the 2008 to 2009 timeframe.
Page 80 PREV PAGE TOP OF DOC
Centers Have Time to Meet Challenges, But More Coordination and Definition of Plans Are Warranted
Between now and the first NPOESS satellite launch, the four processing centers and the integrated program office have time to meet the challenges in effectively using NPOESS data, but more can be done to coordinate and define these efforts: The four centers' views on their ability to use NPOESS EDRs in their models and products highlighted that the centers are not always consistent on whether an NPOESS data product requires major scientific advancements or not. Specifically, the centers' views differ on over 30 EDRs. For example, in the case of one key EDR—atmospheric vertical temperature profile—one center states that it will require only minor software changes to use these data; another center states that it will require a major advancement in science to use the data; and a third states that it will not require a science change, but instead will require an upgrade to its supporting infrastructure. Appendix II lists the processing centers' views of which EDRs require major scientific advancements in order to be used.
While there may be valid reasons for some of these differences—such as the centers' differing uses for these EDRs or their varying customers' needs—the centers have not yet compared their differing views or identified opportunities for learning from other centers' expertise. Agency officials generally agreed that such coordination would be valuable and stated their intentions to coordinate.
In addition to coordinating on EDRs determined to pose scientific challenges, it will be important for the centers to identify what needs to be done to meet these major science challenges and to define their plans for doing so. However, the centers have not yet determined what actions are needed to effectively incorporate NPOESS EDRs in their respective models and derived products. Further, they have not yet established plans for addressing the specific EDRs that require major scientific advancements. Agency officials agreed that such planning is necessary and stated that they will likely accelerate these efforts after the development and production contract is awarded.
Page 81 PREV PAGE TOP OF DOC
Clearly, there are opportunities for the processing centers to coordinate their particular concerns, learn from other centers' approaches, and define their plans for addressing challenges in using EDRs. Given the years it takes to effectively incorporate new satellite data into operational products, it is critical that such coordination and detailed planning occur so that NPOESS data can be effectively used.
- - - - -
In summary, today's polar-orbiting weather satellite program is essential to a variety of civilian and military operations, ranging from weather warnings and forecasts to specialized weather products. NPOESS is expected to merge today's two separate satellite systems into a single stateof-the-art weather and environmental monitoring satellite system to support all users. This new satellite system is expected to provide vast streams of data, far more than are currently handled by the four central processing centers. To prepare for these increased data volumes, the four data processing centers must address key data management challenges—including building up their respective infrastructures and working to be able to efficiently incorporate new data in their derived weather products and models. Because the NPOESS launch date is still several years in the future, the four processing centers and the integrated program office have time to continue to develop, define, and implement their plans to address key data management challenges.
Each of the processing centers is planning activities to build its capacity to handle increased volumes of data, but more can be done to coordinate and define these plans—including sharing information on what is needed in order for the centers to use particular weather products and developing a plan to address these scientific issues. Unless more is done to coordinate and define these plans, the centers could risk delays in using NPOESS data in operational weather products and forecasts.
Page 82 PREV PAGE TOP OF DOC
- - - - -
This concludes my statement. I would be pleased to respond to any questions that you or other members of the Subcommittee may have at this time.
Contact and Acknowledgements
If you have any questions regarding this testimony, please contact Linda Koontz at (202) 512–6240 or by e-mail at koontzl@gao.gov. Individuals making key contributions to this testimony include Ronald Famous, Richard Hung, Tammi Nguyen, Colleen Phillips, Angela Watson, and Glenda Wright.
Appendix I
Objectives, Scope, and Methodology
The objectives of our review were to (1) provide an overview of our nation's polar-orbiting weather satellite program, (2) identify plans for the 'NPOESS program, and (3) identify key challenges in managing future NPOESS data volumes and the four processing centers' plans to address them.
To provide an overview of the Nation's polar-orbiting weather satellite system, we reviewed NOAA and DOD documents and Web sites that describe the purpose and origin of the polar satellite program and the current POES and DMSP satellites' supporting infrastructures. We assessed the polar satellite infrastructure to understand the relationships among the satellites, ground control stations, and satellite data processing centers. We also reviewed documents and interviewed officials at the integrated program office and four processing centers to identify the processes for transforming raw satellite data into derived weather products and weather prediction models.
Page 83 PREV PAGE TOP OF DOC
To identify plans for the NPOESS program, we obtained and reviewed documents that describe the program's origin and purpose, and interviewed integrated program office officials to determine the program's background, status, and plans. We assessed the NPOESS acquisition strategy and program risk reduction efforts to understand how the program office plans to manage the acquisition and mitigate the risks to successful NPOESS implementation. We reviewed descriptions of each of the NPOESS sensors and assessed NPOESS program requirement documents to determine the types of products that NPOESS will produce and how these products will be used.
To assess NPOESS data management challenges, we reviewed documents from the program office and the four processing centers and discussed challenges with DOD and NOAA officials. We assessed descriptions of each center's current and planned polar satellite infrastructure to identify plans for infrastructure growth. We also identified each processing centers' views on which NPOESS products will require infrastructure changes or scientific advancements in order to be used. We analyzed this information to determine whether the centers face challenges in their ability to process NPOESS data and their scientific capability to assimilate NPOESS data into their weather prediction models. We reviewed'documents that describe NOAA, DOD, and integrated program office efforts to address the challenges that we identified, and we evaluated current and planned efforts to address those challenges. We interviewed program office and processing center officials to discuss these documents and their plans to address NPOESS data management challenges.
We obtained comments from NOAA and DOD officials on the facts contained in this statement. These officials generally agreed with the facts as presented and provided some technical corrections, which we have incorporated. We performed our work at the NPOESS Integrated Program Office, located at NOAA headquarters in Silver Spring, Maryland; the NESDIS Central Satellite Data Processing Center in Suitland, Maryland; the NCEP Environmental Modeling Center in Camp Springs, Maryland; the Air Force Weather Agency at Offutt Air Force Base in Omaha, Nebraska; the Fleet Numerical Meteorology and Oceanography Center in Monterey, California; and the Naval Oceanographic Office at Stennis Space Center in Bay St. Louis, Mississippi. Our work was performed between October 2001 and July 2002 in accordance with generally accepted government auditing standards.
Page 84 PREV PAGE TOP OF DOC
80811ee.eps
80811ff.eps
BIOGRAPHY FOR LINDA D. KOONTZ
Linda Koontz is Director, Information Management Issues, at the U.S. General Accounting Office. Ms. Koontz is responsible for issues concerning the collection, use, and dissemination of government information in an era of rapidly changing technology.
Recently, Ms. Koontz has been heavily involved in directing studies concerning privacy, electronic records management, and government-wide information dissemination issues. In addition, she has lead responsibility for information technology management issues at various agencies including the Department of Commerce.
Chairman EHLERS. Thank you very much. Twenty-one seconds to spare. Dr. Abbott.
STATEMENT OF DR. MARK ABBOTT, DEAN, COLLEGE OF OCEANIC AND ATMOSPHERIC SCIENCES, OREGON STATE UNIVERSITY; CHAIR, COMMITTEE ON EARTH STUDIES, NATIONAL RESEARCH COUNCIL
Dr. ABBOTT. Chairman Ehlers, and members of the Environment, Technology, and Standards Subcommittee, I would like to thank you for the opportunity to speak today about satellite data management at NOAA from the perspective of the Earth science community.
Page 85 PREV PAGE TOP OF DOC
Satellite observations provide a unique vantagepoint from which to study the Earth's system. From measurements of ocean circulation to observations of ocean and land productivity to measurements of upper atmosphere temperatures, satellites connect the sentinels of the global system. As satellite missions and their associated data records become longer, the research community has begun to use satellite data to study processes associated with climate change. NPOESS, with its more capable satellite instruments and stable Earth orbits, represents a significant opportunity to increase the usage of satellite data in climate research. However, there are serious issues associated with NPOESS data management that must be first addressed.
Today, I will focus primarily on the needs of the climate research community in regards to NPOESS data management. First, it must be acknowledged that the delivery of climate data and services has an enormous research component. The science community is continuing to investigate the complexities of the climate system, expanding the types of data that are needed, refining the requirements for the measurements, and other aspects of data quality. Data products that are useful for operational needs, generally do not meet the needs of the Earth science community, especially for the study of climate-related processes.
Long-term archiving, careful calibration, and regular reprocessing with state-of-the-art software are critical for climate research. Operational needs, however, are generally characterized by rapid, predictable, and extremely reliable delivery schedule of standardized data products. Thus if NPOESS is to serve the data needs of climate research, it must provide the necessary services and functions. A data management system that is designed solely to meet short-term operational needs will not meet the needs of climate research.
Page 86 PREV PAGE TOP OF DOC
The Earth science community will increasingly rely upon NOAA to acquire and deliver fundamental data sets about the Earth system. Although the traditional operational data products may appear to be the same as those used in climate research, rainfall, sea surface temperature, etcetera, this is not the case. Climate data records require continuous scientific research and reprocessing to develop new data products and to ensure their suitability for climate studies. NOAA must provide new services and functions that go beyond its previous capabilities. And it must develop these new capabilities against a backdrop of significant increases in data volume from its satellite observing systems.
Scientific stewardship of these data records is an enormous responsibility. Preserving knowledge on the Earth's system requires far more than simply moving data into a magnetic tape archive. The series sensor on NASA's EOS satellites required nearly 200 person-years of effort just on data production alone and generated nearly one million lines of computer code. Its intellectual heritage stretches over decades of continuous analysis and improvement. Calibration of the sensor was not merely appended to the data record, it was embedded within the entire data processing stream.
Many times, we focus on the technical issues and challenges, considering the problems and their solutions simply from the types of tools that we need. How many disk drives and tape silos do we need to accommodate NPOESS data? How many Internet connections do we need? This approach may allow the tools to drive our system architecture. It is though we go to the lumberyard and buy so much plywood and so many two-by-fours, and then decide on the type of house to build. Instead, we need to clearly define the services and tasks that the NPOESS data management system needs to provide to support the climate research community.
Page 87 PREV PAGE TOP OF DOC
Although the integrated program office for NPOESS has made significant progress to involve the science community and the definition of the NPOESS sensors, we have not yet made similar progress in the area of data management. NOAA management recognizes some of the challenges, but obstacles remain. Interactions between the climate research community and NOAA up to this point have indicated that NOAA does not recognize the scope of the problem aside from the technical issues of storage, network bandwidth, and computational requirements. Scientific stewardship requires a far deeper understanding and more comprehensive approach than the definition of the technical implementation.
This effort must begin now even before the launch of NPOESS. Small prototype activities with existing satellite data sets are one component. Active engagement of the scientific community and the design testing and evaluation of data management and functions and services is essential. NPOESS will advance our understanding of the Earth's system, but an effective and efficient data management system is necessary if we are to realize its full potential.
Thank you for the opportunity to testify this morning. And I would welcome any questions that you may have.
[The prepared statement of Dr. Abbott follows:]
PREPARED STATEMENT OF MARK R. ABBOTT
Chairman Ehlers, Ranking Member Barcia, and Members of the Environment, Technology, and Standards Subcommittee of U.S. House of Representatives Committee on Science, I would like to thank you for the opportunity to speak today about satellite data management at NOAA from the perspective of the Earth science community. My name is Mark Abbott and I am the Dean of the College of Oceanic and Atmospheric Sciences at Oregon State University.
Page 88 PREV PAGE TOP OF DOC
Satellite observations provide a unique vantage point from which to study the Earth system. From measurements of ocean circulation, to observations of ocean and land productivity, to measurements of upper atmosphere temperatures, satellites can act as sentinels of the global system. Moreover, they can observe rapid changes such as severe storms, floods, and even harmful phytoplankton blooms in the coastal ocean. The Earth science community has come to rely on a suite of research satellites, such as NASA's Earth Observing System (EOS) missions, and operational missions, such as NOAA's Polar-orbiting Operational Environmental Satellites (POES) and DOD's Defense Meteorological Satellite Program (DMSP). As satellite missions and their associated data records become longer (for example, the DMSP series now spans nearly 20 years), the research community has begun to use satellite remote sensing to study processes associated with climate change. NPOESS, with its more capable satellite instruments and stable Earth orbits, represents a significant opportunity to increase the usage of satellite data in climate research. However, there are issues associated with NPOESS data management that must be addressed.
Data management must be understood in the context of end-to-end management, including data acquisition, processing, distribution, and archiving. To build a data management system, one must understand the flows of data through the system, as well as the services that one is trying to support. For example, a financial information system for a Wall Street brokerage firm must accomplish the basic tasks of acquisition, processing, distribution, and archiving as does the NPOESS data management system, but the actual systems will be quite different because of the differences in the types and volumes of data as well as the services that are being supported.
The existing POES data management system is designed to accommodate approximately two terabytes (trillion bytes) of data per year. (For context, the entire Library of Congress holds approximately 25 terabytes of text information). NPOESS will produce over 200 terabytes per year. Moreover, NPOESS will produce far more types of data products than POES, including many data types that have not been acquired before by NOAA. For example, NPOESS will measure the color of the ocean to infer the concentration of phytoplankton (the microscopic plants in the ocean), which has only been measured by NASA research missions. Thus both the data volumes and data types will increase, posing serious technical and scientific challenges to NOAA.
Page 89 PREV PAGE TOP OF DOC
The NPOESS data management system must support an array of new services as well as larger data volumes and more types of data. This is especially true in the area of climate research which will require access to long time series of data (years to decades) that have been processed with state-of-the-art software. To detect the small changes associated with climate change, these data records must reduce or eliminate artifacts, such as changes in sensor performance or changes in processing software. Moreover, climate research will require an enormous array of different types of data, ranging from physical properties of the ocean and atmosphere to information on the structure of marine and land ecosystems. Such information is critical to the understanding of the processes and impacts of climate change.
In addition to supporting climate research, the NPOESS data management system must support new weather and other short-term forecasting services within its traditional customer base. These may include customized, neighborhood-scale weather forecasts, road conditions delivered to in-dash devices in automobiles, and atmospheric visibility maps delivered to a soldier's handheld device for weapons targeting. Integration of satellite data into new types of predictive models, such as ocean circulation, is being developed by the research community, and these assimilation models will eventually produce operational forecasts. Although NPOESS will probably not develop and deliver these services and products directly (commercial providers, the military, etc. may be responsible for the final products), NPOESS must ensure that its data management system can support these value-added providers in terms of data delivery and model development.
Today, I will focus primarily on the needs of the climate research community in regards to NPOESS data management. First, it should be acknowledged that the delivery of climate data and services has an enormous research component. The science community is continuing to investigate the complexities of the climate system, expanding the types of data that are needed, refining the requirements for the measurements in terms of spatial and temporal resolution and other aspects of data quality. Climate research is not as mature as weather forecasting, where data product requirements are well defined. Data products that are useful for operational needs, such as short-term forecasting, generally do not meet the needs of the Earth science community, especially for the study of climate-related processes. Long-term archiving, careful calibration, and regular reprocessing with state-of-the-art algorithms are critical for climate research applications. Operational needs are characterized by rapid, predictable and extremely reliable delivery schedules of standardized data products. Thus if NPOESS is to serve the data needs of climate research, it must provide the necessary climate research services and functions. A data management system that is designed solely to meet short-term operational needs will not meet the needs of climate research.
Page 90 PREV PAGE TOP OF DOC
The Committee on Earth Studies of the National Research Council developed several guiding principles for the NPOESS data management system in the context of climate research and climate services.
The data system must be accessible and support the production of policy-relevant environmental information
The Federal Government must ensure long-term stewardship of the data records, develop a system based on open and flexible standards, and ensure open access to the archives
The system should focus on simplicity and sustainability, rather than try to accomplish every possible task and meet every conceivable requirement
The system should be adaptable and flexible so that it can respond to changing user needs and changing information systems technology
Because data systems for climate research are likely to be complex and evolve rapidly during their initial deployment, small, end-to-end prototypes should begin now
Climate Data Records will require substantial reprocessing as scientific understanding and knowledge of satellite sensor performance improve over time
Climate Data Records will require access to data from multiple sensors and satellites as well as rigorous evaluation as NPOESS moves from one satellite to another
Page 91 PREV PAGE TOP OF DOC
The data system must be developed with the active collaboration of the science community
These principles may appear to be overwhelming, but given the upcoming launch of the NPOESS Preparatory Project (NPP) in 2006, it is time to begin. NPOESS data will be an essential component of the observations, analysis, and models that the scientific community will use to provide sound information and knowledge to policy-makers. Therefore it is essential that the NPOESS data management system begin now to meet the needs of climate research. An enormous investment in observing systems may not return the necessary information and knowledge as we work to develop climate policies.
The Committee on Earth Studies made several recommendations to both NOAA and NASA. These recommendations include:
NOAA should start now to begin building capacity to preserve the basic satellite observations as well as the ancillary data (such information regarding sensor calibration, performance, operations, etc.). Given the cost of collecting the data, it is only prudent to ensure that the data are preserved.
NOAA should ensure that, at an absolute minimum, the fundamental, unprocessed satellite and ancillary data can be accessed, even if only minimal services are provided. For example, it should be possible and affordable to retrieve long time series (for example, 10–30 years) for reprocessing.
NOAA and NASA should work together on Climate Data Records using existing research and operational satellite data. These data products go beyond the usual weather and short-term forecasting products as they rely on thorough and careful analysis to detect small changes in the Earth system that may be obscured by ''noise'' in the satellite sensor, processing algorithms, etc.
Page 92 PREV PAGE TOP OF DOC
NOAA and NASA should begin now to define and develop a basic set of user services and tools to meet the needs of the science community. These basic services should focus on ensuring that order fulfillment is reliable, rapid, and affordable.
Where are we today? The Earth science community will increasingly rely upon NOAA to acquire and deliver fundamental data sets about the Earth system. Although the traditional operational data products may appear to be the same as those used in climate research (rainfall, sea surface temperature, etc.) this is not the case. Climate data records require continuous scientific research and reprocessing to develop new data products and to ensure their suitability for climate research. Moreover, NPOESS will acquire many new observations that previously have been the responsibility of research agencies such as NASA. Thus NOAA must provide new services and functions that go beyond its previous capabilities. NOAA must develop these new capabilities against a backdrop of significant increases in data volume from its satellite observing systems.
Scientific stewardship is an enormous responsibility. Preserving knowledge on the Earth system requires far more than simply moving data into a magnetic tape archive. The CERES sensor on NASA's EOS series required nearly 200 person-years of effort just on data production alone, and nearly one million lines of computer code. The intellectual heritage stretches over many decades of continuous analysis and improvement. Calibration of the climate data records is not merely appended to the data record; it is embedded within the entire data processing stream. Software development is most efficient if done in small, distributed teams. Testing should follow the commercial software development model of ''build and burn,'' where new capabilities are added and tested on a daily basis. At this point, NOAA appears to be following a centralized development effort where integrated tests of the software (where most problems emerge) are not done until the software system is completed. This approach emphasizes overly-detailed documentation and repeated reviews, and it is likely that the final system will produce over one million pages of documentation. Many technical decisions on issues such as data formatting, version control, and even file naming are following a bureaucratic approach at NOAA, rather than one based on services.
Page 93 PREV PAGE TOP OF DOC
Many times, we focus on the technical issues and challenges, considering the problems and their solutions simply from the types of tools that we need. How many disk drives and tape silos do we need to accommodate NPOESS data? How many Internet connections do we need? This approach allows the tools to drive our system architecture. It is as though we go to the lumberyard and buy so much plywood and so many 2x4's and then decide on the type of house to build. Instead, we need to clearly define the services and tasks the NPOESS data management systems needs to provide to support the climate research community. Although the Integrated Program Office for NPOESS has made significant progress to involve the science community in the definition of the NPOESS sensors, we have not yet made similar progress in the area of data management. NOAA management recognizes some of the challenges, but obstacles remain. Interactions between the climate research community and NOAA up to this point have indicated that NOAA does not recognize the scope of the problem, aside from the technical issues of storage, network bandwidth and computational requirements. Scientific stewardship requires a far deeper understanding and a more comprehensive approach than a definition of the technical implementation. We need to begin an aggressive program of definition, testing, and evaluation for data management to support climate research.
This effort must begin now, even before the launch of NPP. Small prototype activities with existing satellite data sets are one important component of this effort. Active engagement of the scientific community in the design, testing, and evaluation of data management functions and services is essential. NPOESS will advance our understanding of the Earth system, but an effective and efficient data management system is necessary if we are to realize the full potential of NPOESS.
Page 94 PREV PAGE TOP OF DOC
Thank you for the opportunity to testify this morning, and I would welcome any questions that you may have.
BIOGRAPHY FOR MARK R. ABBOTT
Mark R. Abbott is Dean and Professor in the College of Oceanic and Atmospheric Sciences at Oregon State University. He received his B.S. in Conservation of Natural Resources from the University of California, Berkeley, in 1974 and his Ph.D. in Ecology from the University of California, Davis, in 1978. He has been at OSU since 1988 and has been Dean of the College since 2001. Dr. Abbott's research focuses on the interaction of biological and physical processes in the upper ocean and relies on both remote sensing and field observations. Dr. Abbott is a pioneer in the use of satellite ocean color data to study coupled physical/biological processes. He advocated the inclusion of chlorophyll fluorescence bands in MODIS (the Moderate Resolution Imaging Spectroradiometer on EOS Terra) and is developing next-generation ocean primary productivity algorithms that will use fluorescence data to estimate the physiological health of upper ocean phytoplankton. As part of a NASA Earth Observing System interdisciplinary science team, Dr. Abbott is leading an effort to link remotely-sensed data of the Southern Ocean with coupled ocean circulation/ecosystem models. His field research included the first deployment of an array of bio-optical moorings in the Southern Ocean as part of the U.S. Joint Global Ocean Flux Study. He deployed bio-optical drifters in both the California Current and in the Antarctic Circumpolar Current to study the impacts of mesoscale ocean circulation on ocean primary productivity. He is using moored and drifting optical sensors to study the relationship of mesoscale processes to the survival of juvenile salmon in the coastal ocean off Oregon, as part of the NOAA/NSF Global Ecosystem Dynamics program.
Page 95 PREV PAGE TOP OF DOC
Dr. Abbott is also an adjunct professor in computer science at OSU, and he has advised six Master's students in computer science. These students have worked on advanced data mining and data base techniques as applied to issues facing the Earth Observing System. He has also advised the Office of Naval Research and the National Science Foundation on ocean information infrastructure. He authored a white paper for the National Research Council on the future needs of Earth science in regards to computation. He has hosted a series of workshops for NASA, bringing leaders in computer science from industry and academia together with Earth scientists.
Dr. Abbott is currently a member of the MODIS and SeaWiFS Science Teams, and he chaired the Committee on Earth Studies (CES) for the National Academy of Sciences. While chair of CES he oversaw the production of three reports on NPOESS, including one focusing climate data services in support of NPOESS. He is presently a member of the NRC Committee on NASA/NOAA Transition from Research to Operations. He also serves on NOAA's Committee on Long-Term Climate Monitoring. He is chairman of the U.S. Joint Global Flux Study Science Steering Committee.
80811m.eps
Discussion
Chairman EHLERS. Thank you to all of you for the testimony. It has been very useful.
Preparing for NPOESS
We will now begin the question period. First of all, Admiral Lautenbacher, it seems to me, based on what we have seen and heard, that you are really going to have trouble handling the data stream from the new satellites. And it is not clear that you are doing everything that you could with the data stream you currently have. What are the plans for changing that? What resources do you need? How do you propose to get them, etcetera? I would appreciate some comments on that.
Page 96 PREV PAGE TOP OF DOC
Admiral LAUTENBACHER. Yes, sir. I think probably the one slide that I had in here is worthwhile just going back over. I have a slide in there. It talks about preparations for use of NPOESS satellite data on day one of its availability.
Let me say that, you know, NPOESS is a long way in the future, and you have to work through this. We don't even have a contract so you can't decide today exactly what this is going to look like, but we have a plan to work on.
First of all, we are using precursor NPOESS sensors right now that are flying on NASA satellites, taking that data, working with it, trying to manage it, use it, and insert it into our system. We also have some—the NPP project, which is the NPOESS Preparatory Project, which will be going up in 2005 and will have some of the sensors—a few of the new sensors that will be able to start working with those, get the data, start on data assimilation, which is a whole separate subject to talk about. and be ready for the day that you throw the switch on NPOESS.
We have been getting technology assessments from our friends at the National Academy of Science. We have had information technology reviews from our information technology experts, the National Academy of Sciences and outside. We have had science reviews. As a matter of fact, Dr. Abbott has been on a panel, the space panel, and has provided some of the information that he has provided to you today to us and has been an advisor. And so we are reaching out to the research community and the National Academy, academia as well as government agencies to provide the best knowledge, the best advice they can give us so that as we build this development plan, we do it correctly.
Page 97 PREV PAGE TOP OF DOC
I want to mention that we also do a couple of things that help us in total, and those are the partnerships that we have internationally to help use the data and to help share—so we share knowledge with the folks that we have partnerships with, the Japanese, the French, EUMETSAT. We have several international groups that work together, and so we work together with them to share our knowledge of data and the transfer of that data and use of them.
We also have a very strong outreach program to our users in which we have a training program to try to involve people in using our data, providing feedback, giving us the kinds of things we need to do to develop better methods in the future. We have trained over 4,000 people in the last few years through e-training, e-government, e-business. We have provided online training courses for folks. So there is a phase program to get to the point where we believe at day one, we will be able to use that data stream.
Funding Issues for Current Data Products
What is it going to take to get there? It is going to take both strong support within the Administration, which I will work to get for the funding, and the help of Congress in terms of the funding increments that I have shown in that one slide. We have not done so well, and the Senate has earmarked that increase that would help us turn some of those green blocks—some red blocks to green blocks that I showed. And that would be the second year in a row that it would be earmarked. We asked for this last year as well. This would impact on the 65 percent of the products that we are unable to deliver. This money was designed to put into the budget to bite off a chunk of that and to move us up to being able to serve our needs. The Joint Data Assimilation Center is very important, because this is the key to taking data. Let us say processed data from the satellites, and turning it into a useful product that can be put into models, and that is on the edge as well. That is a very important increment. These are small amounts of money when you compare them to the value of this system.
Page 98 PREV PAGE TOP OF DOC
Now let me make one other point, and I realize I am taking a lot of the time, but we are talking about data streams here, but you also have images that come. The images are extremely important that you get. You get them right away, and you can use them and they go right out. We are talking about integrating, in some cases, what I would say, esoteric not arcane, but some very specific data streams from many, many channels in an instrument into a very complex model. You just don't do that instantaneously, as we have heard from our experts here today. It takes time to do that. We plan and we work to try to do that the best that we can. But the imaging is there instantaneously, and we provide that right away. Let me stop, sir. I will answer other questions later.
Chairman EHLERS. My time has expired. I will recognize Mr. Udall.
Metadata Collection and Storage
Mr. UDALL. Thank you, Mr. Chairman. Welcome to the panel. It is great to have you all here today. I was thinking that the good news is we have enormous volumes of data, and the bad news is we have enormous volumes of data. I had read an interesting little comment recently where somebody pointed out that an individual living in the 1700's would have had access throughout—and it probably would have been a man at that point in time, access to the amount of information in his whole lifetime that is available on a Wednesday edition of the New York Times. It is something to certainly think about, not to mention the Tuesday edition with the science page and all the rest.
Page 99 PREV PAGE TOP OF DOC
I had a question first for Dr. Abbott. The value of a long-term science database, like NPOESS, is critically dependent on collecting so-called metadata as well as the actual instrument readings. Has NOAA made adequate provision for metadata management in its current planning? Maybe you could explain metadata as well.
Dr. ABBOTT. Okay.
Mr. UDALL. Would you do that?
Dr. ABBOTT. Sure. Metadata is basically data about data. It talks about things like where the data were collected, what algorithms were used, what was the calibration of the sensor, what version, maybe even who did it. So it is all of that. It is like going into a card catalog and finding where a book is, so that in that card catalog there is information, and that is metadata about that record.
I think that that is an enormous challenge. It is not clear to me yet that NOAA recognizes what it takes to do that. I have talked to other scientists who have begun those discussions. It is a hard, hard problem, but it is really the fundamental thing that has to be done first. And I think rather than trying to design the complete system all at once, we need a lot of prototype activities that can begin now with the satellite data records we have now and begin to put them in a climate research suitable framework to begin to develop that metadata. But that is an excellent point.
Mr. UDALL. When you are going back and reviewing the data that is in front of the metadata, then at some point you are going to want to study, perhaps, the metadata for a better understanding of your methodology and where you could adjust future data collection efforts. Is that—that is the importance of it?
Page 100 PREV PAGE TOP OF DOC
Dr. ABBOTT. That is true, because for example sometimes people change the operating conditions for the satellite. Maybe it is operating at a little bit higher temperature and they write that down in a notebook or something.
Mr. UDALL. Um-hum.
Dr. ABBOTT. That information needs to be captured so that people 10, 20 years down the road can come in and understand why did all of a sudden the record look different? Is this a change in the Earth's system or was it a change in the satellite or the way the data were handled? And that has been, traditionally, a very hard thing to do. We have not captured all of that information. And so trying to get that consistent record for 20 and 30 and 40 years has been one of the big challenges in climate research. And that is true for satellite data as well as data collected on the ground or in the ocean or in the atmosphere.
Archiving Data Products
Mr. UDALL. Thank you, Doctor. Admiral, good to see you again, and you have clearly put a lot of thought into how we need to proceed. Has there been given any thought to reducing the future load on NCDC by distributing these archives, perhaps, to the National Center for Atmospheric Research or the National Geophysical Data Center? Is there any chance we could use those facilities to assist in this effort?
Admiral LAUTENBACHER. Obviously we have a number of contracts in association with the scientists who help us in using the data, and the models and all of that. So there are things going on. Now in terms of starting another data center, we have not explored that potential. I mean, certainly it is something we can look at. We have three large data centers and we appreciate very much the fact that you visited—in fact all of our distinguished panel up here has visited our centers in Boulder. And we appreciate that, because that means a lot to the folks out there who work on these very difficult problems.
Page 101 PREV PAGE TOP OF DOC
But we have these three major data centers. To us, right now, it looks like this is a reasonable architecture for maintaining data. It doesn't mean in the future that it wouldn't make sense to have other data centers or that we don't need more help and assistance in working on the issues that Dr. Abbott has talked about. I am a firm believer in partnerships and working with academia and our research-using community.
Mr. UDALL. Dr. Abbott, would you have a comment as well on my question?
Dr. ABBOTT. I think the first challenge is really to preserve the raw data and the metadata that goes with it. That is going to be an enormous challenge. It is—as far as the next level of archives, I would say looking at ones where there are maybe more specialized kinds of products and distributing those responsibilities rather than trying to keep it all central, I think that is certainly worth exploring. It makes a lot of sense.
Mr. UDALL. I might—Mr. Chairman, one last comment. I note that Mr. Gutknecht here, Mr. Ehlers, and myself, and—the President recently was reported as calling three Senate candidates ''The Three Amigos'', but I think you are really looking at ''The Three Amigos'' right here. We had a great visit in Boulder, and I look forward to further opportunities in that regard. Thank you, Mr. Chairman.
Chairman EHLERS. The gentleman from Minnesota, Mr. Gutknecht, for five minutes.
Page 102 PREV PAGE TOP OF DOC
Meeting Requests for Data Products
Mr. GUTKNECHT. Thank you, Mr. Chairman, and I thank my colleague from Colorado. And I did enjoy my visit. In fact, Mr. Chairman, I would recommend, if it is at all possible, that we send some staffers out to visit the facilities not only in Boulder but also the renewable energy facility, which I also toured in Golden, Colorado. I think it would be very instructive for our staff to be able to go out there some time during the interim.
First of all, though, Mr. Chairman, I would like to submit, and ask unanimous consent to submit, for the record, a column, which appears in today's Wall Street Journal, which I think is fascinating, and I would encourage all members to read.(see footnote 10)
It leads with a quote from Mark Twain, and I love this quote. He said, ''There is something fascinating about science. One gets such wholesale returns of conjecture out of such a trifling investment of fact.'' And while I appreciate that work that goes on at NOAA, the story in this column is really about what is going on in California and the whole debate about global warming. And I think we all have to step back once in a while. I have a friend in Minnesota, who is a scientist and who helped develop the first spectrometer. And he told me something rather interesting. He said, ''Just because we can measure parts per billion does not necessarily mean that they are statistically significant.'' And I think as we learn more and more about these things, I think we have to be very, very careful about the conclusions that we draw.
I do have a question, Admiral. You mentioned in your testimony that the National Environmental Satellite Data and Information Service now produces 500 satellite products, however NOAA's FY 2003 budget summary stated that that service could actually only deliver about 175 of those products. That is roughly 35 percent of the products requested by the National Weather Service, importantly, in a timely manner. I guess the question the Committee would like to know is what specific actions is NOAA taking to ensure that more of these products can be delivered in a timely way?
Page 103 PREV PAGE TOP OF DOC
Admiral LAUTENBACHER. Yes, sir. That is a good question. We have asked now actually two years in a row. We asked last year, in fiscal year '02, for an increment of about $6 million to our $25 million operating budget that takes care of those things. It was rejected by Congress. And we have asked for it again. I have been able to convince the Administration to ask for that increment again. We have asked for it in our budget. This will take—this will help in meeting the—it is not going to cover the whole thing. But I think it is an executable bite of those additional products, which I think the country needs and we would like to give to the country for our—as part of our service.
Mr. GUTKNECHT. I yield back the balance of my time.
Chairman EHLERS. The gentleman yields back his time. The Chair recognizes Mr. Baca from California.
Funding Requests
Mr. BACA. Thank you very much, Mr. Chairman. The first question I have is for the Admiral. You know, this is a time everybody is requesting for additional funding. And why has NOAA requested a $4.3 million cut for FY 2003 for its data center information services when everybody is asking for money?(see footnote 11)
Admiral LAUTENBACHER. I am not sure I understand where that $4.3 million is coming from. The chart that I have covers the subject that we are talking about today. I don't know. I have got to get back to you. I don't know exactly. If we have asked for a cut, it shows up somewhere else, because in this area, we are asking for more money. There is no doubt in my mind about that. Was it an earmark? Okay. We are talking about an earmark, I guess, that was—it is no longer in the budget. The Administration's position on earmarks is that we don't like them, I guess. So earmarks, you know—as I mentioned when I was here for my testimony originally, a lot of the earmarks that were in the budget were removed before they came over here to the Hill, so I am again——
Page 104 PREV PAGE TOP OF DOC
Mr. BACA. Yeah. When we saw that information, it is like everybody should be requesting. Here is a cut, you know, for——
Admiral LAUTENBACHER. Yeah.
Mr. BACA.—4.3.
Admiral LAUTENBACHER. Yeah. I——
Mr. BACA. Especially for its data center and information services.
Admiral LAUTENBACHER. And obviously, the more money we can put into this area, the better I like it. But I have to stand with the request that we have asked for, because we think that is the most efficient way to use the money that is available for this mission right now. That is the best answer I have for that one, sir.
Technology Needs and Planning
Mr. BACA. The next one is for Ms. Koontz. The next question, has NOAA identified the source for the computation and storage capacity it intends to use for upgrading the agency's data system? That is question number one. And are the technologies selected for the panel realistic and likely to be available in time to support NOAA and the Department of Labor requirements? And are these estimates, the cost make it necessary for the upgrades?
Page 105 PREV PAGE TOP OF DOC
Ms. KOONTZ. If you are talking about the money that is needed for infrastructure improvement, I don't think that there is a number yet associated with that. One of the things that you have to remember is that, again, the contract has not been awarded yet. It won't be awarded until the end of next month. And without knowing what the specific characteristics of the system are going to be until it is designed, it is very difficult to come up with a number such as that.
Mr. BACA. I guess it is the actual technology that is going to be available. That is really the question, not the money.
Ms. KOONTZ. That is also an issue. I mean, we are talking about a program. We are talking about six or seven years out. It is very difficult to predict what your technology options are going to be at that point.
Mr. BACA. Well, the next question, I guess, is for Dr. Abbott. What time horizon should NOAA and Congress employ in making decisions about resources—commitment to support NPOESS data?
Dr. ABBOTT. Well, I think first we have to separate out the science issue and clearly the records that people need to have are years to decades to begin to understand the changes in the Earth's system. But I think a lot of the—in regards to satellite data management, I think we need to begin now and start small with a lot of very specific focused activities to test the science requirements, to understand the technology so that we are really building small simple systems that, in some sense, are sustainable but have inherent flexibility, so that we can take advantage of new technology and new capabilities and new science as we begin to learn more as the data records get longer.
Page 106 PREV PAGE TOP OF DOC
My concern is that if we say this is going to be the system for the next 20 years and we have to build that system now, that is probably going to get us into a lot of very unhappy people. We are not going to be able to deliver the services and functions that we need, either for the public or the science community.
Mr. BACA. How long will the NPOESS data be useful?
Dr. ABBOTT. Well, we are still using DMSP and POES data and that is now 20 years old, some of it. And I would imagine that if we are careful and preserve it and do the calibrations right, that that data will be useful for generations, potentially.
Mr. BACA. How accurate will that information be then?
Dr. ABBOTT. How accurate?
Mr. BACA. You were right. Yeah.
Dr. ABBOTT. Excuse me?
Mr. BACA. How accurate will that information be?
Dr. ABBOTT. Oh, I think that we will continually——
Mr. BACA. If you look at the horizons and the times and the——
Page 107 PREV PAGE TOP OF DOC
Dr. ABBOTT. Oh, clearly probably at least 10 to 20 years, and then I would imagine that as the sensors improve and our knowledge improves, we are going to walk the next generation on sort of an every 10 year time frame, but the old records are still useful. Even though they may be noisier and not quite as accurate, they will still be useful for climate research.
Oversight Panel for NPOESS
Chairman EHLERS. The gentleman's time has expired. We have time for a second round of questions. First, Admiral Lautenbacher, do you have a standing user-advisory committee that meets with you regularly and that advises you on the planning of the data handling, archiving, processing, etcetera?
Admiral LAUTENBACHER. I don't have a user panel that reports to me directly. It is certainly a good idea. I have a number of constituent groups that come in and talk to me on a regular basis. I think that is a good idea, and I will look into it. I know that my satellite service does make contact with users on a regular——
Chairman EHLERS. Right.
Admiral LAUTENBACHER [continuing]. Basis and has consultations with them, but I don't have one at my level yet, but I will do that.
Chairman EHLERS. I think it would be useful just to—well, for several reasons. Frankly, I am very concerned about the project, and I am concerned about that you—to be sure that you have the resources that you are going to need when the data starts pouring in. You know as well as I that it may take two fiscal years to persuade the Congress to appropriate something, particularly in these tough budget times. And so even though you can't anticipate the specific projects and precisely how you will do it or use the resources, it is certainly important to start building your case early——
Page 108 PREV PAGE TOP OF DOC
Admiral LAUTENBACHER. Yes, sir.
Chairman EHLERS [continuing]. And an outside user-advisory committee can substantiate your request and gives you greater credibility with the Congress, so you might consider that.
Admiral LAUTENBACHER. If I could add just one more comment, sir, to that. On the oversight panel of which I am a prime member for the NPOESS project, that includes a users group who have essentially given their requirements to us and have been assimilated and evaluated, so there is a direct connection in the NPOESS project with a user community at large. It is within the NPOESS set up. It is not a separate data. It is part of the NPOESS project management system.
Chairman EHLERS. Right. I am basically concerned about the research committee, and in particular, I know from personal experience that you get new people, younger people in very frequently who have very good ideas, but tend not to get heard until they have built a reputation. So what I am envisioning is a user committee, not just gray-haired or bald-headed old men advising you, but rather bright young men coming up who can anticipate some things that they would like to see. But at any rate, I wanted to mention that.
Modernizing Information Technology
Also, Ms. Koontz, you have some experience, and you have started other agencies, which have large computer infrastructure projects, which are facing similar problems. Do you have any specific estimates or ideas of what is going to be required here and how we should proceed? How many dollars will be required? What is a good procedure to follow in this based on your experience with other agencies?
Page 109 PREV PAGE TOP OF DOC
Ms. KOONTZ. I don't think that I could have the expertise to put a dollar figure on what needs to be done. But what I would like to highlight is I think that a lot of the conversation here has been talking about the importance of understanding the interrelationships among the various projects that NOAA is proposing and has ongoing to get them to be able to fully utilize the data that they are going to get from the new satellite systems and also be able to archive it and make it available to the public. And I think one way that is very important in terms of getting the visibility over that big picture is an enterprise architecture. An enterprise architecture is a blueprint for modernizing an organization's information technology.
Just as Dr. Abbott said, you don't run out and buy the lumber and the nails until you have decided what it is that you are going to build in the first place, and this is a blueprint for doing that. And what it does is say where you are, your baseline. It says where you are going, a target architecture. It should provide a plan for transitioning, and it should associate all of the individual projects with that plan. And it is a good way to highlight what those interrelationships are. And it is a good way to ensure that you don't have redundancy in your efforts. And I do know that NOAA has been working on an architecture, I believe, since 2000, and has recently come out with a new version, continuing to work on that architecture will be important to this whole area.
Chairman EHLERS. Well, would you agree that the most important thing is to make certain that all of the incoming data are archived and also the metadata immediately so that even though we may not know what use we may make of it later, at least it is archived and retrievable? Would you say that is the first and most important thing?
Page 110 PREV PAGE TOP OF DOC
Ms. KOONTZ. I guess it would be difficult for me to say what the highest priority is, because there is obviously a high priority as well in terms of using the data for operational purposes as well. But it is very important that the data be archived. And I would add that the whole issue of archiving data, preserving it for later use—is a challenge that is not just associated with NOAA, but it is a government-wide issue, and it is something I think facing everybody right now. And it is a tremendous challenge.
Chairman EHLERS. Well, I can assure you, as a scientist, I have been in situations where I was extremely frustrated that certain bits of data have not been archived and made it extremely difficult to get meaningful results out of a great deal of work. If you have it, you can always use it later, but if you don't have it, you are lost. My time has expired. Mr. Baca has left, so I will continue to roll onward. The next question is about the private sector interaction here and we have had some testimony a few years ago there was somewhat of a thorny relationship.
Role of the Private Sector
It seemed to me, the private sector wants NOAA to do whatever they can't do, but they don't want NOAA to do anything that they can do and charge for. That may not be a fair representation, but I just wanted to see what is their relationship with the private sector at this point, and in particular, how much are they paying you for your services, Admiral Lautenbacher? And does their payment, in any sense, offset the cost that you have in providing the information to them, and a related question is how do you allocate the cost? Obviously you can't charge them for all of the satellite use, but I would just like to have an update on that.
Page 111 PREV PAGE TOP OF DOC
Admiral LAUTENBACHER. Yes, sir. We continue, obviously, to have relationships and provide data to commercial users. Certainly in the weather part of the world, a strong civilian sector that provides services. And there is, you know, a market for our products and a market for their products. There is this gulf in between you have talked about that—well, who is responsible, and of course, they would like to take as much over that as possible.
Right now, there is a study ongoing by the National Academy of Sciences to take a look at that split to see what makes sense, what is in the realm of a public utility that really needs to be provided by the Government, and what is in the realm of the private sector that needs to be really left to the private sector and handled as either niche markets or particular economic development sectors aside from the public utility and public safety issues that we cover. I can't claim that that study will solve the controversy, but that is what is happening right now. We do—in charging for the data, what the general—the allocation of cost and the charging for the data really is the incremental cost that we incur in data transmission.
So it is—you know, you are paying for the servers and the extra set up. So you are really paying a very small marginal cost of what these systems are costing the taxpayer. In my view, we are not making money or able to pay for our services based on what we charge for the users that come in and use our products commercially.
Chairman EHLERS. They are only reimbursing you for the marginal cost, basically?
Page 112 PREV PAGE TOP OF DOC
Admiral LAUTENBACHER. Yes, sir. Marginal cost of transmission of data and providing it to them, whatever that takes.
Chairman EHLERS. That is a real bargain.
Admiral LAUTENBACHER. It is a bargain. Yes, sir. The commercial sector has a bargain, in my view.
Chairman EHLERS. Well, what about the researchers or institutions who use this information? If they are not—if they don't have a research contract with you, do they—suppose they are supported by the National Science Foundation, is there any charge for them to use the information?
Admiral LAUTENBACHER. I don't think—free of charge up to $2,000 worth of data. Generally speaking, a lot of our data is on the Internet. It is online, and you can go to our data centers and obtain that data, if you are a researcher.
Aerosol Sensors on NPOESS
Chairman EHLERS. Okay. I have been handed a question by the staff, which I will transmit to you. It is—well, I am always honest about this. Most of my questions are my own. It is our understanding that NOAA will be requesting money for an extra sensor on NPOESS, a new aerosol sensor at an additional cost to the project of about 40, $50 million. Was there any discussion of eliminating one of the current sensors to offset the cost of the new aerosol sensor, and if not, why not?
Page 113 PREV PAGE TOP OF DOC
Admiral LAUTENBACHER. I don't have the full background. I would like to provide that for the record, if I could, and give you a more——
Chairman EHLERS. Okay.
Admiral LAUTENBACHER [continuing]. Complete answer than something off the top of my head at this point. We would like to have more aerosol coverage. That is true. I don't know whether we are talking about—I don't have enough details to tell you at this point. I will get back to you on it, sir.(see footnote 12)
Chairman EHLERS. Okay. I have intimidated my staff. He won't hand me any more questions. I think that covers all of the urgent questions that I had in mind. We will, of course, ask you to respond to any written question that we send in and return those. We will keep the record open until we receive your replies.
Improvements in Predicting Weather
Just in conclusion, let me say, I really appreciate what you are doing. I was just commenting a few weeks ago to a colleague who is concerned about all of the money we spend on various aspects of research and specifically mentioned NOAA. And I said all I can do is comment on the difference in weather forecasting between when I was a child, which was quite some time ago, and the current time, and I lived in the Midwestern State of Minnesota. It is not quite the tornado belt, but we did have tornadoes, or as we called them back then, cyclones, and we had no warning, absolutely no warning. The only warning was to look at the sky. If it looked too threatening, you dive for the basement.
Page 114 PREV PAGE TOP OF DOC
If it happened during the night, there was no hope whatsoever. You just—unless the thunder and lightening awakened you and scared you enough to seek shelter. The weather forecast was scarcely better than guessing. In fact, most of the farmers I lived among trusted their own judgment on the weather more than the radio reports. Today, it is a totally different system. We have become a nation of atmospheric researchers, amateur weather forecasters. And I know when I have something that is weather related, I turn on the weather channel, I look at the radar myself, and particularly every time I am flying, I check it and decide which route to fly. If it looks like I might encounter some weather on the path I am taking, I will switch to another airline. I think a good many people do that, not just for flying but for other weather planning activities.
It is an amazing, amazing system that we have in today's world that the lay person can actually make some sense out of the weather and make their own forecast for their own area. And a good example is something which you don't ever have to do, but something we have to do regularly and that is participate in parades. And regularly, if the weather looks threatening, I am looking at that radar and I am timing exactly, you know, how much time we will have in that parade before it is going to rain, and whether I should use the convertible that I have borrowed for it or whether I should use a hard top.
These sorts of decisions lots of different people are making in lots of different ways based on the data that you provide.
Admiral LAUTENBACHER. Um-hum.
Page 115 PREV PAGE TOP OF DOC
Chairman EHLERS. So I just wanted to go on record to say I think that we, as a nation, and you in particular, have developed an amazing weather system. The satellite program has done wonders to transform that, and I, for one, appreciate it, and I believe Americans appreciate it. They may not know the source of the information. They may never have heard of NOAA, and in fact, I am aware of a person who said why do we even have the National Weather Service now? I get all my weather off the TV. You know, there is a great deal of ignorance about what you are doing, but I can tell you, our nation depends on it, and a great many people in this country depend upon it. And I just want to express my appreciation to you and to all of your employees for all the great work they do, and Dr. Abbott and other researchers whom you represent here for the work you do.
We have much to do. We need to talk about issues such as global climate change. We need the ARGO system. We need far more data points produced in various parts of the world, but we have done an amazing amount with the information that we are able to accumulate just within the continental United States. And I, for one, appreciate it. On that note, I will declare the hearing closed. Thank you very much.
Admiral LAUTENBACHER. Thank you, Mr. Chairman.
[Whereupon, at 11:26 a.m., the Subcommittee was adjourned.]
Appendix 1:
Additional Material for the Record
Page 116 PREV PAGE TOP OF DOC
QUESTION FOR THE RECORD
Response by Vice Admiral Conrad C. Lautenbacher, Jr., Under Secretary of Commerce for Oceans and Atmosphere, National Oceanic and Atmospheric Administration
Question submitted by Representative Joe Baca
Why is NOAA requesting a $4.3 million decrease to data (Archive, Access, and Assessment) activities?
Answer: The total for Archive, Access, and Assessment activities does have a $4.3 million reduction in the President's budget request. However, this net change is made up of three components, a reduction to two special projects and an increase to core data activities. First, the Climate Database Modernization program has a decrease of $9.6 million, a change from $15.8 million to $6.2 million. This level of funding will allow NOAA to continue efforts converting historical climate data to digital media. Second, the GOES Data Archive has a decrease from $2.0 million to $0. These funds were provided for one year only in FY 2002. Last, core activities in Archive, Access, and Assessment are requested to increase by $7.6 million. This funding will allow for the continuation of current activities as well as increases in key programs.
These increases include: $2.2 million to continue current activities through adjustments to base; $1.7 million for an improved climate data and information delivery service; $1.6 million for a new generation of World Wide Web accessible climate information and statistics for primary use by the energy sector of our economy; $1.3 million to update the World Ocean Database to include new sources of data and a data management infrastructure; $0.5 million to extend the climate record through Paleoclimatology; and $0.3 million to establish a long-term Solar X-ray Imager archive that will address the need to mitigate space weather events.
Page 117 PREV PAGE TOP OF DOC
QUESTION FOR THE RECORD
Response by Vice Admiral Conrad C. Lautenbacher, Jr., Under Secretary of Commerce for Oceans and Atmosphere, National Oceanic and Atmospheric Administration
Question submitted by Chairman Vernon Ehlers
Admiral Lautenbacher: It is our understanding that NOAA will be requesting money for an extra sensor on NPOESS, a new aerosol sensor at an additional cost to the project of about $40M, $50M. Was there any discussion of eliminating one of the current sensors to offset the cost of the new aerosol sensor, and if not, why not?
Answer: The proposed Aerosol Polarimetry Sensor (APS) on NPOESS is a cost shared with the Department of Defense (DOD) spread over FY 2004–2008, so the cost to NOAA is half of the current total estimate of approximately $31.1M. The program office reviewed different alternatives for meeting this user validated requirement, such as integrating the capability into an existing sensor or replacing one of the existing sensors.
No current NPOESS sensor has the multi-angular scanning geometry required to actually measure the aerosol physical properties needed to determine the type and source of the aerosols, so modifying an existing sensor was not possible. Replacing an existing sensor with the APS was not viable because all existing sensors are essential to meet validated civil and military requirements and were strongly defended during the recent requirements document update and revalidation.
Page 118 PREV PAGE TOP OF DOC
80811n.eps
80811o.eps
(Footnote 1 return)
Admiral Lautenbacher edited his testimony to indicate that he intended to say 10 times more data.
(Footnote 2 return)
Within NOAA, NESDIS processes the satellite data, and the National Centers for Environmental Prediction (NCEP), a component of NOAA's National Weather Service, runs the models. For simplicity, we refer to the combined NESDIS/NCEP processing center as the NESDIS processing center.
(Footnote 3 return)
NOAA uses different nomenclature for its data processing stages: raw data are known as level 0 data; raw data records are known as level la data; temperature data records and sensor data records are known as level lb data; and environmental data records are known as level 2 data.
(Footnote 4 return)
Volcanic ash presents a hazard to aviation because of its potential to damage engines.
(Footnote 5 return)
The four sensors supporting key EDRs are (1) the advanced technology microwave sounder, (2) the conical microwave imager/sounder, (3) the cross-track infrared sounder, and (4) the visible/infrared imager radiometer suite.
(Footnote 6 return)
The five sensors include (1) the conical microwave imager/sounder, (2) the cross-track infrared sounder, (3) the global positioning system occultation sensor, (4) the ozone mapper/profiler suite, and (5) the visible/infrared imager radiometer suite.
(Footnote 7 return)
The five program office-developed sensors are (1) the cross-track infrared sounder, (2) the conical microwave imager/sounder, (3) the global positioning system occultation sensor, (4) the ozone mapper/profiler suite, and (5) the visible/infrared imager radiometer suite.
(Footnote 8 return)
NASA is developing the advanced technology microwave sounder sensor.
(Footnote 9 return)
National Research Council, From Research to Operations in Weather Satellites and Numerical Weather Prediction: Crossing the Valley of Death (2000).
(Footnote 10 return)
Please see article in Appendix 1: Additional Material for the Record, pp. 74–75.
(Footnote 11 return)
Please see Admiral Lautenbacher's written response in Appendix 1: Additional Material for the Record, p. 72.
(Footnote 12 return)
Please see Admiral Lautenbacher's written response in Appendix 1: Additional Material for the Record, p. 73.