Methodology: How the data was collected

EDF and Google Earth Outreach, along with researchers at Colorado State University, spent several years developing and testing these systems, demonstrating that we could deliver accurate results in real-world conditions.

Project scientists have published the methodology and mapping results in two peer-reviewed papers in Environmental Science & Technology in March 2017 and September 2018.

Data collection

This study uses two different sets of vehicles, each with different instrument packages: one vehicle, based at Colorado State University, used for experiments and intensive local monitoring; and another set of Google Street View vehicles that are deployed to cities around the country. All of the data for the maps is generated by the instruments aboard the Google vehicles.

The CSU research vehicle measures methane concentrations with a LiCor 7700 methane analyzer. The analyzer is an open-path design that measures methane in the air as it passes through the analyzer space. GPS location is determined with a Hemispheres A100 GPS with precision of 24 inches. Wind speed and direction is measured with a Gill WindMaster sonic anemometer and location with a Hemispheres GPS unit.

For these sensors, the frequency of data recordings is ten times per second, which corresponds to about three feet when driving at 20 mph. The CSU vehicle also uses a methane carbon isotope analyzer to discriminate between biological and fossil fuel sources of methane when required. This instrument records data approximately one time per second, corresponding to about 35 feet of street-level observation at 20 MPH.

The Google Street View vehicles which collect data for this study use a Picarro methane concentration analyzer that has a closed-path design with a sampling frequency of one data point every half-second, resulting in samples every 13 feet when driving at 20 MPH. These vehicles are also equipped with a Hemisphere GPS Receiver MD-100 GPS unit and Climatronics Sonimometer RS₂₃₂ anemometer for measuring wind speed and direction.

Drivers are asked to drive every public street in a study area at least twice to ensure accuracy. Re-driving helps confirm the persistence of elevated methane readings and characterize their emission rate accurately. Drivers do not view the data as it is being collected.

EDF and CSU carefully select survey areas to ensure that a representative sample of a city's local distribution system is mapped. In particular, when surveying large cities, it is impractical to map every street. Survey areas are selected to represent the variety of conditions in the city – different residential and commercial areas, type of pipe materials and age of the pipes.

Measuring leaks in downtown areas presents special challenges: Tall buildings can interfere with GPS signals and create wind tunnels that reduce the ability to locate leaks. Because of this, EDF and CSU generally choose other neighborhoods in which to measure leaks.

Data Variables: Location, Attribution, Magnitude, Uncertainty

Ambient methane concentrations in the atmosphere average around 1.8 parts per million (ppm). There is natural variability so it is possible to see average daily concentration values between 1.75 and 2.2 ppm in some cities and some days, depending on local sources and winds. Such variability is well within the sensitivity of the methane analyzers in the vehicles; these instruments can precisely measure methane concentrations down to 0.02 ppm.

To determine when the vehicle is driving through an area where methane readings are elevated above the background levels, researchers examine not just the concentration values, but also variability. When driving near a natural gas leak, the atmospheric methane concentrations tend to be higher and more uneven than ambient concentrations, much like the swirling patterns of white and brown in a coffee cup when first adding milk. When driving through air that is unaffected by a local leak, the instrument reads very stable values (a statistically low standard deviation), like the even color of coffee when the milk is well mixed. The CSU team has established thresholds for the enhancement and variability that are used to determine where methane readings rise significantly above background levels.

Air movement and below ground migration of natural gas can cause the plume location to differ from leak location. Based on validation data, the researchers have confidence that indicated leak locations are accurate to within 300 feet.

Biological processes can cause elevated methane concentrations in air, so it is important to discriminate whether methane emissions are due to leaks, vehicle emissions, landfills or other biological processes. The overwhelming majority of natural gas leaks are observed at elevated methane concentrations over a distance of 500 feet or less.

In contrast, methane emissions from landfills or wetlands cause elevated readings over thousands of feet.

An area is identified as having a natural gas leak if elevated methane readings extend over a driving distance of less than 500 feet.

Elevated concentrations that persist over 500 feet are screened to see if measurements are adjacent to known biological source areas, and if so, the elevated readings are set aside. Areas of elevated readings longer than 500 feet that are not adjacent to known biological sources of methane require further investigation.

There is evidence that some large-area observations of elevated methane concentrations arise from long stretches of natural gas distribution lines with many leaks. In other cases, the most likely cause is city buses or other vehicles powered by compressed natural gas. In some highly complex situations, both vehicle and natural gas sources overlap to generate complex patterns that are difficult to resolve.

Under these scenarios, attribution of elevated methane readings to leaks from natural gas local distribution infrastructure may be difficult. In other words, the leak patterns for local distribution leaks and non-local distribution sources (like traffic or biological sources) may be similar if elevated methane readings are detected over a large continuous area. CSU excludes these complex situations that present unclear attribution, resulting in conservative estimates of leak frequency and magnitude. Ongoing work will study ways to better resolve these complex patterns.

The CSU research team developed a way to estimate the leakage rates from verified sources. To develop this leak rate estimate, researchers conducted controlled gas release experiments, where methane was released into the air at varying rates, and vehicles were driven through the resulting plume. They found that the methane concentration patterns are distinctly different with different methane leak rates, allowing researchers to assign leaks to bins of small (0-13 cubic feet per hour), medium (13 to 85 cubic feet per hour) and large (above 85 cubic feet per hour) leak rate. Due to variability in wind and other uncertainties, it is not possible to provide more precise estimates of leak rates at this time.

Results were validated through blind controlled releases followed by a comparison of predicted versus observed results. In addition, approximately seven leaks were measured using comprehensive in-place tenting methods, the results of which were compared to the emissions rates using the procedures described above. Both validation efforts demonstrate the ability of the technique to discriminate between relative leak sizes.

The methane analyzers used in this study are high-precision and high-accuracy instruments when used as per the manufacturer's specifications, so it is extremely unlikely that any elevated readings are a result of errors with the instrument. False positives and false negatives are both possible but are not evenly spread across the study area. Statistical analysis shows the probability of missing an observable leak, when the area is driven three times, is less than 6%.

This variability is taken into account when estimating the overall leak frequency from a city or part of a city. When driving past known leaks, the study's best data indicates that with a single drive-by, there is a 76 percent chance of observing the leak.

The probability of missing or identifying a leak depends on a variety of factors including the magnitude of the leak, local atmospheric conditions and soil conditions. Researchers are actively working to further assess the reasons that leaks go undetected.

Data Analysis: Algorithm

Because this project accumulates large amounts of data (about 194,000 data entries per hour), it is necessary to automate the analysis for extracting patterns in the data. The CSU team developed a set of data analysis algorithms used to automatically process the data in a number of steps.

First, data are checked for quality and are corrected for any known influences (e.g., the travel time of air from the front bumper to the methane analyzer).

Then the data are examined for elevated readings and the repeated observations of elevated readings are collected.

Data undergo an additional manual screening for outliers and verification. Where possible, these readings are compared against leak locations known to the utility, as well.

Like software updates to your smartphone, we are continually improving our ability to accurately and appropriately interpret the data we collect. In the summer of 2017, we implemented several updates to make our software more sensitive for detecting natural gas leaks and allow us to more accurately quantify their size and location.

Leak information comes from the data we collected near the time of map release, as analyzed by the version of software used at that time. In addition, as natural gas leaks are repaired and new leaks develop, the number and location of leaks in a city changes over time. Because of all these factors, our leak maps represent a time- and technology-specific snapshot of the city.

Future Goals and Launch

The current industry standard practice of leak surveys is relatively labor intensive, requiring two or more highly-trained drivers plus well-trained walking survey workers.

The types of gas analyzers utilities typically use for leak detection vary widely. While some utilities use older flame ionization detector (FID) systems, others have adopted newer laser-based instruments. The FID analyzers are not able to detect small changes above background methane levels, and thus require a higher minimum methane concentration to identify a leak. The FIDs are also unable to discriminate between methane and other hydrocarbons like gasoline.

The laser-based instruments are more effective in identifying primarily methane even at lower concentrations. Older laser instruments from the 1990s and early 2000s have about 1 part(s) ppm precision and a minimum concentration of about 3 ppm, which is less sensitive and precise compared to the newest instruments used for this project. And even within laser-based systems, there are differences among different aged instruments.

This project takes advantage of improved sensor technology and sophisticated data analysis allowing minimally-trained drivers to gather the needed data for rapid leak detection.

The underlying data used in the development of these maps is now public. Request the data.

Over the next year, the data analysis and map publication will become increasingly automated. EDF will continue to estimate city and/or sub-city leak rates and validate data with utilities. Over the long term, the team hopes to enhance understanding of how this study's techniques can be applied to other vehicle-based environmental observations.

Date updated: July 25, 2017