How Accurate are the RSEI Data?

The Risk-Screening Environmental Indicators (RSEI) of the U.S. Environmental Protection Agency provide a screening measure of risks to human health, building on data on releases of toxic chemicals into the environment from the Toxics Release Inventory (TRI). On the RSEI website, the EPA characterizes the database as a tool “to identify risk-related situations of high potential concern, and which warrant further evaluation.”

In addition to the mass (pounds) of releases reported in the TRI, the RSEI data incorporate information on toxicity, fate and transport, and population densities. The RSEI data thereby provide a more robust measure of human health risks than the simple mass of releases reported by the TRI. But the RSEI data are not perfect – no data are.

Potential sources of error

1. Incorrect data submitted by facilities.  Facilities sometimes provide incorrect data on pounds of releases on the ‘Form R’ reports that they submit annually for the EPA’s Toxics Release Inventory (TRI). These data are used to construct the RSEI measures for these facilities. In several cases, firms have contacted PERI to explain that they incorrectly reported numbers that were substantially higher than the true values. Where these new numbers have been reported to and accepted by the EPA, we are able to revise RSEI scores (on the assumption of a linear relation between pounds released of a specific chemical and that chemical’s contribution to the facility’s RSEI score). In principle, facilities also could error on the side of underreporting true values. No firms have contacted us to report mistakes in this direction. But the Environmental Integrity Project has argued (in this .pdf document) that there is a systematic bias toward underreporting in the TRI data.

2. Imprecise data submitted by facilities.  For chemicals other than persistent and  bioaccumulative toxics (PBTs), firms can choose to report a range of pounds released (1-10, 11-499, 500-999) rather than the precise quantity, as long as the release is less than 1,000 pounds. In the 2005 air release data, such reporting accounted for approximately 13% of non-zero release reports (6% were reported as 1-10 pounds, 6% as 11-499 pounds, and 1% as 500-999 pounds). In these cases, the EPA uses the midpoint of the range (5, 250, and 750 pounds, respectively) to calculate RSEI scores. If the chemicals in question have large toxicity weights, the resulting imprecision could substantially affect the facility’s RSEI score. More precise data could change the results in either direction, depending on whether the true value is higher or lower than the range midpoint.

3. Inadequate information on stack heights.  Where facility-specific information on stack heights is not available to EPA, the RSEI uses estimated stack heights based, for example, on industrial-sector averages. Insofar as actual stack heights differ from those used in the EPA’s calculations, this may affect the final scores. In general, higher-than-average stack heights would be expected to result in lower RSEI scores by virtue of ‘solution by dilution.’

4. Toxicity weights for chemical groups.  In some cases, TRI groups together several related chemicals or species of the same chemical, which are reported as a category (e.g., chromium compounds, diisocynates) rather than separately. In such cases, the RSEI assigns a toxicity weight to the group as a whole. The toxicity of individual compounds within the group may vary, however. Depending on which compound is used as the basis for the RSEI toxicity weight, and which compounds are released by the facility, the resulting RSEI score may overstate or understate true risks.

5. Lack of toxicity weights for some chemicals. The EPA has not yet assigned toxicity weights to all chemicals in the TRI database. In version 2.1.5 of the RSEI data, 412 of the 595 chemicals and chemical groups in the TRI database have toxicity weights and thus are included in calculating the RSEI scores. In 2005, the chemicals for which toxicity weights were not available accounted for roughly one percent of total pounds of TRI air releases nationwide. If these chemicals are highly toxic, or if they constitute a substantial fraction of the releases at a specific facility, their omission could significantly reduce RSEI scores.

6. Geographical limits of modeled dispersion.  In the case of facilities with tall stack heights, high exit velocities, and strong prevailing winds, the fact that the RSEI air-release model only accounts for impacts within an area that extends 50 kilometers in each direction from the one square kilometer cell that contains the facility (i.e., within a territory measuring 101 km x 101 km, yielding a total area of 10,201 square kilometers) means that the RSEI score misses impacts beyond this area. This problem may be most relevant in the case of electric utilities.

Ways to improve RSEI accuracy

The first three problems – incorrect data on the mass of releases, imprecise data on these releases, and inadequate information on stack heights – can best be addressed by the firms themselves. Greater public visibility for TRI and RSEI data will strengthen their incentives to submit accurate data and to submit timely revisions in case of erroneous reporting. This incentive is asymmetric, however: firms have stronger motives to correct over-reporting than under-reporting. How to strengthen incentives for accurate and precise reporting more symmetrically is a question worth further exploration.

The fourth problem – the use of toxicity weights for chemical groups – could be addressed by EPA modifications of TRI reporting requirements to provide more detailed breakdowns within important categories, including those with high toxicity weights in the RSEI. The RSEI project has begun to address the problem in the case of chromium compounds by assuming the industrial sector average share of highly toxic hexavalent chromium among total chromium releases as the basis of RSEI modeling a given chromium release.

The fifth problem – lack of toxicity weights for some chemicals – ultimately must be addressed by further toxicological research to provide the necessary information. In the meantime, the accuracy of the RSEI score might be improved by assigning to these chemicals a weight equal to the average toxicity weight for all other chemicals (rather  than their current implicit weight of zero).

The sixth problem – geographical limits of modeled dispersion – could be addressed by modifying the RSEI air-release model to extend the range of impacts beyond the current 50-km limit.