How reliably do arborists judge decay and stem failure? Inter-assessor consistency studies
Studies of credentialed arborists rating the same trees find wide spread in risk ratings and in the work recommended. In a 2017 study the assessor was four times more likely than the tree to shape the final rating.
Checked against the sources at the bottom of this page on October 15, 2026. Rules, fees and pay change: the source has the last word.
This page draws on two AUF papers and a set of conference slides. It is not training and does not teach the work. The research comes from the University of Florida team of Ryan Klein and Andrew Koeser, and it appears in Arboriculture & Urban Forestry (AUF), ISA's journal.
The assessor and the tree
The 2019 AUF literature review by Klein and colleagues screened about 1,000 articles and kept 150. It found significant variability in the final determinations of risk assessors and of arborists using the common methods. The authors read this as inconsistency that best management practices and training programs have not yet resolved. They add that personal bias probably explains some of it.
The 2023 AUF paper, discussed below, summarises an earlier study by Koeser and Smiley (2017). In it nearly 300 arborists did basic (Level 2) assessments of three trees. The assessor was four times more likely to affect the final risk rating than the characteristics of the tree. Arborists with experience and credentials were four times more likely to recommend retaining and monitoring a tree rather than removing it than those without. Trained assessors also gave lower likelihood of impact, lower likelihood of failure and lower final ratings.
The 2023 reproducibility paper
Klein, Koeser and co-authors published "Evaluating the Reproducibility of Tree Risk Assessment Ratings Across Commonly Used Methods" in AUF in November 2023. They sent videos of urban trees to arborists and asked for assessments of three trees each. Of 450 arborists chosen from public directories, 60 replied, a 13.3 percent response rate. Of those, 28 held the ISA Tree Risk Assessment Qualification (TRAQ), 21 held Quantified Tree Risk Assessment (QTRA) training, and 11 had neither.
| Rating | TRAQ credential (82 assessments) | ISA BMP, no TRAQ (32 assessments) |
|---|---|---|
| Low | 14% | 31% |
| Moderate | 38% | 31% |
| High | 43% | 25% |
| Extreme | 5% | 13% |
The QTRA group used different terms, so the paper makes no direct comparison. Of 62 QTRA assessments, most were broadly acceptable, a risk of harm of 1 in 1,000,000 or less. The rest split nearly evenly between tolerable at 24 percent and unacceptable at 23 percent.
Tests of variability found no method more reproducible than another, with a lowest P-value of 0.251. The mitigation recommended did differ. Arborists without risk assessment training recommended removal half the time, more than the QTRA group did (P = 0.040). QTRA-trained arborists were more likely than the TRAQ group to recommend monitoring (P = 0.037).
The study also asked the ISA BMP and TRAQ arborists to rate likelihood of failure over one year and over three years. Ratings rose with the longer timeframe, but their variability did not (lowest P-value 0.161).
Decay and levels of assessment
Slides by Andrew Koeser of the University of Florida, hosted by the Arboricultural Association, report two further experiments. In the first, 153 southern live oaks and 86 laurel oaks were checked for internal decay. For some species a basic visual assessment by a trained arborist with minimal experience agreed well with a resistance recording drill.
In the second, 70 arborists assessed five trees at several levels, from limited visual to advanced. Raising the level of assessment often made the ratings more variable, not less. Ratings were highest after basic visual and mallet assessment, similar to the drill. They tended to be lower after limited visual assessment and after sonic tomography. The slides conclude that equipment can give precise estimates of decay and occupancy, but without defendable thresholds or decision rules risk assessments stay variable.
The slides also cover the target. In a video study, risk ratings correlated with actual occupancy (r = 0.29), and the correlation rose to 0.62 once the arborists saw the traffic data. The time of day at the site changed the ratings (P below 0.001).
Limits of the evidence
The 2023 sample was small, with 60 replies, and assessors rated videos, not trees standing in front of them. The slides are a conference presentation and print few numbers. Levels of assessment are compared in the guide to tree risk assessment levels.
Sources
- Klein, Koeser and others, Evaluating the Reproducibility of Tree Risk Assessment Ratings Across Commonly Used Methods, Arboriculture & Urban Forestry 49(6), November 2023, read October 15, 2026
- Klein, Koeser, Hauer, Hansen and Escobedo, Risk Assessment and Risk Perception of Trees: A Review of Literature Relating to Arboriculture and Urban Forestry, Arboriculture & Urban Forestry 45(1), January 2019, read October 15, 2026
- Andrew Koeser, Assessment of Likelihood of Failure Using Limited Visual, Basic, and Advanced Assessment Techniques, slides hosted by the Arboricultural Association, read October 15, 2026