Enhanced Fujita Ratings Debate Thread

[Grand Poo Bah]

If we stick with Empirical Approach to Evaluating the Tornado Fragility of Residential Structures for now we can examine some interesting implications.

The assumptions behind this theoretical that the treefall method used to independently estimate the wind in the paper is accurate (different treefall models produce different results) and that the distribution shown by the fragility curves in the paper are representative (whereas the low-bound biased approach of EF-scale assessments effectively assumes they're more left skewed).

What I've done is draw a dashed line for a theoretical 70 m/s (157 mph) tornado, and placed arrows over 4 DoDs (6, 7, 8 and 9) showing their ranges in the Enhanced Fujita Scale, produced by expert elicitation mainly based on engineering analysis. The vertical axis is the probability of a DoD for the given windspeed:

View attachment 54580

It can be seen that the DoDs have roughly the following probabilities:
DoD 6: 29%
DoD 7: 18%
DoD 8: 17%
DoD 9: 22%
Below DoD 6: 14%

The most likely DoD to be observed is 6, even though the windspeed lies outside the upper bound for that DoD. Only two DoDs are capable of producing an estimate that includes the true windspeed (8 and 9). Therefore, if our theoretical tornado was to encounter a single house where we otherwise know nothing about the structure except that it conforms to these fragility curves, an EF-scale analysis has a better than even chance of definitely underestimating the windspeed, as windspeed lies outside the upper bound for DoDs 7 or lower. Even the remaining two DoDs have enough room to produce underestimates.

Excellent work here. This is a really useful visual. I'd love to see it paired with empirical research that shows the average wind speed assigned to each DOD. If we looked at this year alone we'd see about 85% of all DOD 9s below your 157 mph line, with Enid being the only exception. Is that why you chose this windspeed to represent your example?

 

[Grand Poo Bah]

If we stick with Empirical Approach to Evaluating the Tornado Fragility of Residential Structures for now we can examine some interesting implications.

The assumptions behind this theoretical that the treefall method used to independently estimate the wind in the paper is accurate (different treefall models produce different results) and that the distribution shown by the fragility curves in the paper are representative (whereas the low-bound biased approach of EF-scale assessments effectively assumes they're more left skewed).

What I've done is draw a dashed line for a theoretical 70 m/s (157 mph) tornado, and placed arrows over 4 DoDs (6, 7, 8 and 9) showing their ranges in the Enhanced Fujita Scale, produced by expert elicitation mainly based on engineering analysis. The vertical axis is the probability of a DoD for the given windspeed:

View attachment 54580

It can be seen that the DoDs have roughly the following probabilities:
DoD 6: 29%
DoD 7: 18%
DoD 8: 17%
DoD 9: 22%
Below DoD 6: 14%

The most likely DoD to be observed is 6, even though the windspeed lies outside the upper bound for that DoD. Only two DoDs are capable of producing an estimate that includes the true windspeed (8 and 9). Therefore, if our theoretical tornado was to encounter a single house where we otherwise know nothing about the structure except that it conforms to these fragility curves, an EF-scale analysis has a better than even chance of definitely underestimating the windspeed, as windspeed lies outside the upper bound for DoDs 7 or lower. Even the remaining two DoDs have enough room to produce underestimates.

It's also worth noting DOD 10 would have a 0% chance of producing this wind speed estimate, but a statistically significant amount of homes are given those ratings anyways. (Ie. Lake City and Matador). Unfortunately, a lot of DOD 10s are given the label of DOD 9 so lower wind speeds can be assigned, which drastically increases the difficulty of empirical analysis in that range.

 

[Grand Poo Bah]

I’ve read this paper over a few times now and finally it’s clearer to me. It was a bit hard for me to decipher, but basically it seems like scientific evidence of something I have been saying for a long time, which is:

“The windspeed at which something COULD potentially begin to happen, is very different from the windspeed at which something TENDS TO happen”

For example, tree damage. The EF scale currently has total stubbing and debarking occurring largely in the EF2 to EF3 range. In real life though, actual case studies that compare tree damage to nearby structural damage, computer based physics modeling, and general statistical correlation from my personal observations over the years all suggest that total stubbing and debarking has a much stronger correlation with the EF4+ range. Now can really severe stubbing and debarking happen in the EF2 to EF3 range? Yes, but it’s quite rare and the only examples are cherry picked outliers. Thankfully the concept of severe debarking/stubbing as an EF4 indicator is slowly catching on, but you get my point, and I’m sure the same sentiment can be applied to other DIs as well. The scale and its application operates on a “windspeed at which this degree of damage could begin to happen” basis rather than a “windspeed at which this usually happens” basis. If the objective is finding the most likely actual max windspeed of the tornado based on damage, this approach isn’t favorable.

But with this said, at this stage I don’t really care about the wind speed estimates handed out in actual surveys themselves, because I’m not at all confident that they are genuinely accurate. They are simply a safe, conservative estimate rather than an accurate reading of the tornado’s actual intensity. Instead, I more prefer to rate and discuss tornadoes via a spectrum of EF-scale rankings rather than trying to assign an accurate windspeed estimate. For example, I prefer to simply say “low-end EF4 tornado” rather than “170 MPH tornado” because I’m not so sure that 170 MPH tends to be the most likely maximum real life windspeed for any given tornado that produces that kind of damage. When I do use numbers and wind speeds, I’m actually more just using them as placeholders to convey where the tornado was within the spectrum of an EF scale ranking (high-end, low-end, etc) rather than actually believing that this was the actual maximum windspeed.

But yeah overall, the paper seems to back up my thinking on this topic: EF scale windspeed estimates that get handed out are the lowest conceivable failure point of the DIs, but are conveyed to the public as the tornado’s maximum windspeed, when in reality these can be and often are two very different things.

I'm one that cares far more about accurate windspeeds than damage classification, but I'm mostly resigned to the fact that until we get consistent classification across WFOs, accurate wind speeds are much more difficult to achieve across the entire tornado database.

 

[slenker]

Looking back... I can understand the reason behind them not rating Enid an EF5. But why not use the results the results we saw in the models to at least upgrade the tornado to a 190-195 EF4? Stating it was "very highly likely EF5 intensity" strongly suggests that it was straddling the line between the categories, so in my eyes, the conservative option would be to be just underneath the threshold; it's going beyond conservative to maintain 180 EF4 here. I think that's my biggest gripe with the rating, there was no change to the peak estimated windspeed even after they went through all the trouble of modeling the treefall and RV throw. The results certainly would be in support of such a thing...

 

[buckeye05]

Looking back... I can understand the reason behind them not rating Enid an EF5. But why not use the results the results we saw in the models to at least upgrade the tornado to a 190-195 EF4? Stating it was "very highly likely EF5 intensity" strongly suggests that it was straddling the line between the categories, so in my eyes, the conservative option would be to be just underneath the threshold; it's going beyond conservative to maintain 180 EF4 here. I think that's my biggest gripe with the rating, there was no change to the peak estimated windspeed even after they went through all the trouble of modeling the treefall and RV throw. The results certainly would be in support of such a thing...

Given the fact that they changed the Shawnee wind speed estimate many years after the fact, the door is still open. They really should, as it was obviously a high-end event, but I’m not holding my breath.

 

[Grand Poo Bah]

Looking back... I can understand the reason behind them not rating Enid an EF5. But why not use the results the results we saw in the models to at least upgrade the tornado to a 190-195 EF4? Stating it was "very highly likely EF5 intensity" strongly suggests that it was straddling the line between the categories, so in my eyes, the conservative option would be to be just underneath the threshold; it's going beyond conservative to maintain 180 EF4 here. I think that's my biggest gripe with the rating, there was no change to the peak estimated windspeed even after they went through all the trouble of modeling the treefall and RV throw. The results certainly would be in support of such a thing...

I remember one or two years ago, one of the big talking points in this thread was the total disregard of contextuals in tornado ratings, to the point it was majority opinion they were basically unofficially "outlawed". I think this is still very clearly the policy at the vast majority of WFOs. Regardless of how much public discourse and scientific literature has shifted its focus to contextual evidence, the NWS still probably wants to stay consistent with the "established" ratings practices, even if they're egregiously unjustified. It's a textbook sunk cost fallacy. "It's been this way for 12 years, so it has to continue to stay this way until it can all be fixed at once" (which is an impossible goal).

The maximum damage to a structure was determined to be 180 mph. No amount of contextual evidence can change that, and if it could it would have to be such overwhelming evidence it was beyond the shadow of any doubt for everyone involved. If there's one thing current practices are good at it's sucking all of the energy and enthusiasm out of the scientific community. With how crappy we feel about this process getting fixed in this thread, I can't even imagine how the scientists feel. Hundreds of combined hours from a dozen different people likely went into the Enid analysis.

All of this is assuming the Enid rating is finalized, which I don't think has been 100% confirmed. If the rating is ultimately upgraded it would be an ENORMOUS step in the right direction.

 

[buckeye05]

The thing is though, Enid WAS a context based rating. The highest rated DI was a barely anchored old farmhouse that almost certainly would not have deserved an EF4 rating on its own, and the DAT cites contextual factors as the basis for the intensity estimate at that exact damage point. The fact that this irrefutably proves that an office is allowed to hand out a violent rating based on violent contextual damage OVERRIDING clearly poor construction, sets a great precedent.

While there’s no such thing as tornado case law, any time anyone tries to say something like “Oh well you can’t use things like ground scouring, vegetation damage, and lofted heavy objects to upgrade a rating to EF4+ when the house is poorly built, because there are no official DIs for two out of those three things, and the construction quality is the primary thing that matters scientifically”, you can now call full BS on that with full confidence. Why? Because you can cite that recent damage point, especially given that NWS Norman, one of the most knowledgeable and trusted survey teams in the country, are the ones who made the call. It’s a small victory, but it’s something that is going to be incredibly useful in any debate against any dismissive contextual damage naysayers throughout the lead up to 2030, especially when combined with the fact that like half of the half of the EF4 damage points were just debarked/stubbed or root balled trees! Trust me, when anyone is dismissive of either of those two things, we’re going to love having the Enid survey to cite in the coming years.

Now the part that sucks is the windspeed estimate. It’s too low. There is now overwhelming evidence that this was a top of the scale tornado intensity wise, and perhaps more importantly we don’t want people associating the mid-EF4 range with contextual damage that looks like that. Yes the soil composition in that area made it look a little more dramatic, but there were plenty of overlapping contextual hallmarks to give full confidence that it was a very high-end tornado. It’s not like it was JUST the ground scouring alone, and it’s clear that grass was actually fully removed in some areas, not just stained by dirt. That’s why even if we don’t get it bumped up to EF5, getting it into the 190s would still be a big win. But another silver lining is that you can extrapolate this: if extreme contextual damage can get a frail old farmhouse up to 180 MPH EF4, just imagine what it could get a modern, anchor bolted home with a poured concrete foundation up to.

 

[buckeye05]

Also another point semi-related to the last part of my previous post. The other day I was asking myself, “what is possibly the most important precedent that could be set in the next few years survey/EF scale application wise?”, and I think I have my answer. It unfortunately would likely coincide with something awful happening, but I think the answer to that question is a regular old house damage-based EF5. More specifically, one based on just some regular bolted-down suburban homes being slabbed amid extreme contextual damage. Because at the end of the day, that’s all it should take, and that’s where the bar is supposed to be. But I worry that the bar instead has drifted upward over the years in surveyors minds, and the bar is now at slabbed reinforced masonry houses or gigantic mansions.

Last year, Jim LaDue said in his presentation “It doesn’t have to be a fortress”, and expressed that it simply has to be a reasonably well-built modern home when it comes to defining EF5 house damage. Those are powerful words, but the unfortunate reality is I think we would have to see real world proof of that for it to really sink in. I genuinely hope this post doesn’t foreshadow something terrible happening, but it’s been so long, I think that kind of awful scenario is the only one that would bring the concept of EF5 house damage back into the realm of attainability, as grim as that is.