Utterly baffled by a change made by NWS to its reporting

[Edward]

About a week ago, when checking the 3-day history at the NWS webpage for my city--as is my custom on most days-- I noticed that some of the temperatures were not whole numbers (as had always been the case until then), but included tenths of a degree. My initial reaction was, "Great, more precision is always better!" But then, looking more closely, I noticed something very strange indeed. Yes, certain of the temperatures were not whole degrees, but all I saw were .1 and .9, and no other tenths. So, for example, the chart would show 64.1 or 72.9 (or, of course, just plain 76) but never 53.2 or 53.3 or 53.4, etc. This, at first blush-- and all subsequent blushes-- made no sense to me. I observed this for a few days, and the pattern remained exactly the same. Then I checked at the NWS local webpages around the country and found precisely the same situation prevailing at every one I checked-- but only for three day history.

I don't have to explain to people on this site why such a change is peculiar, and I'm hoping someone has an explanation. I sure don't!

 

[Jellyroll]

Their 3-day history is showing the exact number that is stored. It's a function of correlating Celsius to Fahrenheit. I don't know if this answer will piss you off or not, but I am happy to explain in a more technical way if you would like.

 

[Edward]

Hi Jellyroll,

First, it takes a genuine outrage to piss me off, and your answer was only mildly annoying. Just kidding-- your answer was fine but I'm still baffled. Here's why: whether the original reading was Celsius or Fahrenheit, there's no reason in this universe-- or any parallel universe-- for all the posted values to be confined to one-tenth or nine-tenths. Or any reason why the NWS would choose to post only readings that end in .1 or .9 and decide to give us whole numbers whenever it was .2 or .3 or .4 or .5 or .6 or .7 or .8. Would that make sense even to someone certifiably insane? I think not.

If the original readings were Fahrenheit, and in tenths, then all tenths from 0 to 9 should appear, equally represented. If the original readings were Celsius, and converted to Fahrenheit, then whether the Celsius readings were strictly whole numbers or in tenths, there's no reason the conversion to Fahrenheit would result in values ending only in .1 or .9. For example, suppose the Celsius temperature was in whole numbers, say 31 degrees. Then the conversion to Fahrenheit would be 87.8. Ending in a .8, not a .1 or .9. Or if the temperature in Celsius included tenths, and was, for example 31.4, then the Fahrenheit conversion would be 88.52, rounded to 88.5 and again not a .1 or .9! So, these easily-created counter-examples demonstrate that it's totally bizarre for only .1 and .9 to appear in every 3 day history of the NWS for several weeks now! And this madness is apparently the new way 3 day history is appearing at every NWS station around the country, based on my sampling!

If you have a further explanation, Jellyroll-- technical or otherwise-- I'm happy to hear it! Or can someone from the NWS who's reading this inquire among your colleagues and get an answer for me?

 

[Sawmaster]

That does seem odd. I checked my local office (NWSGSP) and couldn't find a 3-day product, but the monthly past data (NOWData) showed temps in full numbers with averages being whole numbers or .5, and departure in wholes or various tenths. Other products showed only whole numbers.

Have you tried contacting your local office with your question? If you go to their homepage you should find an "About" button in the header which has a dropdown menu with a "Contact us" link giving an email addy and phone number. It might take a few days to get a response. If that proves fruitless come back here with a link and or a screenshot of what you're seeing and I'll email the folks here with it; they've always been good in getting me answers when they have the time to do that.

My wild and uneducated guess is that there's something in a software program which is supposed to do 'rounding' of numbers which was intended to show less than a half or more than a half, and the product had to be numeric so this was the result. I do know that they write their own programming at local and national levels so somebody somewhere in the NWS will know what it's all about. HTH

 

[Edward]

Hi Sawmaster, I followed your directions-- which, unlike most directions, proved unerringly accurate-- and now possess an email address which I intend to use shortly.

Regarding your 'wild and uneducated guess': it sounds plausible-- especially when you consider the fact that even when software goes ludicrously wrong, people 'in control' don't notice for quite some time. I had to laugh that it took AI chatbots claiming (something like) 'Carol Burnett led the Nazi armies in World War II' for the CEO to acknowledge that the devices sometimes had a 'hallucination' problem.

 

[Sawmaster]

Please let us know what you find out. I know little of software but I do know that they use Linux, and that all their coding, programming, and data entry is 100% in-house. Or at least it was about ten years ago when I still had friends at the NWS here. I lost touch with them as one moved to another office and the other retired. Both may be retired by now.

 

[Edward]

To Sawmaster, Jellyroll, and all others who are intrigued by this quirky manifestation of human behavior I've brought up: Well, we have another wrinkle in the mystery that provides new clues to what's going on. After the NWS had only posted whole numbers and numbers ending in .1 and .9 for about two weeks on their 3 day history page, when I clicked on that page today-- incidentally this is for New York City but I have every reason to expect that this is true for every 3 day history around the country-- I immediately noticed a significant change. At first, (and I emphasize those words) I thought rationality had finally descended upon people in the NWS and we now would have all temperatures on the 3 day history page reported in whatever tenths were appropriate. But no! We now enter the realm of the truly bizarre! The only additional tenths were .2 and .8! Still no .3, .4, .5, .6 or .7! And still perfect symmetry-- first .1 and .9, now .2 and .8 are added.

So evidently there's some software problem, but that raises an additional question: why would the NWS roll out this new way of reporting temps before it was ready? I have no idea but can only say, thank goodness these same minds were not in control of when the Brooklyn Bridge was opened to the public or people may have plummeted into the East River off the end of the yet-to-be-completed bridge!

 

[Jellyroll]

This is great!!!! I had a whole post typed up in my typical windy format about rounding error and conversions, but this latest twist is truly bizarre as you stated. Not only .1, .9, .2 and .8 but whole number as well.

I have seen the .1 .9 errors in previous lives of Unix programming for Uncle Sam but the .2, .8 and whole numbers would indicate something additional going on. Something I hadn't done was take a look at the script to see if that's where the error lies (to which I hope) or if this is truly bad data. I enjoy a good mystery!

 

[Jellyroll]

This is what I found:
1) I pulled the 3-day history for NYC Central Park and put it in a spreadsheet to play around with. The concern you had with the trailing .1,.2,.8,.9s are exactly what they should be when converting C to F.
2) The temperatures are shown every hour, so for 3 days, that would be 72 temperature readings.
3) The concerning thing I found was that there were only 22 unique temperatures (out of 108 possible for that temperature range). Many were found to be repeating one as many as 8 times.
4) In looking at other local offices, there seems not to be a standard way of presenting the 3-day history. Some only show whole number, some show the temperature every 15 minutes.
5) I did find a similar trends to NYC in other offices for certain locations but even for the same office, not all the locations.
6) That Sawmaster's comments are never wild or uneducated.

My ideas about what possibly could be going on. (I'm going to dive into some FAA topics, so any pilots or ATCs please correct or add too if I provide a wrong take on things). FAA has a standard for temperature reporting to be given in whole number Celsius. These values come from mainly from regional and national airports. It is my assumption and could be totally wrong, that if NWS has multiple temperatures for a certain location, that they would average them for the 3-day history. When the airport temperature is averaged with another weather station the following example could happen where the whole number station virtually blocks out the change from the other station:

Time​	Station 1​	Station 2​	Average 15 Minute Temp​	Hourly Temperature​
:15​	13.6​	14​	13.8​	14.0​
:30​	13.9​	14​	14.0​	​
:45​	14.1​	14​	14.1​	​
:00​	14.3​	14​	14.2​	​

This could account for the repeated numbers I mentioned earlier as well, provided each day was similar in temperature profile. I have no way of factually verifying this because I do not know if they are averaging temperatures and I do not know if they are sampling every 15 minutes, but I do not feel my assumptions are unrealistic either.

Concerning (5: I did not check every NWS office and certainly didn't check every location covered by each office but what I found was that for many locations that were geographically distant from regional and nation airports, this was not the case, and a typical variable string of temperatures were shown. This peculiarity only seems to crop up in the 3-day history, whereas products such as the NOWdata do not show this happening.

Could it be a programming glitch? Yes. Could it be an instrument malfunctioning, Yes Should this not even be an issue and the NWS standardize on whole number reporting for this product? It would be helpful. I would be curious to know the real answer or answers to these issues, but I'm satisfied enough for now and as far as I know the NWS doesn't plan to get into the bridge building business anytime soon.

 

[Edward]

Hey Jellyroll, if it's a matter of converting Celsius to Fahrenheit, although I'm sure some C to F conversions would end in .9 or .1 (or .8 or .2), the central issue remains: what about all the other conversions that obviously wouldn't end that way? Why arbitrarily exclude those? The way you wrote point 1) above, saying, "The concern you had with the trailing .1,.2,.8,.9s are exactly what they should be when converting C to F." puzzles me. There are many temperatures in C that, when converted to F, do not end in .1, .2, .8, or .9.

And what do you make of the fact that after a brief period where the temps ended in all four of the above possibilities, the NWS 3 day history has reverted to the .9 and .1 only endings for the past three days! Not a .2 or .8 to be found! Their computer is having a nervous breakdown-- and in public! How embarrassing.

 

[Jellyroll]

Edward,
From my best reading its server / javascript issues. It appears they are in the process of migrating and reorganizing their systems and they are aware of this issue among many. They actual mention issues with the 10th's reading for the 3-day history. I apologize for all the rabbit trails and thinking it was an actual programing issue. That said, they may not be aware of this specific issue and have this email for feedback: [email protected]

Links to information regarding their migration.

Changes to NWS Web Pages as of June 4

Changes to NWS Forecast & Marine Pages as of April 30

 

[Edward]

Hey Jellyroll!

Jellyroll says, "I apologize for all the rabbit trails...."

No! Don't apologize! Life is at its best-- and most interesting!-- when we go down the most absurd of rabbit holes. I've recently been laid low by a surprise case of rhabdomyolysis, and I knew I was finally recovering-- and back to my usual self-- when I heard that the tongue of a blue whale weighs as much as an elephant, and I was curious enough to pursue it! My first thought was "As much as an elephant? A baby elephant perhaps, but surely not anything larger." But no, the tongue of a blue whale weighs as much as as an adult female African elephant-- 10,000 to 15,000 lbs.! The fact that I was once again able to be astonished by such things means a lot to me.

When you're ill you get a radically different perspective on life, but not a better perspective.

 

[Sawmaster]

Hey Jellyroll!

Jellyroll says, "I apologize for all the rabbit trails...."

No! Don't apologize! Life is at its best-- and most interesting!-- when we go down the most absurd of rabbit holes. I've recently been laid low by a surprise case of rhabdomyolysis, and I knew I was finally recovering-- and back to my usual self-- when I heard that the tongue of a blue whale weighs as much as an elephant, and I was curious enough to pursue it! My first thought was "As much as an elephant? A baby elephant perhaps, but surely not anything larger." But no, the tongue of a blue whale weighs as much as as an adult female African elephant-- 10,000 to 15,000 lbs.! The fact that I was once again able to be astonished by such things means a lot to me.

When you're ill you get a radically different perspective on life, but not a better perspective.

Glad you're doing better Edward. If you ever want to go down the rabbit hole of tornadoes, this forum is as good as it gets. I found it after about 5 years of rabbit-holing my own self.

Be well!

 

[Edward]

Hi Sawmaster, thanks for the sympathetic words. The worst thing about rhabdomyolysis (apart from the symptoms) is that when you tell someone you have it, they look at you with a blank stare. There was one exception though: An acquaintance of mine yelled across a room, "Hey Ed, I haven't seen you in a while, where have you been?" I replied, "I was knocked out by rhabdomyolysis." At hearing that, his face took on an expression that was equal parts puzzlement and concern. He walked over to me and said, "What??? Rabbi Myerwitz knocked you out? Who the heck is he and why would a man of God do a thing like that?" By the time I stopped laughing long enough to explain, any sympathy he might have had for me was gone!


Regarding tornadoes: When I was a teenager, I hated when things followed the normal routine, and always rooted for disruption. Living in New York City, the most likely route to what I craved was a mammoth snowstorm, preferably a blizzard-- schools would be closed, roads impassable, and if you merely stepped outside at its height, you were regarded as kin of Davy Crockett. But for all its fury, the worst blizzards generally were more dramatic than actually damaging, so I didn’t feel guilty for always wishing for one. Of course, tornadoes are seldom seen in NYC, but if they were as common here as they are in Tornado Alley, I would have been very conflicted as a teenager: sure, they were excitingly disruptive, but in a way that scared me more than thrilled me and did real damage too great to ignore. But they do fascinate me-- as long as I’m far enough away to never hear the freight train that people always say they sound like!