WunderBlog Archive » Category 6™
Category 6 has moved! See the latest from Dr. Jeff Masters and Bob Henson here.NWS LOWERS ITS VOICE: All-Caps Phaseout Starts in May
NOAA’s two operational supercomputers for weather prediction can carry out 5,000 trillion calculations per second. Until now, though, forecasters at the National Weather Service (NWS) could use only a measly set of 30 characters to translate that prodigous model output into the worded watches, warnings, advisories, and discussions that millions consult each day. A new era of commas, colons, parentheses, and lower-case letters--and maybe even the occasional question mark?--starts in May, when local NWS offices begin converting to mixed-case products. As NOAA put it in a news release on Monday: “LISTEN UP! BEGINNING ON MAY 11, NOAA’S NATIONAL WEATHER SERVICE FORECASTS WILL STOP YELLING AT YOU.”
Figure 1. A NOAA montage illustrates the evolution of communication technology at the National Weather Service.
As a 24/7 operational agency, the NWS is by nature cautious and deliberate when making changes that could affect how its work is disseminated in life-or-death situations. The agency’s shift to mixed case has been even more gradual than one might expect, mainly to ensure that its variegated user base can deal with the newly introduced character set. Many users (including WU--see below) feed NWS products into a wide variety of software.
Testing of the mixed-case format has been unfolding for most of this decade. At the National Hurricane Center, tropical weather outlooks and tropical cyclone discussions have been issued in mixed case beginning with the 2014 season. Four local NWS offices--Kansas City, MO; Louisville, KY; Spokane, WA; and Tallahassee, FL--have served as experimental testbeds since August 2011 for three mixed-case products: area forecast discussions, public information statements, and regional weather summaries. The national changeover in May involves all local offices switching to mixed case for these three products, with watches and warnings going mixed-case by August and most other NWS products by the end of the year.
Figure 2. Staff at the U.S. Weather Bureau office in Washington, D.C., prepare a forecast in July 1943, using data received by teletype from across the nation and plotted on maps. Image credit: Photo by Esther Bubley/Library of Congress, via Circuitous Root.
Figure 3. A Teletype Model 33 with paper tape reader and punch, on display at the National Museum of Computing. Many thousands of these models were sold in the 1960s and 1970s. Image credit: AlisonW/Wikimedia Commons.
A relic goes on the shelf
The practice of using a limited character set for NWS bulletins is a byproduct of the telegraph era. Telegrams were in all caps from the very first telegraph message ("WHAT HATH GOD WROUGHT", 1844]. As far back as the mid-1800s, telegraph machines were able to convert typed messages into Morse code and vice versa. By the early 1900s, these had evolved into hybrid printer/typewriter units called teleprinters. The most successful producer of these was a company founded in 1906 and renamed the Teletype Corporation in 1928, shortly before it was acquired by AT&T. Teletype machines were the standard mode of transmitting weather data when meteorology went through its enormous growth phase during and after World War II. They remained in common use as late as the 1980s, until personal computers became widespread.
The all-caps format of NWS bulletins emerged from the limits of the teletype keyboard, together with standards set by the World Meteorological Organization designed to accommodate nations with widely varying technological capabilities. No capital letters were allowed, and the only punctuation permitted was periods (.), ellipses (…), forward slashes (/), dashes (—), and pluses (+). These conventions went largely unseen by the public until the NWS website was established in the 1990s. Before that point, NWS text products were reformatted by newspapers or read aloud on TV and radio, including NOAA Weather Radio.
Forecasters found many ways to be creative within the old all-caps format, especially in forecast discussions that were originally designed for internal consumption and then opened to the public during the Internet era. You can find a rich set of classics archived in the Forecast Discussion Hall of Fame, including the best all-caps example of “talking like a pirate” you’re likely to encounter on land or sea.
Figure 4. The wide world of 95 characters now available to NWS forecasters to include as part of mixed-case messages.
From the front lines of mixed case
I asked Parks Camp, science and operations officer (SOO) at NWS/Tallahassee, about his office’s experience as a mixed-case testbed. “We received very little feedback one way or the other when we switched to mixed case,” Camp said. “Probably the biggest adjustment was internal. For example, with the area forecast discussion, forecasters have had to adjust how they write, taking advantage of the full suite of punctuation (not just periods and ellipses). When you spend most of your career writing in a particular style, it can take a little time to make the adjustment.”
Ron Miller, the SOO in Spokane, also chimed in: “Going with mixed case took a little getting used to, for both the meteorologists (who weren't used to typing mixed-case products) and the users,” he told me. One radio station commented that the mixed-case bulletins were harder to read and that the upper-case format was standard in the media world. However, says Miller, “In our digital world, upper-case is shouting. As such, many of the NWS upper-case products didn't sit well with web-savvy folks who saw this as poor etiquette. So it was nice to adhere to the societal norm.”
NWS meteorologist Art Thomas, who has been overseeing the national mixed-case changeover, said that about 20 out of 325 comments in a user survey advocated against making the change. “They fell into two broad categories: those that felt it was just easier to read upper case products (matter of personal preference) and those who felt upper-case letters added importance to our products.” Using all-caps will remain an option when forecasters truly want or need to emphasize something, said Thomas.
Jeff Masters on the Weather Underground solution: using hashtables to convert to mixed case
Back in 1998, two of Weather Underground's co-founders, Chris Schwerzler and Alan Steremberg, got tired of seeing the ALL CAPITAL LETTER ADVISORIES OF THE NATIONAL WEATHER SERVICE and decided to implement a clever programming trick to convert the files to mixed upper and lower case. Their solution: put every possible place name that could appear in the advisories into a giant 5 MB hash table, then write software that would search the hash table to recapitalize the text. They got their hands on a geographical database for the U.S. that had the names of every city and geographical feature, and used that as the starting point. For example, here are the entries in the hash table for everything Ann Arbor-related:
Ann Arbor
Ann Arbor Gulch
Ann Arbor Municipal Airport
Ann Arbor Township
It was then a matter of adding a few more items that needed recapitalization, like the rotating list of all hurricane names in the Atlantic. When a storm got its name retired, I would have to add all the potential variants of the new storm name to the hash table; when the hurricane season of 2005 rolled around, and we got into the Greek alphabet, I had to add a whole slew of new entries, like this:
Tropical Storm Alpha
Tropical Depression Alpha
Hurricane Alpha
Alpha
Alpha's
As the years went by, I noted over 1500 phrases that were not being recapitalized properly, and edited the hash table to add/delete these items. There were a few things that could never be properly handled this way: for example, the word "Orange" should be capitalized when it refers to Orange County, but not when it refers to the color orange. Likewise, the abbreviation for Sunday (Sun) should not be capitalized when it refers to our faithful star, the sun. It will be nice to have things in mixed case from the source, instead of trying to do the job ourselves!
Jeff Masters (WU perspective) and Bob Henson
Figure 1. A NOAA montage illustrates the evolution of communication technology at the National Weather Service.
As a 24/7 operational agency, the NWS is by nature cautious and deliberate when making changes that could affect how its work is disseminated in life-or-death situations. The agency’s shift to mixed case has been even more gradual than one might expect, mainly to ensure that its variegated user base can deal with the newly introduced character set. Many users (including WU--see below) feed NWS products into a wide variety of software.
Testing of the mixed-case format has been unfolding for most of this decade. At the National Hurricane Center, tropical weather outlooks and tropical cyclone discussions have been issued in mixed case beginning with the 2014 season. Four local NWS offices--Kansas City, MO; Louisville, KY; Spokane, WA; and Tallahassee, FL--have served as experimental testbeds since August 2011 for three mixed-case products: area forecast discussions, public information statements, and regional weather summaries. The national changeover in May involves all local offices switching to mixed case for these three products, with watches and warnings going mixed-case by August and most other NWS products by the end of the year.
Figure 2. Staff at the U.S. Weather Bureau office in Washington, D.C., prepare a forecast in July 1943, using data received by teletype from across the nation and plotted on maps. Image credit: Photo by Esther Bubley/Library of Congress, via Circuitous Root.
Figure 3. A Teletype Model 33 with paper tape reader and punch, on display at the National Museum of Computing. Many thousands of these models were sold in the 1960s and 1970s. Image credit: AlisonW/Wikimedia Commons.
A relic goes on the shelf
The practice of using a limited character set for NWS bulletins is a byproduct of the telegraph era. Telegrams were in all caps from the very first telegraph message ("WHAT HATH GOD WROUGHT", 1844]. As far back as the mid-1800s, telegraph machines were able to convert typed messages into Morse code and vice versa. By the early 1900s, these had evolved into hybrid printer/typewriter units called teleprinters. The most successful producer of these was a company founded in 1906 and renamed the Teletype Corporation in 1928, shortly before it was acquired by AT&T. Teletype machines were the standard mode of transmitting weather data when meteorology went through its enormous growth phase during and after World War II. They remained in common use as late as the 1980s, until personal computers became widespread.
The all-caps format of NWS bulletins emerged from the limits of the teletype keyboard, together with standards set by the World Meteorological Organization designed to accommodate nations with widely varying technological capabilities. No capital letters were allowed, and the only punctuation permitted was periods (.), ellipses (…), forward slashes (/), dashes (—), and pluses (+). These conventions went largely unseen by the public until the NWS website was established in the 1990s. Before that point, NWS text products were reformatted by newspapers or read aloud on TV and radio, including NOAA Weather Radio.
Forecasters found many ways to be creative within the old all-caps format, especially in forecast discussions that were originally designed for internal consumption and then opened to the public during the Internet era. You can find a rich set of classics archived in the Forecast Discussion Hall of Fame, including the best all-caps example of “talking like a pirate” you’re likely to encounter on land or sea.
Figure 4. The wide world of 95 characters now available to NWS forecasters to include as part of mixed-case messages.
From the front lines of mixed case
I asked Parks Camp, science and operations officer (SOO) at NWS/Tallahassee, about his office’s experience as a mixed-case testbed. “We received very little feedback one way or the other when we switched to mixed case,” Camp said. “Probably the biggest adjustment was internal. For example, with the area forecast discussion, forecasters have had to adjust how they write, taking advantage of the full suite of punctuation (not just periods and ellipses). When you spend most of your career writing in a particular style, it can take a little time to make the adjustment.”
Ron Miller, the SOO in Spokane, also chimed in: “Going with mixed case took a little getting used to, for both the meteorologists (who weren't used to typing mixed-case products) and the users,” he told me. One radio station commented that the mixed-case bulletins were harder to read and that the upper-case format was standard in the media world. However, says Miller, “In our digital world, upper-case is shouting. As such, many of the NWS upper-case products didn't sit well with web-savvy folks who saw this as poor etiquette. So it was nice to adhere to the societal norm.”
NWS meteorologist Art Thomas, who has been overseeing the national mixed-case changeover, said that about 20 out of 325 comments in a user survey advocated against making the change. “They fell into two broad categories: those that felt it was just easier to read upper case products (matter of personal preference) and those who felt upper-case letters added importance to our products.” Using all-caps will remain an option when forecasters truly want or need to emphasize something, said Thomas.
Jeff Masters on the Weather Underground solution: using hashtables to convert to mixed case
Back in 1998, two of Weather Underground's co-founders, Chris Schwerzler and Alan Steremberg, got tired of seeing the ALL CAPITAL LETTER ADVISORIES OF THE NATIONAL WEATHER SERVICE and decided to implement a clever programming trick to convert the files to mixed upper and lower case. Their solution: put every possible place name that could appear in the advisories into a giant 5 MB hash table, then write software that would search the hash table to recapitalize the text. They got their hands on a geographical database for the U.S. that had the names of every city and geographical feature, and used that as the starting point. For example, here are the entries in the hash table for everything Ann Arbor-related:
Ann Arbor
Ann Arbor Gulch
Ann Arbor Municipal Airport
Ann Arbor Township
It was then a matter of adding a few more items that needed recapitalization, like the rotating list of all hurricane names in the Atlantic. When a storm got its name retired, I would have to add all the potential variants of the new storm name to the hash table; when the hurricane season of 2005 rolled around, and we got into the Greek alphabet, I had to add a whole slew of new entries, like this:
Tropical Storm Alpha
Tropical Depression Alpha
Hurricane Alpha
Alpha
Alpha's
As the years went by, I noted over 1500 phrases that were not being recapitalized properly, and edited the hash table to add/delete these items. There were a few things that could never be properly handled this way: for example, the word "Orange" should be capitalized when it refers to Orange County, but not when it refers to the color orange. Likewise, the abbreviation for Sunday (Sun) should not be capitalized when it refers to our faithful star, the sun. It will be nice to have things in mixed case from the source, instead of trying to do the job ourselves!
Jeff Masters (WU perspective) and Bob Henson
The views of the author are his/her own and do not necessarily represent the position of The Weather Company or its parent, IBM.