In these days of internet disruptions of complex systems like medical care and even elections, our obsession with data security is swelling. We’re all too aware of how signals from myriad sources can help us construct patterns of human behavior and make plans to take advantage of that behavior. Although it may be no comfort to know this, such pattern-building has been hugely significant in history.
Seventy-five years ago, in the Battle of Midway on June 4, 1942, a narrowly won triumph of signals intelligence enabled the U.S. Navy to reverse the strategic balance of the Pacific war, and gain a chance to win it much sooner than expected. Indeed, to everyone’s amazement and overwhelming relief, the Japanese surrender of 1945 came less than four months after Germany’s.
American code-breakers were able, just barely, to discern a Japanese plan to lure the Pacific fleet into an ambush, and to trump this with an ambush of their own. On the vast chessboard of an ocean, the U.S. Navy was able to sink four of the big carriers Japan used as their spearhead of conquest, putting Japan suddenly—and permanently—on the defensive.
The crucial work was done in a cold, fetid Pearl Harbor basement called the Dungeon. There, a disheveled crew of desperate code-breakers, often working 20-hour shifts, were led by an unpopular 43-year-old Commander Joseph Rochefort. Grasping fragments of Japanese messages, they used a risky ruse and banks of punched-card calculating machines to connect the dots. They established that Japan’s Admiral Isoroku Yamamoto, the victor of Pearl Harbor, was sending his triumphant force to Midway, west of Hawaii. U.S. Pacific Fleet Commander Chester W. Nimitz could dare to put his slim forces in the right place for a decisive counterblow. Then, by the narrowest of margins, around 10:20 a.m. on the Fourth of June, Nimitz could deliver it via dive bombers to Japanese carrier decks just when they were laced with fuel lines and dotted with piles of explosive shells.
In a note posted last year by the National Security Agency, Patrick Weadon described the code-breaking task as “daunting.” Rochefort and his comrades broke into the Japanese navy’s operational code of 45,000 five-digit numbers. Over many months of mathematical analysis, they stripped off a super-encoding “additive” and tracked the usage of the numbers over time to get at which word or character in Japanese the five-digit groups represented.
In the desperate conditions of early 1942, Rochefort’s team had to win the confidence of Nimitz and his staff as they began “reading fragments” of the code messages, “albeit with many words missing,” as the English military historian Max Hastings described it last year. On March 2, 1942, they accurately predicted a Japanese air raid on Hawaii on March 4, and did it again with a raid on Midway. By May 9, Rochefort could predict that the Japanese fleet would sail for a major operation on May 21, but not where. Rival sigint analysts in Washington predicted it would be on Johnston Island, but the denizens of the dungeon zeroed in on Midway and convinced Nimitz.
The stunning events near U.S.-held Midway Island came only six months after a horrific failure of intelligence left the U.S. Pacific fleet at Pearl Harbor open to devastation by Japanese carrier planes on Dec. 7, 1941. High officers in Washington, including Army Chief George C. Marshall, deduced from a decoded message to the Japanese embassy in Washington, hours before the attack, that it was imminent, and then botched sending the warning to commanders on the spot.
Today, we often forget how sweeping and disastrous the early triumphs of the dictators were in World War II. How imminent the collapse of freedom seemed when Europe and Asia were swiftly absorbed into totalitarian empires. In the months after Pearl Harbor, Admiral Chuichi Nagumo’s attack-carrier group raged freely over the Pacific and Indian Oceans. Just a month before Midway, the last American and Filipino troops on Corregidor at the mouth of Manila Bay surrendered to the Japanese.
The sting of defeat after defeat was forcing U.S. and allied leaders into a new discipline, one that was usually repellent to the men of action who commanded the military. Strange people like Rochefort, who often wore a red smoking jacket to stay warm in his Dungeon, were struggling to pull intelligence, in the form of blocks of numbers, out of the air and build patterns out of them. They had to translate that fragmentary stream of numbers and make sense of it. They had to rigorously analyze what they found, and then they had an even harder task: explaining their findings so lucidly that decision-makers would act on them.
This three-step task deserves recognition alongside the other unfamiliar tools that the Allies in World War II were forced to harness, so as to resist and then crush the rampant dictators. These included science-based new technology like radar, the proximity fuse, penicillin, and the atomic bomb—along with America’s productive muscle.
In the spring of 1942, the competing group in Washington and Rochefort’s in Hawaii had been picking their way into Japan’s “JN-25” naval code, but Rochefort’s charges, serving Nimitz, drew clearer conclusions. Perhaps the pressure on them was greater than back near Washington. Nimitz’s resources and options were few. The code-breakers had to give him enough certainty to make a risky, come-from-behind bet.
To clinch Midway as the Japanese focus, they sent a message via a submarine cable from Honolulu to the lonely island 1,000 miles west, instructing those on Midway to complain openly in a radio broadcast that the Japanese could pick up about problems distilling enough water. They guessed that the Japanese, instead of fearing “disinformation,” would consider such a mundane, but urgent, message a plausible excuse for transmitting “in the clear,” uncoded. The aim was to confirm that “AF,” a term Rochefort remembered from the captured data-stream two months earlier, designated Midway. Shortly after, the Japanese took the bait and referred in one of their own messages to a water shortage at AF.
The effects of the code-breaking feat were massive, but Rochefort received almost none of the recognition he deserved. Nimitz recommended him for the Distinguished Service Medal, but Rochefort’s sigint rivals in Washington quashed the award and Rochefort soon found himself commanding a floating dry dock in San Francisco. Toward the end of his life, he was a consultant on movies about Pearl Harbor and Midway. He died, still without the DSM, on July 20, 1976, the day that America’s Viking craft landed on Mars. The medal was only conferred posthumously in 1985, thanks to efforts by Nimitz’s former chief of intelligence. In 2000, the National Security Agency added Rochefort to its honor roll in Cryptography. Historian Hastings observed, “Rochefort deserves to be remembered as a man who changed history.”
[Editor’s note: This is the 13th of a series of notes about major anniversaries in innovation and what they teach us. Earlier notes have highlighted the U2 spy plane’s first flight in 1956 and the first communications satellites “on station” above the equator in 1965. You’re invited to suggest other Milestones of Innovation for our forum].
Further reading:
Max Hastings, The Secret War, Spies, Ciphers, and Guerillas, HarperCollins 2016.
Eric Larrabee, Commander in Chief: Franklin Delano Roosevelt, His Lieutenants, and Their War, Simon and Schuster, 1987.
Edwin McDowell, “Officer Who Broke Japanese War Codes Gets Belated Honor,” New York Times, Nov. 17, 1985.
