But during a recent trip to Beijing he was uncharacteristically forthcoming, telling reporters that the US maintained "direct contact" with North Korea.
His candor was met with an agitated response -- not from the media, but from his boss back home.
But the revelation -- and the hope it might have encapsulated -- was quickly shot down by US President Donald Trump, who dismissed his Secretary of State's efforts as "wasting his time."
"Save your energy Rex, we'll do what has to be done!" Trump tweeted from his personal account on Sunday.
But has Tillerson really been wasting his time? How effective can diplomacy be at this juncture of an increasingly tense standoff?
Channels, back channels and academic retreats
For a country that is largely isolated diplomatically, North Korea does have several means to communicate with even its adversaries.
- A key avenue is the New York channel -- via the North Korean mission at the UN headquarters in Manhattan. The main contacts there are Ambassador Pak Song Il and Counselor Kwon Jong Geun, says CNN's Will Ripley, who regularly travels to North Korea. "The channel is always open on some level," he said. It was the avenue through which the US negotiated the release and return of American college student Otto Warmbier, sentenced to 15 years hard labor in North Korea for allegedly attempting to steal a political poster. He spent 17 months in detention and was returned to Ohio in June unresponsive with severe brain damage. He died less than a week after he arrived home.
- Another communication method is within North Korea itself, where the government of Sweden represents US diplomatic interests. Sweden enjoys a particular status in North Korea for being the first Western European nation to establish diplomatic relations with the Hermit Kingdom, and set up its embassy in Pyongyang in 1975. It hosts and sponsors talks, distributes aid that comes in through the UN and the Red Cross, and is a member of the Neutral Nations Supervisory Committee, which was set up to oversee the armistice and "promote trust between the two sides." In May this year the North Koreans allowed the Swedish embassy to pay a consular visit to four Americans in custody.
- Another avenue of contact? Along the de-militarized zone, or DMZ. North Korean military personnel communicate occasionally with duty officers on the other side of the demarcation line, among them South Korean and US military. There are small blue buildings that straddle the north and south with entrances on each side in the village of Panmunjom, located within the Joint Security Area of the DMZ, says Ripley. "It's not a regular occurrence, but talks do occasionally take place there," he said. "Each side would enter through their respective door and sit at the table."
- There is also what is known as the "1.5 Track" talks, where former envoys meet with North Korean diplomats in low key meetings around the world. "They are not publicized and are used to transmit messages back and forth," said David Maxwell, associate director of the Center for Security Studies at the Walsh School for Foreign Service at Georgetown University. "So there's multiple ways to communicate."
Good cop bad cop?
"Clearly it is part of his management style, which seems to be to undermine his people at every turn," said Christopher Hill, who represented the US in negotiations with Pyongyang during the George W. Bush administration.
What can talks achieve?
Even as he confirmed that he'd asked North Korea: "Would you like to talk?" Tillerson acknowledged that it would be an "incremental process."
"You'd be foolish to think you're going to sit down and say: 'OK, done, nuclear weapons gone'," he told reporters in Beijing. "This is going to be a process of engagement with North Korea that will be step-wise."
The closest the US came to an actual agreement with North Korea on its nuclear ambitions was in 1994 during the Clinton administration.
"North Korea has cheated, lied, and stolen and reneged on every agreement," said Maxwell of Georgetown University, who advocates not for a return to negotiations, but a military-to-military relationship that focuses on continued deterrence.
"There's only one solution to this," he said. "The problem is that as long as the Kim family regime exists, its nuclear program will continue, its crimes against humanity conducted against its people will continue. We have to be realistic in that there is nothing we can do diplomatically through coercion or co-opting that will result in North Korea de-nuclearizing. There are no carrots or sticks."