"I am already 80 years old, and fortunately I am now in good health," said the monarch, who turns 83 in December.
"However, when I consider that my fitness level is gradually declining, I am worried that it may become difficult for me to carry out my duties as the symbol of the State with my whole being as I have done until now," he said.
The announcement was considered by many to be a plea by the emperor to Japanese lawmakers, asking them to change the law so that he can step down.
Japan's imperial household law requires the emperor serve in the position until death. If he becomes incapacitated, his successor can act as regent, according to the constitution.
Akihito said that a regent may be appointed, most likely his son Crown Prince Naruhito, to serve in his place if he is seriously ill or incapacitated.
But he stressed that the regent is not an emperor, and the emperor must serve until death.
The message appears to have been received by Prime Minister Shinzo Abe.
Abe said that he will "think very seriously about what we can do" in order to ease the emperor's burden.
This is only the third time a Japanese emperor has spoken to the public in the television and radio era.
Emperor Hirohito announced by radio that Japan lost World War II and Akihito, his son, addressed the nation on TV after the Fukushima nuclear disaster in 2011.
Japanese people stopped in the street, gathering to watch the historic address in public on big outdoor screens.
What does the speech imply?
Jeff Kingston, director of Asian studies at Temple University, believes that Akihito implied in his speech that Japanese lawmakers make changes to imperial household law that would enable him to abdicate.
Kingston noted that the speech also was interesting because Akihito spoke "as a citizen baring his soul and sharing his anguish about not being able to fulfill his duties as symbol of state."
"It was a powerful address," Kingston said, "sincere and from the heart, asking for people's understanding and support ... and 85% do support him."
Abdication rumors
Akihito is Japan's 125th emperor and records claim the imperial line has been unbroken for 14 centuries.
The emperor's eldest son, 56-year-old Naruhito, is next in line to the throne and has already taken on some of his father's duties. In May, the IHA announced Akihito and Empress Michiko, 81, would reduce their public appearances due to old age.
Health problems
In recent years, the emperor has suffered from declining health. Heart surgery and treatment for cancer have taken a toll on his ability to carry out his duties.
"It was some years ago, after my two surgeries that I began to feel a decline in my fitness level because of my advancing age, and I started to think about the pending future, how I should conduct myself should it become difficult for me to carry out my heavy duties in the way I have been doing, and what would be best for the country, for the people, and also for the Imperial Family members who will follow after me," Akhitio said during his speech.
The emperor and empress have long maintained a demanding schedule of more than 250 public meetings per year and 75 annual trips within and outside of Japan, the IHA said in May. But more than 100 of those meetings per year will now be canceled or reassigned to the crown prince.
A Japanese monarch has not stepped down in about 200 years, but abdication is hardly unprecedented. Nearly half of Japan's emperors quit the throne while alive, according to state broadcaster NHK.
Once revered as a living God, the Japanese emperor became a ceremonial figure in Japan's constitutional monarchy after World War II. Occupying forces seized much of the imperial family's wealth and today Parliament controls the household's annual budget and allowances, which total well over $100 million.
CNN's Yoko Wakatsuki, Junko Ogura and Hiroo Saso contributed reporting.