"The security situation around our country is getting increasingly severe," Abe said, who was addressing the Japanese parliament's diplomacy and defense committee.
"We have just talked about Syria. There is a possibility that North Korea already has a capability to put sarin on warheads to strike the ground," he said.
Abe did not provide any evidence why he felt North Korea had the capability to equip missiles with chemical weapons.
Chemical weapons capability
According to South Korea's 2016 Defense White Paper, North Korea has been developing chemical weapons since 1980.
The paper estimates, as of that year, North Korea had 2,500 to 5,000 tons of chemical weapons, including anthrax, smallpox and plague.
The report said that stockpile could be deployed "by modifying a variety of conventional munitions, including artillery and ballistic missiles."
While analysts have been skeptical of North Korean claims about its intercontinental ballistic missiles capabilities, the country possesses a substantial arsenal of conventional missiles capable of striking South Korea and Japan.
Tong Zhao, an analyst with the Carnegie-Tsinghua Center for Global Policy, told CNN that "it appears technically possible to put chemical weapon agents including nerve agents such as VX or sarin in the warhead of a ballistic missile."
Sarin attacks
Japan is one of only a handful of countries which have experienced a sarin gas attack.
On April 4, more than 80 people were killed and many more injured in a chemical attack in Idlib province, which the US and its allies blame on the Syrian regime.
"Innocent citizens, including babies and children, fell victim in Syria," Abe said.
"As the security situation is getting increasingly severe around our country, we should strengthen the deterrence of the Japan-US alliance based on an appropriate division of roles between Japan and the United States, and continue to study what we must do in order to protect the lives and the assets of the Japanese people."
Risk to Japan
While much attention is paid to Pyongyang's threats of striking the US mainland with intercontinental ballistic missiles, South Korea and Japan are the most at risk of North Korean attack.
"Ballistic missiles could strike US military positions beyond the Korean Peninsula, specifically in Japan," the report said.
"Whatever the targets, Pyongyang's existing ballistic missile stockpile could easily deliver approximately one kiloton (1,000 metric tons) of high explosives, as well as other non conventional munitions -- chemical, biological or even nuclear."
CNN's Yazhou Sun, Jung-eun Kim and Pamela Boykoff contributed reporting.