(CNN) -- Calling the controversy over Bowe Bergdahl's release from Taliban captors "whipped up in Washington," President Barack Obama said Thursday he doesn't apologize for swapping the captured Army sergeant for five Guantanamo Bay detainees.
"I make no apologies for making sure we get a young man back to his parents," Obama told reporters in Belgium at the end of the G7 summit.
The issue has roiled the political debate, with conservative critics calling the price for Bergdahl's release too high and politicians in both parties complaining that Obama failed to notify Congress of the exchange ahead of time, as required by law.
Obama repeated his stance that he acted with legal authority to seize what might have been the last good chance to get Bergdahl out alive.
"We had a prisoner of war whose health had deteriorated and ... we saw an opportunity and we seized it, and I make no apologies" for that, the President said.
Asked if he was surprised at the backlash, Obama said, "I'm never surprised by controversies that are whipped up in Washington," calling such a reaction "par for the course."
He also reiterated what he called a time-honored U.S. principle that "we do not leave anybody wearing the American uniform behind."
Bergdahl is resting and showing signs of improvement as he recovers at Landstuhl Regional Medical Center in Germany, according to Pentagon spokesman Col. Steve Warren.
He is conversing in English and is described as being more engaged in treatment. The soldier has not yet spoken to his parents, Warren said.
In Washington, the back-and-forth over the Bergdahl release took on increasingly political overtones, with Republicans who once called for getting him out saying the deal released hardened Taliban commanders who could attack American forces and interests.
Some Democrats joined GOP colleagues in questioning why the swap occurred without 30 days' advance notice, as called for by the National Defense Authorization Act.
Sen. Angus King of Maine, an independent who caucuses with majority Democrats, told CNN the failure to let Congress know ahead of time was a mistake.
At the same time, King defended the administration's reasoning that Bergdahl was in bad physical condition and the window of opportunity to get him out could close.
"He could barely talk," King said of a video the government received during negotiations late last year to prove Bergdahl was still alive. "He couldn't focus his eyes. He was downcast. He was thin."
Other senators who saw the video at a classified briefing Wednesday later said Bergdahl looked drugged, not sick. But King noted "there was a dead silence in the room" after it played.
He also pointed out the Taliban figures involved in the swap may have been freed soon in any event.
"There is a reasonable legal argument that these five guys would have had to be released any way within the next year under the law of war," King said, referring to the coming end of the U.S.-led NATO combat mission in Afghanistan. "They were being held in Guantanamo as enemy combatants. Under the law of war, when hostility cease, enemy combatants have to be released."
According to King, "this may have been the last chance to get Bergdahl where these guys had true value to us as a negotiating tool because if they had to be released any way, we'd be in the same situation without Bowe Bergdahl home."
Meanwhile, a knowledgeable source told CNN that efforts to freeBergdahl from Taliban hands were long and arduous, but the deal mediated by Qatar might pave the way for possible future talks between the United States and the Taliban.
"Often there could be a very long delay, even weeks, between passing a message and getting a response" from the Taliban, said the source, who spoke only on the condition of anonymity.
Bergdahl backlash surprises White House
For example, the negotiation team would "go to Qatar for a few hours, pass the message, and it could be a week or more before we heard back."
Then, on May 23, a U.S. negotiating team landed in Qatar and started another series of negotiations, the source said. That's when the team realized it was closer to a deal.
The pace of the talks picked up. Messages were passed back and forth on a daily basis. By May 27, the basic structure of the deal was reached, and Obama spoke with the Emir of Qatar by phone, the source said.
Implications of the swap
The deal showed that the Taliban had the capability of reaching and executing an agreement, the source added, and that the Taliban's political commission can speak for the movement and can give orders to foot soldiers who then carry them out.
The source said the real conversation going forward needs to be between Afghanistan and the Taliban, but did not discount the possibility that the U.S. would have future discussions with the Taliban.
Bergdahl's captors handed him over to the United States in exchange for the release of five prisoners held at Guantanamo Bay, Cuba. Under terms of the swap, the five were taken to Qatar and are supposed to remain their under watch by the government for at least a year.
The five men, while seen as senior officials in the Taliban, have not been in touch with their colleagues for a decade, and their ability to be operational and hurt U.S. troops if they go back to Afghanistan is believed to be minimal, according to the source.
In contrast, Republican Sen. Marco Rubio of Florida said Wednesday that the released Taliban figures were going back to the battlefield "to kill Americans."
Rubio: Five members of the Taliban are going back to the battlefield
Both Democrats and Republicans have criticized the prisoner swap, while Obama administration officials repeatedly said they fast-tracked the operation over concerns about Bergdahl's health and safety.
Administration officials have said they consulted the Justice Department and acted legally. But some Republican legislators have said they're not buying that argument, and they want proof that
Bergdahl's health was really in jeopardy.
King said Thursday that the administration had intelligence that "had even the fact of these discussions (about the proposed swap) leaked out, there was a reasonable chance Bowe Bergdahl would have been killed."
"That was one of the pieces of information that we learned ... that gave it some credence in terms of why it had to be keep quiet so long," King told CNN's "New Day."
John Bellinger, a former State Department legal adviser in President George W. Bush's administration who was not at the classified briefing, told CNN that the prisoner swap deal was "defensible."
"This is one of those tough national security situations that presidents face," he said, "where all the options are bad."
Deserter or hero?
National Security Adviser Susan Rice said Bergdahl served with honor and distinction. But there's a growing chorus of criticism from some who served with him,describing him as a deserter.
"I believe that he totally deserted not only his fellow soldiers, but his leadership that wanted the best for him and for our country," said former Army Staff Sgt. Justin Gerleve, who was Bergdahl's squad leader.
Some soldiers involved in operations to find Bergdahl have said at least six soldiers were killed searching for him. Asked about this point, Defense Secretary Chuck Hagel told reporters Wednesday that he did not know of specific circumstances or details of soldiers dying as a result of the efforts to find Bergdahl.
Gerleve told "The Lead with Jake Tapper" that he believes Bergdahl is at least partly to blame for the soldiers' deaths.
"I can't really say I blame Bergdahl to fullest extent, but if he wouldn't have deserted us, these soldiers very well could have been in a different place at a different time," Gerleve said.
An Army fact-finding investigation conducted in the months after Bergdahl's 2009 disappearance concluded that he left his outpost deliberately and on his own free will, according to a U.S. military official briefed on the report. The official spoke to CNN on Tuesday on the condition of anonymity because of the sensitive nature of the information
There was no definitive finding that Bergdahl deserted, because that would require knowing his intent -- something Army officials couldn't do without talking to the soldier.
Long road ahead
Bergdahl will remain at a U.S. Army medical center in Germany until he completes treatment, a U.S. defense official there told CNN. After that, he will return to the United States and go to a military base in San Antonio, Texas, the official said.
As the controversy continues, Bergdahl's hometown of Hailey, Idaho, canceled a "Bowe is Back" event planned to celebrate his return "in the interest of public safety."
The city said organizers expected a large number of supporters and protesters.
"Hailey, a town of 8,000, does not have the infrastructure to support an event of the size this could become," the city said.