"Mike Pompeo, Director of the CIA, will become our new Secretary of State. He will do a fantastic job! Thank you to Rex Tillerson for his service! Gina Haspel will become the new Director of the CIA, and the first woman so chosen. Congratulations to all!" Trump tweeted.
"What is most important is to ensure an orderly and smooth transition during a time that the country continues to face significant policy and national security challenges," Tillerson told reporters in a packed briefing room at the State Department.
Sacked US Secretary of State Rex Tillerson also warned of Russia's "troubling behaviour and actions" in a parting statement.
Trump also nominated intelligence officer Gina Haspel to replace Pompeo at the CIA.
She would become the first woman to run the spy agency. Both Haspel and Pompeo would need to be confirmed by the Senate at a time when the closely divided chamber has stalled on confirming dozens of Trump nominees.
Trump said Tuesday he has ousted Secretary of State Rex Tillerson and plans to nominate CIA Director Mike Pompeo to replace him
Before Tuesday's announcement, rumors circulated for several months that Secretary Tillerson was considering resignation.
A White House official has indicated that the cabinet reshuffle was connected to Trump's desire to shake up his team ahead of upcoming talks with North Korea.
The White House and Mr Tillerson have had a series of public disagreements - and that continued until the very end.
Trump told reporters Tuesday that he ultimately decided to fire the secretary because they disagreed over strategy in key areas of foreign policy, such as the 2015 Iran nuclear deal, the approach to North Korea and the overall tone of U.S. diplomacy.
A growing number of people have left the Trump administration in just over a year – a group that includes FBI director James Comey, chief strategist Steve Bannon.
Some, such as Comey, were fired by President Trump, while others resigned for a variety of reasons.
