The announcement is the latest twist in a long-running saga after the world's richest person decided to buy Twitter in April.
Mr Musk said he had backed out because Twitter failed to provide enough information on the number of spam and fake accounts.
Twitter says it plans to pursue legal action to enforce the agreement.
"The Twitter Board is committed to closing the transaction on the price and terms agreed upon with Mr. Musk," Twitter chairman Bret Taylor wrote in a tweet, setting up a potentially long and protracted legal battle between the two sides.
The original merger agreement includes a $1bn (£830m) break-up fee.
In May, Mr Musk said the deal was "temporarily on hold" as he was awaiting data on the number of fake and spam accounts on Twitter.
The billionaire businessman had asked for evidence to back the company's assertion that spam and bot accounts make up less than 5% of its total users.
In a letter filed with the US Securities and Exchange Commission, Mr Musk's lawyer said Twitter had failed or refused to provide this information.
"Sometimes Twitter has ignored Mr. Musk's requests, sometimes it has rejected them for reasons that appear to be unjustified, and sometimes it has claimed to comply while giving Mr. Musk incomplete or unusable information," the letter reads.
Spam accounts are designed to spread information to large numbers of people and manipulate the way they interact with the platform. On Thursday, Twitter said it removed around 1 million such accounts each day.
Mr Musk believes that spam or bot accounts could account for 20% or more of Twitter users.
Shares in Twitter fell by 7% in extended trading after the announcement.
Mr Musk, said to be the world's richest person, is the founder of rocket company SpaceX and electric car company Tesla.
A self-described "free speech absolutist", he had pledged to loosen Twitter's content moderation rules once the company was under his ownership.
He has long criticised Twitter's ban of some accounts, like that of former US President Donald Trump.
He has also called for more transparency over how the platform presents tweets to users, a system that currently allows some to be promoted and given wider audiences.
-- Courtesy of BBC News