Sun, 53, was the youngest member of the Politburo and considered a rising star ahead of a key leadership reshuffle later this year.
State media reported earlier that he was replaced by a provincial party chief Chen Miner, an associate of President Xi Jinping.
A party meeting in Chongqing to announce Sun's departure was told he was under investigation, the Journal reported, adding that no details were given.
Chongqing is a fast-growing mega- city and one of four centrally administered municipalities, alongside Beijing, Shanghai and Tianjin.
If confirmed, it would be the first probe of a sitting Politburo member, a key leadership body in the Communist Party, since the 2012 downfall of another one-time Chongqing chief, Bo Xilai, who was later sentenced to life in prison for corruption and abuse of power.
It comes before a party congress where many of the 25 members of the Politburo are expected to be replaced as Xi heads into the second half of his scheduled term in power. Five of seven members of the supreme Politburo Standing Committee - all except Xi and Premier Li Keqiang - are also scheduled to retire.
Xi has for more than four years embarked on a sweeping crackdown on graft at all levels of the party. He's warned corruption is one of the biggest threats to the party's future. The campaign has also allowed him to amass greater power.
Sun had previously drawn praise from the president.
During a visit to Chongqing in January 2016, Xi stood side by side with Sun and admired various projects under his purview, according to Xinhua News Agency.
While inspecting port and railway projects to support Xi's initiative for a new Silk Road trading route to Europe, the president said: "This place is very promising."
Still, the party's top anti-graft agency, the Central Commission for Discipline Inspection, said in February after an inspection trip to Chongqing that the municipality fell short of "clearing toxic residue" from Bo's era. And in June, state media said Chongqing's top police officer, He Ting, had been relieved of his post.
Sun's removal could help Xi solidify his grip on power ahead of the party reshuffle, said Willy Lam, an adjunct political professor at the Chinese University of Hong Kong. Sun, together with Guangdong party boss Hu Chunhua, had been regarded as potentially in line to be elevated to the Standing Committee.
In June 2015, Zhou Yongkang, a retired member of the supreme Standing Committee, was sentenced to life in prison, the highest-ranking official convicted for graft in party history.
Chongqing, a sprawling municipality of nearly 30 million people, has prospered as the country's economic boom rolled inland, reinventing itself as a regional hub for manufacturing, logistics, financial services and foreign investment.