The Beijing-based company, which also owns popular social network TikTok, entered the gaming market in 2019 to compete with industry leader Tencent.
But it has failed to grab market share.
The BBC understands that games with active players, such as Crystal of Atlan and Earth: Revival will continue. Titles which have not yet launched will be shut down in December.
The decision will likely impact hundreds of employees.
A spokesperson from ByteDance told the BBC the company regularly reviews its business and it has "made the difficult decision to restructure our gaming business".
The global video game market was estimated to be worth $217bn (£172bn) last year, according to market research and consulting company Grand View Research.
ByteDance's creation of its gaming division Nuverse in 2019 was widely seen as a major push into a once booming sector.
In 2021, the company formalised its status as one of six business units under a broader structural overhaul.
To build up production capacity, Nuverse acquired external studios such as C4games.
But Nuverse's performance has been patchy with none of its games becoming a commercial hit.
The latest move marks a retreat from the competitive video gaming sector.
ByteDance's competitor Tencent is the world's biggest gaming company in terms of revenue.
Formed in 2012, ByteDance created the global smash-hit app TikTok which has an estimated 1.1bn active monthly users.
Its massive popularity has meant it has been scrutinised by governments around the world, including in the US and China.
During his term, US President Donald Trump regularly attacked ByteDance, accusing TikTok of being a threat to US national security.
Politicians and officials raised concerns about users' personal data being passed to the Chinese government.
TikTok has denied accusations that it shares user data but its CEO Shou Zi Chew faced hours of questioning at a US congressional hearing in March.
There has been growing pressure on ByteDance to spin off its shares in TikTok or risk facing a ban.