Isn't the business decision exactly whats being questioned? Not saying it is a bad decision, but I did have the same question. Was it really necessary to write a completely new DE just to make some tweaks easier to implement? Would love to hear more details about this.
At first glance, it seems like a truly massive effort for a marginal improvement, that could have been made with a fraction of the cost. I'm probably wrong though, hopefully.
The problem was that the Gnome crowd is going increasingly in a different direction. Mutter already doesnt support a number of otherwise universal protocol. The way theming work is compelely changed. There are lots of other reason why staying with gnome was an increasing problem for them. All those reasons are only growing bigger.
So to support what they wanted and what they promise their costumers they would have to do increasingly more and more work, in direct oppositon the the gnome community. Essentially eventially forking it. And then they have to maintain a huge legacy codebase.
And they didnt wirte a completly new one. The compositor was already a project. So was Iced. They use lots of other existing project in the Rust ecosystem.
Now they are on their own, free to build what they want, rather then endlessly pushing a rock up a mountain. They can now make more fundamental changed that would never have been possible in Gnome. They are also free to build the community they want rather then deal with the Gnome community. This also makes them a player in the Wayland community, another voice that can push protocols.
The situation between System76 and Gnome had become untenable. The modifications they wanted to make to the shell were too many and the Gnome devs don't want to support any of it.
I read somewhere that 50% of the development time on Pop_Os was spent simply un-breaking the Gnome shell extension between releases of GNOME.
Now they can put that time into creating what they actually want.
At first glance, it seems like a truly massive effort for a marginal improvement, that could have been made with a fraction of the cost. I'm probably wrong though, hopefully.