You and I seem to completely agree on this. I really dislike a backlog of bad conscious with things we "mustn't forget". My reasoning is more (akin to yours): if it's a necessary and actually requested or useful feature, then we'll rediscover it. (This doesn't apply to bugs!)
I really like that. It also helps reassuring the importance of items. Like, if you want to create the same story three times in a year, it might actually be important.
It's that for a team of a given size, we can only collectively produce so much. So as long as we're always picking tickets of a priority that stakeholders and we deem of the highest priority, then we're doing the right things.
This necessarily means that the backlog probably grows.