I don't know whether to agree or disagree with you. The rules for time travel in Endgame made no sense to begin with, so I'll just take your word for it.
They absolutely made no sense. After a while I stopped giving a fuck and just enjoyed the movie.