From what I can tell it’s an issue with the pathfinding that even jagex was never able to fully fix.
In OSRS there is no more backtracking but it came at the cost of messing up pathing to bankers and other NPCs you can interact with at range, forcing them to add bushes and crates in places to help out the pathfinder.
I think they’ve tried subsequent fixes a couple times but they only ended up making things worse, so they’ve always reverted it to how it is now in the live game.
Why does the backtracking happen, what makes the pathfinding so hard to fix, and is there any way in practice to avoid it from happening?
My guess (it is only a guess) is that it’s a latency issue or race condition. You click to move whilst already moving, and there is a race between moving along your current path and generating the new path, and the current path wins. You advance a tile, then the new path finishes and you begin the new path, but the starting tile of the new path is already behind you, so you backtrack.
My further hypothesis is that this happens only during some parts of a tick (I would guess near the end of the tick) and that to practically avoid it you’d need to change paths only at a certain part of the tick to make sure the pathfinder has time to generate and set your new path before your next movement. Someone with a metronome could try this as an experiment.