Hey @Gagan_Sukmawan,
I’ve just had a look at your project. I didn’t find the movement laggy as such, but if you mean that the user needs to keep holding down the mouse button to make the position move rather than click to go to a certain point like before, pathfinding might be the best way to fix this problem, especially since you want your character to be able to navigate around rigid bodies. For this, you can check out either PathFinding3D.js, @Leonidas’s extension, or this thread.
I hope this helps.