Andy and Barney left town in the squad car to backtrack the bus route to look for Virgil. But they drove up behind him as he was walking on the road, indicating that he was walking away from Mayberry, not toward it as he should have been.
Dinner with Virgil featured a particularly large roast, or something similar, which is promptly dumped into Andy's lap. It just came from the kitchen, so it should be quite hot, yet Andy does not react to the heat when it lands in his lap, nor shortly thereafter when he is holding it in both hands for a rather extended period of time.
When Virgil calls Andy to the cell to let him know that Otis can leave, the rope can be seen tied to the window bars, but it is tied differently when the car pulls the bars out of the wall.
Otis turns his back to the cell door while Virgil works on getting the door open. During this time, Barney is tying a rope to the window bars and when he yanks the bars out with the car, everybody is shocked. Otis would have seen Barney tying the rope while he was waiting for Virgil to do his work.
The concept of Virgil's clumsiness is not just that he can only succeed if no one is watching, but that nothing is expected of him, either. Virgil clearly states this to Andy. So, even though he was alone while polishing the keys, he had the added pressure of everyone's expectations, which should have inhibited him just as much.
The concept of Virgil's clumsiness is that he can only succeed if no one is watching. However, several of his blunders, most notably polishing the jail keys, occur when he is alone.