When Ray goes across the street to bring Frank some cake and talk to him, he does not bring his coat with him. However, a few scenes later, when Ray comes home after going to the store, he suddenly has his coat. It's very unlikely that he went home first, especially since in this scene, he and Debra are apologizing to each other like they haven't seen one another since Ray left the first time.