The title is a play on the lyric "Glory Hallelujah " which can be found in "The Battle Hymn of the Republic". It's not in Handel's "Messiah ".
The title is based on the lyrics from the "Hallelujah Chorus" from the "Messiah," an oratorio by George Frideric Handel composed in 1741.
When Felix calls Oscar, "The last of the lukewarm lovers" he is referencing another Neil Simon play,"The Last Of The Red Hot Lovers"
Oscar's line "What am I smiling about?" while leaving the apartment, was dubbed in.