"The Slave of Duty" is the subtitle of Gilbert and Sullivan's "The Pirates of Penzance." In his eulogy Hotch talks about how he and Haley met during a student performance of this operetta. The opening quote is from another of their works.
When Reid and Det. Kaminski are checking the names of patrons who paid for V.I.P. valet parking, two if the names that come up are "Jennifer Ketelsen" and "Heather Aldridge." These are in-jokes, as Jennifer Yates (also known as Ketelsen) is actually one of the series' associate producers, and Heather Noël Aldridge was often identified as "additional crew."
At the end of the episode when Hotchner is setting on the bench at Haley's grave, all the tombstones are facing one direction while Haley's markers is facing the opposite indicating she was buried the wrong way.