The hardware store where the movie was filmed was over 100 years old and took up an entire city block. The building had previously served as a hotel, a brothel, and, among other purposes, was once a nickel and dime store owned by the Tenenbaum family.
Mr. Kipper's pastel blue Swingline stapler is an homage to the iconic red stapler coveted by Milton in Office Space (1999).
When Jack meets Dave for the first time, he shows him a special hollow-point bullet he's picked out to kill himself with. This is an homage to Mel Gibson in Lethal Weapon (1987) when he shows Danny Glover the hollow-point bullet he intends to kill himself with.
The scene where Mr. Kipper crawls through the air vent and Black Jack shouts after him, "Kipper, what're you doing? You think you're Bruce (expletive) Willis?" is an homage to Bruce Willis in Die Hard (1988).
The scene where Dave asks, "Who's afraid of goats?" and Black Jack retorts, "I ain't afraid a no goats!" is an homage to Ernie Hudson in the original Ghostbusters (1984) when he says, "I ain't afraid a no ghost."