Fear of rabbits is Kouneliophobia.
The portrait behind Bishop Brennan at 3:17 is of Pope Innocent X by Diego Velázquez. This particular portrait was a subject of Francis Bacon's famous "screaming pope" series of studies.
The upstairs of Craggy Island Parochial House (when Ted races upstairs to stop Bishop Brennan from entering the spare room) looks suspiciously like the upstairs of Fawlty Towers (a show itself said to have inspired Father Ted's creation).