Adam Baldwin admitted to swiping the Jayne statue's head as a souvenir after filming the episode. Later on, additional shots were needed for the episode (scene where the crew first sees the statue) and the crack where the head was broken off and then reattached is visible.
The song sung about Jayne borrows a lot from the Ballad of Jesse James.
Episode #7 in the "Joss Whedon Intended Timeline" for "Firefly" and NOT as Fox dictated the episodes were to be run.
The theme of the episode is faith. Shepherd Book's faith in the Bible fixes him. There have been many hints that he was once someone who knew a lot about crime and not in a theoretical way. The mudders have faith in Jayne, a criminal whom they have mythologized into a Robin Hood hero.
Episode 5 is supposed to officially be "Safe" according to the official "Firefly" box set released 2017. "Jaynestown" is officially Episode 7 (after "Our Mrs. Reynolds").