Tom's father has escaped from a prison called Millhaven. There is no Millhaven prison in the US where this movie is set, but there is one in Canada where this movie is actually filmed.
The castle scenes were filmed at Hatley Castle on Vancouver Island, Canada. The castle seems to be used quite a bit for movie and TV filming, previously used in Deadpool (2016), Descendants (2015), and X-Men: First Class (2011) among other film projects. A list of what movie locations used it can be found on the castle's website.