A manor house and chapel was built to the south of St Cuthbert's Church on a plot of land once called Hallgarth, by Bishop Pudsey in circa 1160 as a bishop's palace. In 1806 it was bought for use as a poor-house. It was a long, low building with windows dating from medieval to the 18th century. Most of the palace was destroyed in 1828 during building operations and the chapel was destroyed circa 1870.The site has now been built over.