Richard Oliver of Langport bought the property and built "The Rosery", which stood well back from the street. He later sold it to a Mr. Alfred, who added a wing thus bringing the house forward obscuring the windows. The front door was in the centre facing the street and railings with completed the picture, Oliver then turned his attention to "Askew House". This had been built before the New Inn fire for Miss Risden and her brother, Oliver added a wing and built "The Retreat".Previously a small cottage had stood here, the foundations of which can still be seen. When Oliver bought "Askew House" he required the garden of The Retreat, and altered the house so that the front door faced South instead of North. A. B. Childs, blacksmith, and a cunning worker in bent iron, suggested that the iron railings at the entrance of "The Retreat" against the garden of "Askew House" came from the Minstrel's Gallery in the old Church. Mrs. Smythe lived in "Askew House" until 1894 and added the Drawing Room, after which date it was occupied by Dr. Kerbey and has been the doctor's house ever since.In the eighteenth century the land on which these houses were built was called 'Grange Close", consisting of about an acre and owned by George Webber, a sail-cloth maker