Richard Oliver of Langport bought the property and built "The Rosery", which stood well back from the street. He later sold it to a Mr. Alfred, who added a wing thus bringing the house forward obscuring the windows. The front door was in the centre facing the street and railings with completed the picture, Oliver then turned his attention to "Askew House". This had been built before the New Inn fire for Miss Risden and her brother, Oliver added a wing and built "The Retreat".Previously a small cottage had stood here, the foundations of which can still be seen. When Oliver bought "Askew House" he required the garden of The Retreat, and altered the house so that the front door faced South instead of North. A. B. Childs, blacksmith, and a cunning worker in bent iron, suggested that the iron railings at the entrance of "The Retreat" against the garden of "Askew House" came from the Minstrel's Gallery in the old Church. Mrs. Smythe lived in "Askew House" until 1894 and added the Drawing Room, after which date it was occupied by Dr. Kerbey and has been the doctor's house ever since.In the eighteenth century the land on which these houses were built was called 'Grange Close", consisting of about an acre and owned by George Webber, a sail-cloth maker. In recent years these houses have been slightly altered. The entrance to "Askew House" has been enlarged, the railings of "The Rosery" replaced by a low wall and the front door moved to the side of the house. The cottage above for a time was a sweet shop kept by a Lyme man named Burge. He was followed by Gower, a small draper, and later Cecil Gear turned it into a fish shop. Since then it has been a private residence. During the alterations it was discovered that the south wall was built of very poor material and it was evident that the front had been brought forward 12 inches. The cottage has been, in years gone by, an attraction for vehicles out of control. A policeman riding a bicycle from Lyme lost control of his machine coming down Lyme Road and dashed into the cottage. On another occasion a van was left unattended on Old Lyme Hill, the brakes mysteriously teased to work and the van ended its career against the cottage. But there is no record of any accident in the days when coaches with four horses drove down the hill. The Church Register does not state where the Balloon Coach from Exeter overturned on 6th September 1805 when Anne Pitt was killed.The shop below "Askew House" was for some years a baker's occupied by Dean, uncle of Fred Penny, then by Dix when it was burnt down in the 1880s. He was succeeded by Durrant from Morecombelake it was rebuilt. For many years it was a grocer's and post office in latter years.