The first mention of Jedediah Lathrop in town records is on the occasion of his marriage to Mary Caldwell in 1793. Where he came from or how he made his money is still a mystery. He was prominent in town affairs as a warden of Christ Church, warden of the borough, master of St. Alban’s Lodge, and a member of the reception committee for General Lafayette when he came to town in 1824. Lathrop bought this property “and building thereon” in 1796 from Nathaniel Caldwell, tore down the existing house, and built a new one. With four chimneys, modillioned cornice, carved windowheads, handsome elliptical windows in the side gables, and an imposing porch unlike any other in Guilford at the time, it was and is outstanding and apparently a source of great pride to him. Legend has it that a small girl overheard him say to General Lafayette after they had dined at Bradley’s Tavern, “The house with the four chimneys that you see across the Green is—er— mine.” Judge Ralph D. Smith and his wife, Rachel Seward, bought the property from Lathrop in 1854. A later owner was Bernard C. Steiner, the author of History of Guilford and Madison, Connecticut which included material collected by his grandfather Smith. In c1960 a north wing was added with detail copied from the main house; the Ionic columns were said to have been shipped from San Francisco. At the same time, the barn was connected to the
rear of the house and the interior divided into apartments and an antique shop. The house has been owned by the town since 1970.
From Guilford: A Walking Guide, the Green & Neighboring Streets [by] Sarah Brown McCulloch. Text from 2006 printing, revised, with editorial notes in brackets from 2012.