The original homestead was constructed sometime between between 1742 and 1759 by either Christopher Van Bommell (who owned the land until 1759) or Johannis A. Fort (who bought the land from Van Bommell). There have been many additions to the origin two-room house throughout the years to accommodate the various families and organizations that have owned the house.
Ownership of the property remained in the hands of the Fort Family descendents until the late 19th century. The Nelson Family purchased the property from the Forts. The Dickerson Family purchased the Fort Homestead in 1911 and from then until 1944 the homestead was a working 300-acre farm. From 1944 to 1957, IBM owned the property. During this time, Thomas Watson, Sr. refurbished the homestead and he and his wife furnished the house with many of their own personal authentic period furniture. The Miller Family purchased the homestead from IBM in 1957 and called it the “Colonial House.” In 1991, ownership of the homestead reverted to IBM. IBM rented the homestead and other nearby residences to employees (including the family of Janet Mills) and used it for other occassions. The property was sold to Casperkill I LLC, a development company (Ginsburg Development), in 2004.