Westerly, the handsome home of Porter R. Chandler, was begun about 1850 by Major William H. Spencer, a veteran of the War of 1812. One of the early settlers in the Genesee Valley, Mr. Spencer came to Geneseo in 1805 at the urging of James Wadsworth. In the first decades of the nineteenth century, Geneseo was a market town for this section of the country, and Messrs. Wadsworth and Spencer opened a store on the village square. Allen Ayrault (builder of the Big Tree Inn) was one of their clerks during these early years. Soon after the close of the War of 1812 Major Spencer purchased the land across the river, today known as “Westerly,” then called “Spencer’s Flats.” He first built the farmhouse which today is across the road from the entrance to the mansion. In 1838, Major Spencer took as his bride Miss Laura Adams, and soon afterwards made an addition to his home. This addition was in recent years moved to its present site and is referred to as the upper farmhouse. The old Genesee Valley Canal ran through the estate and the section near the railroad tracks was known as “Spencer's Basin.”  Located there was a warehouse (now a cow barn), and an office for the warehouse (today used for storage). Major Spencer died suddenly in 185l while the present home was nearing completion. His widow saw it finished and lived there with her son, William H. Spencer Jr. Upon her death in 1864 the property went to her son and a daughter, Mrs. Fleurat of Philadelphia. The farm had become proverbial for its rich soil and high state of cultivation, and the lovely home commanded a fine view of the valley. William Spencer Jr. continued to farm and to maintain the estate in good order. About 1869 he married President Tyler's daughter, who two years later died in childbirth. President Tyler’s widow had come to be with her daughter and carved her name with a diamond ring on a window pane in the bedroom. The baby was named Julia and was adopted by relatives.

Not long after becoming a widower, Mr. Spencer sold the estate to Charles Wadsworth, a son of the late General who had given his life in the Civil War. Mr. Wadsworth and his wife, the former Jessie Burden, spent the rest of their lives there. An avid cattleman, Mr. Wadsworth introduced the shorthorn breed of cattle to the valley and maintained a magnificent herd. The new blood greatly improved the quality of beef cattle in the valley and throughout the Northeast. Mary Wharton, their only daughter, married Porter Ralph Chandler of Batavia in an impressive ceremony in 1896. The newlyweds, after a trip to Japan, took up residence on the home farm, as it was known then (the farmhouse referred to earlier). Three years later Mrs. Chandler was left a widow. Their son, Porter R. Chandler, Jr., was born after his father's death. Later that same year her father died. The young widow and her son remained in the farmhouse until the death of Mrs. Wadsworth in 1917, when they moved into the family home. Porter R. Chandler Jr., who became a prominent New York attorney, married Gabrielle Chanler, daughter of Mr. and Mrs. Winthrop Chanler and a sister of the late Admiral Hubert Chanler. Although business forced them to establish a second home in New York City, Mr. and Mrs. Chandler and their family returned to the valley as often as possible. The handsome home has been named a National Historic Site.