Imperial Isle in the 1000 Islands

Long before Gilbert T. Rafferty built his castle-like summer home on Imperial Isle in “Millionaire’s Row,” the island was described in the book Meanderings Among A Thousand Islands as once being “not much more than a little cluster of rocks with a few trees on one of them.” It would be enlarged by filling between and around the rocks, creating approximately 1 acre squarely between Heart and St. Elmo Island on the St. Lawrence River opposite Alexandria Bay.  

Imperial Isle was once owned by Joseph McNaughton, dean of the St. Lawrence County bar, who died at the age of 93 in May 1916.  In a Watertown Daily Times article at the time of his death, it was said—

It was recalled yesterday that Mr. McNaughton was at one time the owner of Imperial Isle opposite Alexandria Bay, now known as Rafferty’s Island, being the site of the magnificent summer home of the Pittsburgh millionaire.  Mr. McNaughton was so fond of firing an old muzzle loading cannon that he kept on the island that the people of Alexandria Bay objected, and the village fathers adopted an ordinance which, while general in its scope, was intended to suppress McNaughton and his cannon. 

The night before the law went into affect the master of Imperial Isle filled the cannon to the muzzle with slugs and he and his companions retired to a safe distance to pull the lanyard.  The charge not only made a prodigious farewell salute, but blew the place into bits.  Mr. McNaughton afterward sold the island to C. J. Singer of the Singer Sewing Machine company, who in turn disposed of it to its present owner (Rafferty) for $20,000 (in 1888.)

The Imperial Isle that Gilbert T. Rafferty purchased had smaller cottages on it, which he would replace with what looked like a fortress, including a four-story castle-like tower, sometime before 1900. This quickly became known as one of the magnificent, if not altogether imposing, summer homes on the St. Lawrence.

Rafferty would retire at the age of 48 in 1895 as President of the McClure Coke Company, having made his fortune in the Pittsburgh industry. This allowed him to travel and build his summer home in the coveted Thousand Islands. His love of boating led to his being named Commodore of the Thousand Islands Yacht Club for several years, with George C. Boldt serving as Vice Commodore.

In July of 1899, Rafferty’s yacht Consuela, a $10,000 boat built of cedar with nickel railings, was destroyed in a fire. Fortunately, the large yacht house he had recently constructed was saved from further damage by Engineer Fred Stowell, who was sleeping in its quarters at the time and discovered the fire. The yacht house stored two boats, the other being Mr. Rafferty’s “Jessie N.” 

In 1901, Gilbert T. Rafferty purchased Bonnie Castle from the children of the late Doctor and well-known author Josiah Holland and greatly improved it with George Boldt and James H. Oliphant of nearby Neh-Mahbin as stakeholders. Rafferty would turn the peninsula on the mainland into an island, and the Alexandria Bay landmark would remain with the Rafferty family well after his death until 1941. Despite the cooperation between well-known and wealthy Thousand Islanders, everything was not always rosy.

In 1906, fellow Pittsburgh and island neighbor Alexander R. Peacock, who previously struck a deal with Rafferty regarding a strip of land on Wellesley Island, needed more to expand his boathouse.  Rafferty, having given Peacock the first bit of land for free, coaxed Peacock into a game of chance: the toss of a coin would decide if Peacock would subsequently pay him for both pieces of land or if Peacock would pay nothing at all for them. Boldt was there to witness the deal, but unfortunately, it didn’t go Rafferty’s way.

What ensued was a decade-long court battle, all the way to the Supreme Court, which resulted in Boldt being summoned by subpoena as a witness. Boldt made a 600-mile round trip to famously utter two words, “No Sir,” when asked a single question. For more details on the issue and trial between the two Yinzers can be found on the Belle Island article by clicking here.

A year after the trial concluded in 1916, Gilbert T. Rafferty passed away at the age of 69 at his home in Pittsburgh. His obituary in the May 7, 1917, Watertown Daily Times would read in part–

Alexandria Bay, May 7 — Gilbert T. Rafferty, well known millionaire summer resident, whose residence is Imperial Island, died this morning in his home in Pittsburgh, Pa., following a long illness.  

Mr. Rafferty had been coming to the Thousand Islands to spend the summers for the past 35-40 years. He would come early in the season and stay late, and spend a great deal of his time on the river fishing.

Imperial Island, where his magnificent summer home is located, is located opposite Alexandria Bay. It is the first island above Heart Island, where the George C. Boldt castle is located, and is between Heart Island and Belle Island, the summer home of A. R. Peacock. The Rafferty summer home is one of the most elaborate on the river and is valued between $75,000 and $100,000. 

Rafferty’s wife, Harriet Martin Oliver Rafferty, would die three weeks later in Pittsburgh on May 28.  Eight years after his death, the George T. Rafferty Estate still held Imperial Isle, assessed at $20,000, Bonnie Castle at $20,000, and Crescent Cottage, at $3,500.  

Imperial Isle and its 32-room summer home would pass from the Rafferty Estate to Stephen K. Bresee of Syracuse in 1927. Bresee, a well-known automobile dealer, purchased Imperial Island after it had been unoccupied the previous year after having been the summer home for Alvin Markle and his family. Mr. Bresee sold Imperial Island the following year to D. Stewart Tuttle of New York and purchased St. Elmo Island in 1929, while Tuttle rented Imperial Island out.  

The property would remain in the Tuttle family, most notably D. Stewart Tuttle’s wife, Eleanor, of Lyons Falls, N.Y., who would remarry twice, first to Guy Mansfield Rush in 1931 and then Samuel Burr Ogden in 1949. In 1946, Imperial Isle, then owned by Eleanor Rush via husband number two, was sold to Walter E. Ditmars, president of Gray Manufacturing Company of Hartford, Connecticut. Mr. Ditmars would continue a long line of philanthropy of summer homeowners to the local community, donating $20,000 to construct the new hospital in Alexandria Bay.  

Although Ditmars planned to make extensive changes to Imperial Island and its buildings, he never got around to it, and the property was unoccupied over the course of a decade from 1949 – 1959 when it was unoccupied. At that time, Ditmars gifted it to the Edward John Noble Hospital’s board of directors to “dispose of to the best interest of the hospital as the board sees fit.” According to the Watertown Daily Times article published September 26, 1959—

The property has fallen in bad repair in the past ten years, with the furniture and fine woodwork having been removed.  Vandals have broken the windows and caused extensive damage otherwise. 

The hospital would sell the property the following year for $5,500 to William Dunn, an official at Breese Chevrolet Company in Syracuse. Dunn razed the massive castle-like structure in 1962 and replaced it with a modern redwood house, later selling it in 1970 to J. Howard Kidd of Rochester, N.Y. 

In recent years, Imperial Island was part of the Vladi Private Islands group’s holdings but has since been sold. Their website has an archive of the property showing recent photos.  

Interesting Tidbit

Imperial Isle, also known as Imperial Island, had been used as the island’s name since at least the 1880s. It was not officially recognized by the Board of Geographic Names until 1975.