From Kingston historian Edwin Millard Ford's book Street Whys: Anecdotes and Lore About the Streets of Kingston, New York:

"The George H. Sharpe mansion was located at #1 Albany Avenue on the corner of Clinton Avenue. It was built shortly after the Civil War and was occupied by the general until his death. When the site was purchased for the erection of the Governer Clinton Hotel in 1924, it was decided to move the mansion to the rear of the lot rather than demolish it. The 400 ton building was raised by means of jacks and huge timbers were placed underneath. Then, using horses and winches, the brick building was moved, inch by inch, 200 feet. During the move, not a window was broken or a brick displaced. The mansion became an annex to the Governor Clinton Hotel and was later used for storage. It was demolished in the 1960's for extra parking space.
Born in Kingston on February 26, 1828, George Sharpe attended Kingston and Albany Academies. He graudated with honors from Rutgers University in 1847 and proceeded to study law at Yale University. He was admitted to the bar in New York City in 1854. He practiced law in Kingston and enjoyed a prosperous practice, which in time prompted his resignation as Captain of the 20th militia. However, when the news of Fort Sumter broke, he withdrew his resignation and without waiting for orders from his colonel, he hung the flag at daybreak at the local Armory and recruited 248 men to the 20th militia. In 1862, he received a colonel's commission and authority to begin recruiting the 120th regiment. Within 22 days he had recruited 1041 men and on the 23rd day he headed for war.
Attached to the Army of the Potomac, he took part in many battles, from Fredericksburg to Appomattox. As provost general, he served on the staffs of Generals Hooker, Meade and Grant. He was breveted to brigadier general in 1864 and major general in 1865. 
As a chief advisor on General Grant's staff following Lee's surrender, he was entrusted with the duty of paroling the Confederate Army. In 1867, he was sent as a special agent of the State Department to Europe to identify U.S. citizens involved with the assassination of Lincoln and the attempted assassination of Secretary of State Seward. This assignment made him the first CIA agent in the United States.
General Sharpe entertained many famous people at his home on Albany Avenue, including Ulysses S. Grant during his presidency in 1873, and members of his cabinet. Grant visited again with his family some years later. Before he became president, General Chester A. Arthur visited Sharpe's Albany Avenue home, as did many of the general of the Civil War, governors, senators, assemblymen and other officials of the United States Government.
He died at the home of his son-in-law, Ira Davenport, in New York City on January 13, 1900. His wife, Caroline Hasbrouck Sharpe, had died two years earlier. At his bedside were his daughter, Mrs. Davenport, and two sons, Judge Severyn B. Sharpe of Ulster County and Lieutenant Colonel Henry G. Sharpe of the Federal Commissary Department. His funeral was held at his home on Albany Avenue. In 1832, a private cemetery was established at 148 Albany Avenue and named 'Sharp's Burying Ground.' Burials ceased there over one hundred years ago and it gradually became an abandoned cemetery. In 1996, with approval from Kingston's Mayor T.R. Gallo, the Friends of Historic Kingston assumed responsibility for its restoration."