That offseason, the Yankees acquired Carmel and stashed him at AAA Buffalo, where he proceeded to hit 35 longballs, knock in 99 runs, and post a .404 on-base percentage. He went to Spring Training with the Bombers in 1965 amid much fanfare; in Ball Four, Jim Bouton sardonically notes that he was supposed to be "the next Joe DiMaggio". According to Bouton, he struggled in camp and Whitey Ford consoled him by suggesting that he just couldn't hit in South Florida. When he continued to play poorly in Tampa, Whitey amended his remarks: "You're just not a Florida hitter." Of course, Duke then faltered in a few more games further north, prompting the great Yankee pitcher to say, "It looks like you just can't hit south of the Mason-Dixon Line." Nor could he hit in the Bronx. The team put the 28-year-old on the Opening Day roster, and he went hitless in eight at-bats, striking out five times. That was the last that Duke Carmel would see of the major leagues. He spent three more years riding the buses before retiring.
Fun fact: Dateline: April 16, 1963. Duke's first career home run was a pinch-hit, game-tying solo shot leading off the bottom of the ninth inning against Roy Face. He sparked a game-winning rally by the Cardinals, as Curt Flood followed with a double and scored on an error by Pirates second baseman Julio Gotay.