The visitors took the lead midway through the half through man-of-the-match Tes Bramble following a mistake by Chris Todd.
Lee Phillips pulled the Grecians level on the stroke of half-time against the run of play.
Exeter were outplayed for much of the first half and were fortunate to be on level terms at the interval.
After the break the game could have gone either way and appeared to be heading for a replay.
But with eight minutes left, Todd again made a mistake when he failed to clear the ball and substitute Adam Proudlock put his side through to the next round
Exeter started brightly and Phillips deflected the ball inches wide after connecting a long-range strike by Billy Jones.
But as the half wore on the visitors began to dominate.
Robert Clare shot wide after he was picked out by Michael Malcolm and Stephen Gleeson shot past the post from 20 yards.
It came as no surprise when County took the lead in the 21st minute. Bramble took advantage of a mistake by Exeter skipper Todd, who stood on the ball and stumbled on the edge of the box and the striker calmly tucked the ball into the bottom corner.
Exeter attempted to rally and Danny Woodards struck a long range shot just over the bar from 20 yards after he was picked out by Phillips.
But Stockport still looked more likely to score and Jones was forced to save well from both Adam Griffin and Adam Le Fondre.
Exeter's equaliser came completely out of the blue on the stroke of half-time.
Taylor found Stansfield with a lofty ball into the box and when the striker cut the ball back across the face of goal Phillips stretched to tap it home from six yards.
Exeter started brightly after the break. Phillips struck the ball just past the post from the edge of the box and Paul Buckle's goalbound strike was deflected just wide.
At the other end, Bramble picked out Malcolm who had a low shot from a tight angle saved by Jones.
Exeter team-mates Buckle and Challinor were both booked after squaring up to each other following a stand-up row as passions threatened to boil over.
With eight minutes left on the clock Stockport grabbed the winner. Bramble threaded the ball through to Proudlock and the young substitute stroked the ball past Jones with a cool finish from 15 yards to break Exeter hearts.