Exeter defended heroically against an onslaught from the League One side but were good value for the victory after a battling performance.
Steve Flack opened the scoring with a glancing header in the first half and Dean Moxey stretched the scoring after the break.
Gregg Blundell pulled a goal back at the death to set up a nail-biting finish but Exeter held on to book their place in the third round.
Rovers had the best early chance when Dave Morley forced a good save from Martin Rice when he connected with Jermaine McSporran's free-kick.
Exeter responded and Sean Devine forced the goalkeeper to save well with a shot from a tight angle.
Exeter went ahead in the 25th minute. Alex Jeannin's free-kick was met by Flack and the giant striker glanced the ball into back of the net with the deftest of headers.
Just before half time, Blundell headed the ball into the ground and over the bar from point blank range after meeting a cross from Neil Roberts.
Exeter came close to stretching the lead just before the break when Flack met Les Afful's cross and his effort was kicked off the line by Morley.
Exeter stretched the lead four minutes after the restart. Moxey, who came off the bench to replace Kwame Ampadu in the first half, danced past two defenders, spotted the goalkeeper off his line a lobbed a superb strike into the top corner from fully 30 yards.
Devine attempted to lob Andy Warrington from 20 yards but the goalkeeper leapt well to tip the ball over.
Substitute Guy Ipoua almost pulled a goal back with a header against the post.
With four minutes left Blundell finally found the back of the net with a close range strike from James Coppinger's cross.
But Exeter held on for a deserved place in the third round.