The striker converted from 12 yards after Ryan Williams had been fouled in the area to set up a second-round trip to Wrexham on Monday.
It was a dramatic finish to a game which for long periods had been locked in stalemate.
Gateshead had been the better side, but could not turn their possession into an end product.
United lacked quality on the ball, although it was hardly surprising given they began the night with a makeshift side.
- Ryan Williams sets off on a run
The injury list grew longer throughout the night as their side began to resemble a youth team, but once again their character shone through.
It wasn't pretty, but they will face Wrexham in the second round on Monday.
The team sheet brought two surprises, as United's injury-hit squad was stretched further by the loss of Michael Raynes and Johnny Mullins.
It forced a radical reshuffle of the back four, with David Hunt and Tom Newey shifting into central defence.
In their place at full back were Matt Bevans and David Lynn, both making their senior debuts, while Scott Davies replaced the ineligible Asa Hall.
Gateshead had been 2-0 up in the first game at the Kassam Stadium almost a month earlier and they made most of the running before the break.
Gary Mills' side were unchanged and looked the more coherent side.
Josh Walker provided their main threat and the striker headed wide from an early corner.
United's patchwork back four looked unsurprisingly shaky, with Bevans giving away two cheap set pieces early on.
- Dave Kitson keeps possession as a Gateshead defender closes in
Gateshead won five corners in the opening 20 minutes and from the fifth James Curtis rose to head goalwards, forcing Ryan Clarke into a save.
Gateshead looked good until the final third and had relatively little to show for their control of the game.
One promising move late in the half was ended when Walker's shot on the turn was well blocked by Bevans.
United were very cautious and rarely had good possession in the opposition's half, but they showed signs of life early in the second half.
Kitson flicked a Rose corner which Jamie Chandler cleared off the line, while Constable had a shot well blocked from a neat Williams pass.
Just before the hour the boisterous home fans appealed for a penalty when Chandler was sent sprawling by Davies.
The referee awarded a free-kick on the edge of the box and the United wall blocked Sirrell's effort.
Smalley replaced Davies to give the visitors more of an attacking threat on the right flank.
Williams switched to the opposite wing and caused the most problems to the Heed defence.
The visitors were beginning to enjoy their best spell of the game, but Gateshead hit back.
With ten minutes remaining Colin Larkin found some space just inside the area, but his powerful shot was too close to Clarke, who parried.
It was the final chance of the 90 minutes, but the game began to open up as legs tired in extra time.
- James Constable sees his way blocked by two Gateshead defenders
Walker saw an acrobatic effort roll just wide, while at the other end Williams forced a smart save from Bartlett as Gateshead's defence hesitated.
In the 100th minute Larkin had the ball in the net, but it was ruled out for offside.
The home side looked the more likely to find a winner as the minutes ticked by.
But just as it looked a shoot-out was inevitable, Williams was felled by Curtis in the area.
Smalley stepped up and coolly slotted in his seventh goal of the season before being mobbed by his teammates.
Oxford Utd (4-4-1-1): Clarke, Bevans, Hunt, Newey, Lynn, Williams, Davies, Rose, Ruffels, Kitson, Constable.
Subs used: Smalley (Davies 68), Long (Rose 99), Marsh (Kitson 106) Unused subs: Crocombe, Lee, Shama, O'Dowda.
Booked: None Gateshead (4-3-3): Bartlett, Baxter, Curtis, Clark, Sirrell, Oster (Tait 118), Turnbull, Chandler (Ramshaw 115), Larkin, Walker (Hatch 118), Marwood.
Unused subs: Magnay, Baird, Brown, Maddison.
Booked: Marwood Referee: Paul Tierney (Lancashire).
Attendance: 2,632 (98 visitors).