After breezing through the first six innings, facing only three over the minimum, Nova (5-3) came up short in his bid to become the first pitcher in the big leagues this season with three complete games.
He went to the ninth with a 9-2 lead, only to be lifted after giving up three hits.
"It's frustrating," said Nova, who went 8 1/3 innings before Jhan Marinez got the final two outs. "To go out in the ninth inning with that lead, I've got to finish it off. I didn't make the pitches I needed to make and it almost got away from us."
Pittsburgh manager Clint Hurdle had no complaints. Nova retired 19 hitters on three pitches or fewer, and only went to three balls on one batter -- Kurt Suzuki, who walked in the second.
The only solace for the Braves: It was the first time in Nova's three career appearances against them that they've scored an earned run.
"He knows how to attack," Hurdle said. "There's an art to pitching. He's not throwing. He's pitching."
The Pirates split the four-game series at SunTrust Park after losing the first two contests. This one was easy after Pittsburgh put up five runs in the third inning, capped by Frazier's third homer this season.
It was another big day for the rookie who grew up about an hour away near Athens, Georgia. The previous night, Frazier reached base safely in all six plate appearances, including a homer and four walks .
"That's pretty awesome," he said, beaming.
Colon (2-5) had another tough outing. One day after his 44th birthday, he allowed seven runs and 10 hits in a five-inning stint, raising his ERA to 6.96.
An All-Star last season for the New York Mets, Colon has been charged with five or more runs in half of his 10 starts with the Braves.
"It's on me," he said through a translator. "When you get behind on the count, you have to throw it down the middle. That's what hurts you."
When asked if Colon would stay in the rotation, manager Brian Snitker said, "As of right now, yes."
He added, "I look at the guy's track record. I've got the feeling that a guy who has survived as long as he has will find a way to make the adjustments he needs to do to be successful."
Left fielder Josh Harrison made a diving grab in the third inning to deny Colon his first hit of the season.
Harrison raced toward the line, stretched out for the catch and slid along the grass on his stomach. Rolling to his back, he held up his glove for the umpires to see before flipping the ball back to the infield while still on his back.
Colon smiled on his way back to the dugout. He is hitless in 13 at-bats.
McCUTCHEN SITS AGAIN
Mired in a slump, Pittsburgh CF Andrew McCutchen didn't start for the second game in a row.
He had been gone hitless in 15 at-bats before a pinch-hit single in Wednesday's game. Manager Clint Hurdle decided to give McCutchen another day off, starting Frazier in center field.
McCutchen is hitting just .203 with six homers and 20 RBI.
Pirates OF Gregory Polanco, who was on the 10-day disabled list with a strained left hamstring, was activated before the game but didn't play. OF Danny Ortiz was optioned to Triple-A Indianapolis.
Matt Wisler pitched two hitless innings in his return to the Atlanta bullpen.
Wisler was recalled from Triple-A Gwinnett after Josh Collmenter was designated for assignment.
Collmenter was dumped after giving up seven runs and three straight homers in the 10th inning of Wednesday's 12-5 loss.
Pirates: RHChad Kuhl(1-4, 5.85 ERA) will start the opener of a weekend series against the New York Mets. He is coming off one of his best showings of the season vs. Philadelphia, allowing one hit in five scoreless innings.
Braves: LH Jaime Garcia (1-3, 4.07) makes his ninth start of the season when Atlanta begins a West Coast road trip at San Francisco on Friday night. He is 3-1 with a 2.88 ERA in eight career appearances against the Giants.
Follow Paul Newberry on Twitter at www.twitter.com/pnewberry1963 . His work can be found at http://bigstory.ap.org/content/paul-newberry .---