Payton Tolle had just retired Paul Goldschmidt for the third time on one pitch, running his pitch count to a manageable 83 with two outs in the fifth inning Tuesday, when Chad Tracy decided he’d seen enough.

With Boston trailing by one run and Ben Rice, who doubled in the first inning, due up next, the manager opted to remove his pitcher who had settled into the pitching duel by that point.

And it backfired in horrible fashion for Boston.

Lefty reliever Erik Miller immediately surrendered a long solo homer to center field to Rice — the first of his two against Boston relievers — and it completely changed the tone of the game in Boston’s 9-0 loss to the Yankees in Game 1 of the AL Wild Card series at Yankee Stadium.

Boston’s b-tier relievers ultimately surrendered eight runs in 3 1/3 innings.

“Not just the pitch count necessarily, but you’re trailing already, you’re moving through the third time,” Tracy said. “Rice had already hit a double in the gap off of him, and you have a nice lane there for Miller, and obviously he got him. Just felt like it was the time to go.”

This postseason marks Tracy’s first calling the shots and moments like this one Tuesday night are what ultimately can swing series.

Sure, Boston did not score, but being down 1-0 to Cam Schlittler and the Yankees bullpen is a vastly different world than trailing by two entering the seventh.

Tolle allowed four hits over the first nine batters but settled down to retire eight of the next 10 batters and navigated a two-on, no-outs situation in the fifth while trailing 1-0.

He began the sixth by retiring Austin Wells on six pitches and then sitting down Goldschmidt, leaving him certainly with room to possibly navigate six innings if he worked effectively.

Tracy, though, opted for a new look in Miller, ending Tolle’s day after 4 2/3 innings of one-run ball.

“I thought he threw the ball well,” Tracy said. “They did an effective job of grinding the count up as well. He had traffic. They put traffic on him. He was able to navigate his way through that. I think he competed extremely well for the time that he was out there and gave us what he could.

“I think the number being up there in the mid-80s, still trying to get through five, probably higher than what you normally see with Payton, and to be expected in an environment like this. But I thought he competed his butt off.”

Miller, a midseason trade acquisition, struggled to end the regular season and those issues carried over to the postseason when he fired a middle-middle 97-mph heater that Rice hit out to dead center.

Tolle said that as a competitor, he felt he could have gone longer but understood the decision due to the strength of Boston’s bullpen during the regular seaosn.

Boston is now one loss away from a second straight early postseason exit at the Yankees’ hands.

“Wasn’t good enough,” Tolle said. “They had a good plan and I threw a lot of balls … just didn’t feel like I could command the zone, kind of let some at-bats get away from me. A lot of pitches.”

He added about the bullpen: “Those guys picked me up a lot this year.”