Georgios Samaras' headed winner in the 90th minute against Spartak Moscow gave Celtic their first ever away win in the Champions League group stages.
Gary Hooper opened the scoring for the Scottish champions, with Dmitri Kombarov's own goal equalising the tie after Emmanuel Emenike had netted either side of half-time to give the Russians a 2-1 lead.
Juan Insaurralde was sent off for the hosts with the score at 2-1 to Spartak, pulling back Hooper when he was through on goal.
Celtic were good for their win after an indecisive start in the Luzhniki, catching Spartak cold with their first attack of the game.
Mikael Lustig found room on the right hand side, bombing forward from defence to latch onto Victor Wanyama's pass.
His low cross met the front post run of Hooper, who steered the ball across goal and past Sergei Pesyakov with 12 minutes on the clock.
Hooper had the ball in the net for a second time soon after, getting on the end of a chipped Samaras cross from inside the box on the left.
The Englishman however had just strayed too early and his header from close range was chalked off.
Kris Commons then had a shout for a penalty, having his heels clipped in the penalty area but seeing his claims waved away by referee Tony Chapron.
Spartak always looked dangerous moving the ball wide from the middle of the park, with Celtic's defence holding firm until the 41st minute.
Kim Kallstrom's diagonal ball from deep was brilliantly touched down by Ari on the right corner of the six yard box, his touch finding Emenike to turn in from close range.
The Nigerian striker again profited just two minutes after the result. Ari again was involved, his low shot parried by Fraser Forster wide into the path of Demy De Zeeuw, who nudged the ball back across goal to the back post, finding the run of Emenike to steer home unopposed.
Celtic were forced to be patient as Spartak got a foothold in the game, but they were given a way back into the game on the counter.
Hooper bore down Pesyakov just outside the area and, with the keeper advancing, Insaurralde clipped the striker's heels as he went to shoot for goal.
Referee Chapron initially waved away the Celtic man's claims but, following persuasion from one of his assistants, produced a red card for the Spartak defender and awarded the visitors a free-kick which came to nothing.
The numerical advantage allowed Celtic more time on the ball in the middle of the park, with Scott Brown and Kris Commons pulling the strings and retaining possession, waiting for gaps to open up in the wide areas.
It was Commons who started the move for his side's equaliser, threading the ball down the left touchline for Charlie Mulgrew to rush onto.
Playing in midfield for the night, Mulgrew looked up and found Samaras at the near post. The forward, back after an arm injury, allowed the ball to run to Hooper, who turned and held the ball up waiting for the run of James Forrest, on just a minute earlier as a substitute, to come into play.
The young winger took up the invitation and fired in a shot which was blocked by goalkeeper Pesyakov, but only onto the knee of his defender Kombarov and into the net.
Spartak sat deep to repel wave upon wave of Celtic attacks but Neil Lennon's side were not to be denied a maiden away victory in the competition.
Again finding joy down the left flank, Emilio Izaguirre found room to send a swinging cross to the penalty spot and into the path of Samaras, who made a diagonal sprint towards the front post.
The cult hero rose highest to steer the ball to the far post, the ball bouncing just inside the net on the right with exactly 90 minutes on the clock to send the visiting supporters wild, and put Celtic top of the group, temporarily at least, with four points from their opening two matches.
