As the clock ticked down to D-Day the atmosphere in the central London office of Britain’s Secret Intelligence Service, MI6, reached feverish anticipation. Years of carefully calibrated deception, casting spells over the German generals defending the landing grounds earmarked for Operation Overlord was to be finally tested in the largest amphibious invasion ever undertaken.

For the men and women of British intelligence it was the culmination of everything they had been working towards. Failure was unimaginable. If the Germans discovered the destination of the Allied armada, tens of thousands of British, US and Commonwealth troops would be slaughtered on the Normandy beaches and the war would drag on for years. The difference between defeat or victory depended on Hitler believing the Allies’ intended target was the Pas de Calais, 200 miles further down the coast.

Given such high stakes and such high drama, it seems incredible one British intelligence officer involved in the deception campaign crucial to Operation Overlord would walk out on the service before a single ship had set sail for France. Before a single bullet had been fired. And before victory had been assured.

But on June 2, 1944, three MI6 counter-intelligence officers arranged to have lunch in the Café Royal, central London. Two of them were old Westminster School friends: Kim Philby who headed up MI6’s counter-intelligence section and Tim Milne, nephew of Christopher Robin author A.A. Milne and Philby’s loyal number two. The third officer was Graham Greene who was in charge of MI6’s Portuguese desk.

The three spies made a good team. Philby and Milne were proud and protective of their most famous recruit, the celebrated author, Greene. In turn, Greene was grateful for Philby’s loyal support during his troubled tour of duty in Sierra Leone and a damaging row with MI5 over an intelligence operation in the Portuguese-held Azores.

When the three British intelligence officers gathered at the bar in the Cafe Royal, Greene told his MI6 colleagues that the lunch was his treat. Perhaps he hoped this act of generosity would help sweeten the bitter news he was about to deliver.

As they tucked into their food, Greene told his friends he intended to leave the service immediately. He made it plain his mind was made up and there was no budging him. Philby and Milne were mystified and tried to persuade Greene to delay his decision until after D-Day. Greene refused to change his mind.

His explanation for leaving MI6 on the eve of its greatest triumph and with many agents run by Greene still in play in Portugal and the Azores, is not altogether convincing. He claimed Philby had offered him a promotion he didn’t want to accept and told them he was not happy with his new responsibilities. While he didn’t mind “chivvying Portuguese on orders from above… he had no intention of personally doing so”.

Greene’s decision to leave the service at this critical juncture has baffled Greene’s biographers for decades. Had the author suspected Kim Philby of working for the Russians and confronted Philby with his suspicions? And had he resigned rather than betray his friend?

Graham Greene and Kim Philby had met two years earlier in London 1942 as MI6 officers. Books like Brighton Rock, The Power and the Glory, The Ministry of Fear, meant Greene was already a household name. After the war, The Heart Of The Matter, Our Man In Havana and The Quiet American made him the greatest storyteller of his generation while his cinematic prose in films like The Third Man won him world-wide audiences.

Philby meanwhile was the most successful Soviet agent of the 20th Century, the first man of the Cambridge spy ring who worked as head of British counter-intelligence before defecting to Moscow in January 1963. Greene was extraordinarily intuitive and his razor sharp dissection of character meant he could read people and countries better than any other writer. Had he by June 1944 read Philby? Had Philby given himself away?

After Philby defected, Greene controversially stuck by Philby and famously said he preferred loyalty to friends over loyalty to countries. Greene however remained on the unofficial books of MI6 and secretly passed on any useful intelligence he gleaned from his travels to the world’s political hotspots. Throughout all these years he kept in contact with his old boss now living in Russia, writing Philby dozens of letters and passing on the replies to the head of MI6. In 1986 he travelled to Moscow to secretly visit Philby in his flat off Pushkin Square.

They remained close right up until Philby’s death in 1988, weeks after Greene had last seen him in Moscow. Neither man ever disclosed the content of their post-war private discussions. By then the spycraft they had acquired was instinctive and deeply embedded. Greene had used it to great effect to keep biographers off his scent while

Philby had deployed it in the service of the KGB at the height of the Cold War and well into the thawing of the Glasnost era.

In his books, Greene continuously wrestled with the twin themes of loyalty and betrayal. Whether he was writing about religion, relationships or espionage, it was always possible to sense the presence of Kim Philby. In turn Philby, throughout his exile in Russia, wrote to Greene, his only link to his espionage past, seeking an understanding of the old country he had been forced to leave.

It seems the two men were never able to escape each other. Perhaps they were fascinated by the inscrutable nature of the other. Yuri Modin, Philby’s KGB handler after the war, didn’t believe anyone (British intelligence, the Soviets and the women he loved) had “ever managed to pierce the armour” that protected Philby’s innermost self.

The same could be said of Greene. Indeed he said it himself: “If anybody ever tries to write a biography of me, how complicated they are going to find it and how misled they are going to be.”

In their attempts to protect, preserve and enhance their legacies the past became ever more secret. Greene’s lover Yvonne Cloetta said his real secret was “his passion for secrecy”. Only now, through the discovery of the correspondence and files from the cases they worked on, is it possible to shine a light on the spying lives of these two MI6 officers – the traitor and the writer.

Greene had been recruited to MI6 in August 1941 through his sister Elisabeth’s MI6 contacts. He was initially posted to Sierra Leone but did so well that he was brought back to ‘the office’ in March 1943 where he began working for the Iberian section, under Philby.

Greene was already an established literary figure while Philby had acquired mythical status as a foreign correspondent during the Spanish Civil War, having been personally awarded a medal by General Franco after his jeep was blown up while reporting from the front line.

The two men hit it off and ran secret agents and operations that made a considerable contribution to the Allied war effort, including Operation Torch (the invasion of North Africa), the British occupation of the Azores and the degrading of the Abwehr spy machine on the Iberian peninsula.

Greene was fiercely proud of his service to his country as an MI6 officer but rarely spoke or wrote about it. He strictly observed the highest tenet of the Secret Intelligence Service, enforced by the Official Secrets Act that an intelligence officer must never talk about their work. But did he really believe loyalty to his friend the traitor Philby trumped loyalty to Britain and MI6?

In the 1970s and 80s, MI6 would have loved to have brought Philby back in from the cold and scored a huge propaganda coup against the KGB. Moscow had often considered the possibility that Kim Philby, Russia’s most successful penetration agent, had been a plant, working for SIS all along.

As Greene approached his own death in 1991 the author wondered the same thing and questioned whether Philby had played the ultimate betrayal on Greene, secretly working for British Intelligence while setting up Greene as the innocent dupe.

That would make Philby a triple agent.

Greene is reported to have been so disturbed by this notion that he spent the last days of his life re-reading his correspondence with Philby, searching for clues as to where his friend’s true loyalties lay.

But typically for a man whose secrecy was so entrenched, having died aged 86 in April 1991, he went to his grave without revealing his final conclusion.

• The Writer and the Traitor by Robert Verkaik (Headline, £22) is out now