These days, you are more likely to find Roger Bannister on a golf course than a running track. The man who raced into the record books back in May 1954 laughingly admits to playing golf "of a sort", and seems far more interested in telling me about his family than his own sporting heroics.

But if Bannister is willing to dismiss his achievements so lightly, the rest of the world doesn't seem ready to do so just yet. Ever since he became the first man to run a mile in under four minutes a feat that many had deemed impossible he has been revered throughout the sporting world and beyond.

Even though the record was broken by Australian John Landy less than a month later, and has subsequently been broken several times by the likes of Coe, Ovett and Cram, it is still Bannister's name that dominates.

The fact that the 50th anniversary of his record-breaking race was marked by an action replay at Iffley Road, followed a few days later by the unveiling of a Blue Plaque, is testament to that.

It is easy to imagine that Bannister who turned 77 in March has always attracted this widespread admiration. So it comes as a surprise to discover that his early years were overshadowed by setbacks, disappointments and criticism from both press and public.

He was born in Harrow but brought up in Bath, where he struggled to gain acceptance at school due to his studiousness and lack of prowess at the more macho' sports. But he proved his worth on the running track, and began to earn the respect of his peers.

His first taste of competitive running was as a schoolboy, but it was when he came up to Oxford, in the autumn of 1946, that he began to take it more seriously.

One of the first things he did as a freshman was to join the University Athletic Club, combining weekly training sessions with his medical studies at Exeter College although he admits to "not doing much work" at this early stage in his student career. Oxford was too exciting, too full of pleasurable distractions.

In May 1947, Bannister represented Oxford for the first time at White City, winning the Oxford-Cambridge mile a feat he repeated the following year.

He soon caught the attention of the media, but unfortunately he also drew criticism for his decision not to represent Britain in the 1948 Olympic Games, which he felt were still beyond his capabilities.

Waiting for the 1952 Olympics, he reasoned, would allow him to further develop his skills, and also to complete his studies.

After capturing the British title for the mile in 1951, Bannister was confident about his chances in the Olympics in Helsinki the following year. But in the 1500 metres, which he was widely expected to win, he slid ignominiously into fourth place, fuelling yet more criticism from a sceptical press.

Banner could have decided, at that point, to retire from competitive running and concentrate on his medical career. But he felt he had unfinished business. He had to quit on a high. And so the idea grew of attempting to beat the world record for the mile. Did the attitude of the British press contribute to this resolve?

"Yes, I suppose it did," he acknowledges. "I came fourth in the 1500m, people were expecting me to win it, and I was very disappointed. There were reasons why I didn't win it, some of which were outside my control, but I decided I wouldn't give up at that point because everybody had supported me and told me what promise I had."

And so the stage was set for one of the most dramatic scenes in British sporting history. On 6th May 1954, Bannister competed for the British Amateur Athletic Association at Iffley Road. With close friends Chris Chataway and Chris Brasher as his pacemakers, he managed the first three laps in less than 3 minutes. Tension mounted as he began the final lap, and continued to rise as he raced towards the finishing line breaking the tape within the record time of 3:59:4, before collapsing onto the track. At the age of 25, Roger Bannister had broken a record first dreamed about by the Victorians.

A month later, in Finland, John Landy ruthlessly snatched away Bannister's crown when he completed the mile in exactly 3 minutes 58 seconds. But in August, Landy and Bannister were pitted against each other at the Empire Games in Vancouver, and this time Bannister triumphed in 3:58:8 58 seconds ahead of his rival. Three weeks later, he won Gold in the 1500m at the European Championships, which he decided should be his swansong. He had more than redeemed himself in the eyes of the press and public, and had secured a place in the history books.

So it was exit Roger Bannister, sportsman, and enter Roger Bannister, neurologist. He went on to become eminent in his field, and was for many years a Consultant in Neurology at St Mary's Hospital Medical School. In 1985, he was invited to be Master of Pembroke College, a post he held for 8 years.

But he has retained an active interest in sports issues, and became the first Chairman of the Executive Sports Council in 1971. Over the years he has campaigned for greater public investment in sports training and facilities in Britain, and pioneered drug-testing among sportsmen and women. "I regard this as my greatest contribution to sport," he says.

Now settled in Oxford, with his two adult daughters and their families living close by, Bannister is clearly content to pass his twilight years in the embrace of the city that witnessed his great triumph. "There are many good reasons why we are here," he says. "I've got many close friends, and links with three Oxford colleges. I love living in Oxford."

Further reading: The First Four Minutes by Roger Bannister (Sutton Publishing, 50th anniversary edition, 2004), £7.99