They don't do smooth public relations at Ruskin College. It may have been around for 106 years and have close links with serious political movers and shakers, but, happily, Ruskin remains steadfastly untouched by spin or attention to image.

The thought struck me, as I joined the principal of Ruskin, Prof Audrey Mullender, on a grand tour of the college's magnificent site in Old Headington, which she hopes to transform into Ruskin's new headquarters.

The scheme will cost £20m, feature a £5m library to be named after former Prime Minister Jim Callaghan, and both Gordon Brown and Tony Blair, no less, have agreed to act as joint honorary presidents of the virtual fundraising committee.

But there were to be no directors of communications or development officers in attendance as Prof Mullender led me around the 20-acre site, which Ruskin acquired in 1946 as its second home.

Instead, we were to be joined by a gang of Ruskin students, some determined to show their opposition to the college's plans to sell off its city-centre headquarters in Walton Street, in order to fund its Old Headington scheme.

The protesters waved banners which demanded "Ruskin Belongs to the City of Oxford" and "Keep Walton Street Open," as the principal set out details for four new buildings and restoration work.

Not even the constant interruption of one 'mature' student, who inexplicably kept asking "so, where are we going to put our multi-storey car park?" could darken her mood, as we approached the college's listed Crinke Crankle wall.

Perhaps, like me, she was beginning to fear that he might actually have been serious.

It had been the same story a few days before, when Lord Callaghan's daughter, Baroness Jay, had gone along to launch the £5m fundraising appeal for the new Callaghan Library, to commemorate her father's famous "Great Education" debate speech at Ruskin.

Needless to say, 'Sunny Jim' was heckled on that celebrated occasion, nearly 30 years ago, by some Ruskin students, with the old stager putting the whipper snapper militants in their place when they began chanting The Red Flag. "They don't even know the words," he had teased them. "I would be delighted to sing it for you."

No doubt about it, that as well as being "the college of the people", Ruskin can claim to be Oxford's college of protest.

But, then, its former students have included the former Deputy Prime Minister, John "Two Jabs" Prescott, and Dennis "The Beast of Bolsover" Skinner.

The protests will be focused on matters close to home in the coming months, as Ruskin moves forward with its masterplan, which the governing body hopes will secure the college's long-term future.

For opponents of the scheme, the battle is for the college's soul.

For years now Ruskin has been agonising over where its main home should be in Oxford, given the hefty bills for the upkeep of two impressive properties.

Teaching staff and students were in open revolt two years ago when the college proposed uprooting Ruskin from its two historic homes to a Berkeley Homes site in North Oxford.

The college proposed selling off its sites in Walton Street and Old Headington to fund a multi-million-pound relocation to a purpose-built, three-storey building on the old Unipart factory site.

Ruskin students and lecturer groups maintained the former factory site was contaminated, while arguing the partnership with Berkeley Homes amounted to a betrayal of the college's inheritance.

The idea was dropped "for financial reasons", with the college principal, Jim Durcan, handing in his resignation within weeks.

His successor, Prof Mullender, however, said the college simply could not go on trying to make do.

She said: "Ruskin has been in a make-do-and-mend mode for a decade. Serious investment must be put in.

"The two sites duplicate everything. They both have teaching, catering and residential facilities. It is not an efficient way to run a college. All the buildings need refurbishment and people having to move back and forth is not ideal."

Walking around the grounds of Old Headington with the principal and the protesters, it is all too easy to see why this is the site they have decided to keep. At its centre is the Rookery, a fine, grade-II listed Georgian building, still impressive despite the ugly modern buildings that have been attached to it.

The campus contains two other grade-II listed buildings, Stoke House, acquired in 1962, which has already had £330,000 spent on it, and the historic Crinkle Crankle Wall, part of a walled garden that dates from 1733.

It comes as a surprise to see that the grounds sweep all the way down to the Oxford ring road. The masterplan proposes four new buildings, ugly modern buildings such as Bowen House and the Bowerman Building being demolished and the remaining buildings all being refurbished.

No one appears to oppose this aspect of the scheme.

It is rather the price that Ruskin is prepared to pay to get it that is causing anguish.

For if Old Headington has the trees, many students, past and present, are more concerned about the roots of Ruskin - and they undoubtedly lie deep in Walton Street.

Many assume that the Working Man's University must have been created by the likes of Keir Hardie, the National Union of Mineworkers or some North of England co-operative.

But it was, in fact, created by two young Americans, Charles Austin Beard and Walter Vrooman.

It was Vrooman's fabulously wealthy wife who donated the money for the main Ruskin College building. (In its early years, the college had rented its accommodation in St Giles'.) Ruskin was created with an aim as simple as it was revolutionary - to set up a university for the working classes, where dockers, miners and labourers of all kinds could enjoy an education that had previously been the preserve of a privileged elite.

Because it was founded as a response to the perceived elitism of Oxford University, its position in the university heartland was, and remains, hugely symbolic.

The secretary of Ruskin Students' Union, Peter Russell, said: "Ruskin was founded in the heart of Oxford for a reason. It was deliberately put there to give working class people the benefits that come with that academic situation. It is still the same today."

But there are important practical benefits of being a neighbour of an intellectual powerhouse.

Walton Street provides Ruskin students with easy access to the Bodleian Library, also allowing them to attend lectures and even enjoy membership of the Oxford Union.

The loss of its city-centre base, he fears, would loosen these precious ties with Oxford University.

"There is also the general vibe of being in central Oxford, simply joining in conversations in pubs," he added.

Some have detected a whiff of snobbery. Perhaps the Ruskin students have come to relish their place among the academic elite a little too much?

Mr Russell dismisses any such suggestion out of hand.

"No, that would be against the whole ethos of Ruskin as a working-class college. I'm not aware of anyone who pretends to be part of Oxford University. People are proud to belong to Ruskin and have no wish to bask in reflected glory."

He believes people on Ruskin's short-term courses, such as the Ransackers course, for the over 55s, would be particularly hit if everyone was forced to travel to Dunstan Road, in Old Headington.

While there are 260 full-time students, with about 90 in residence, some 2,650 are part time.

"The opposition to the sale of Walton Street has nothing to do with the development at Old Headington," said the student union secretary. "Everyone agrees with that. We simply do not believe it should happen at any cost."

He points to the fact that the majority of lecturers (at least those who belong to a union) and former students who belong to the Ruskin Fellowship, are also strongly opposed to the loss of Walton Street, which offers accommodation for 65 students, a lecture room for 150, a dining room that seats 70 and four seminar rooms.

The president of Ruskin College Fellowship, John Hurlston, has been a prominent opponent of its sale.

"To sever its presence in Oxford itself would mean that Ruskin College would be like any other further or higher education college and its uniqueness would be lost forever."

The principal says one option being explored is to lease back a small part of the Walton Street building after its sale.

"It would have an outreach function and allow us to retain a city-centre presence."

Prof Mullender believes shortage of space on Walton Street is already presenting major problems. Like the New Bodleian down the road, she points to the fact that Ruskin's library has already run out of space and is not fit for purpose. Over a third of the library's holdings is now having to be kept in store and the new Callaghan Library would form the centrepiece of the whole development.

As a Prime Minister who never went to university, Lord Callaghan would have relished the gesture.

But he would never have expected to receive such an honour from Ruskin without a very public row.