“Ibiza Town is far more… how you say… in your face in the summer,” says the tour guide casually to her middle-aged charges, who are still gawping at the sign we’ve passed.

The sign, in English, points the way to a nearby sexual health clinic. I won’t spell it out in a family newspaper like this, but you get my drift.

Perhaps it’s that we’re on our way to a convent to buy some of the nuns’ home-baked ear-shaped cakes.

Perhaps it’s that the stunningly pretty old streets around us seem an unlikely backdrop for all-night raves or sex on the beaches.

But as the elderly nun hums to herself and hands us the bag of orelletes through the metal grille, it dawns on me that this incongruity sums up Ibiza.

The 24/7 island, the capital of clubbing, is also a gentle, stylish gem that thinks it’s more Morocco than Majorca and has more than a few surprises up its sleeve.

The so-called white island is just that. Even in February the sun is startling enough to make you sorry your didn’t bother with sunglasses, and the blank cubes of houses that punctuate the lanes look as if they’ve slipped straight off a Le Corbusier blueprint.

Ibiza’s Old Town – Dalt Vila – is a complete history course in itself, with a past that takes in 27 centuries and more than a handful of skirmishes, but has left a gorgeous legacy of houses, a parador and the castle that have earned it Unesco World Heritage Site status.

If you think Oxford’s well off for museums, you haven’t been to Dalt Villa. Honestly, you can barely turn a corner without running into yet another.

After the archaeological museum (small, but with a brilliant cutting into the earth so you can see prehistory literally being trampled on by the Middle Ages) and the little diocesan museum in the cathedral, I’m a bit museumed out – and there are still the museums of contemporary art and weaponry and history and wall-building to go.

However, we have to go, too. We drive out of Ibiza and head off to the place we’ll call home for the next few nights – a pretty Ibizencan farmhouse which sits among fields of almond trees now in full flower.

The Ibiza Tourist Board, mindful that middle-aged people like me imagine Ibiza is only for the young, single and skinny, are using the island’s famous pink almond blossom as a reason to visit out of season.

Now, I like branches with flowers stuck on them as much as the next man, but there are scores of other reasons why you should do Ibiza before you die.

The food, for a start. At one cliffside restaurant, Es Boldado, we watch for the goats that inhabit the huge rock – or boldado – out across the water, while making short work of the salad of braised peppers and tuna, followed by kilo upon kilo of mussels.

Aha, and here comes the main course! A massive paella, enough to feed the entire restaurant, is wheeled out to our table, the trolley complaining bitterly under the weight of the huge gambas and crab claws.

Thank God there are only two desserts to tackle – a cinnamon egg custard and Ibiza’s own dish, Flao – a rare aniseed-flavoured flan made of goat’s cheese.

At another, we join villagers for Sunday lunch, and again they forget to count the number of courses, so while the wiser of us opt for something light, like two starters of mushrooms with prawns and local wild asparagus plus cod, the more rash among our party order the mixed grill, which consists of every single cut of meat of every single farm animal you could imagine.

Plus rabbit. For each diner.

And then helpings of Flao all round.

A word of warning. If you stayed on Ibiza longer than a week, Ryanair would probably charge you an excess on the way back for exceeding your body mass index.

But of course it’s not all about consumption. There’s plenty of opportunity for activity, too. I notice many people hiking and biking from my seat in the sun outside a village bar. Try the Bar Costa in Santa Gertrudis for a real flavour of Ibiza. Penniless artists used to pay for food there with their paintings in the 1960s, and the oils and watercolours now hang all over the walls, in between huge cured hams. The result is pleasing if unexpected – a bit like if Van Gogh’s dad had run a pork butcher’s.

I’m even persuaded against my better judgment to take part in the island’s latest foray into the exotic – Nordic walking. There’s that incongruity again. Yes, the council is encouraging its citizens and visitors to get active by… er… aping Scandinavian shepherds, and handing out free equipment and advice on how to put one foot in front of another without tripping up.

Given my allergy to exercise I’m pretty nervous I’ll get left behind, but in the end it turns out to be six kilometres of nothing more than strolling with sticks along a lovely sea-shore, and when I reach the finish I swear I feel a trickle of adrenaline.

There’s no better way than relax the tortured body than with a sauna, steam and swim, and happily several of the agroturismo – countryside or farm-based tourist accommodation – offer just that.

The spa holiday here is low key, discreet and all the better for it, and as I lie in the February sun, cocooned in a fluffy bathrobe and toying with a plate of Flao, I wonder how this lovely small island has kept under the radar of food-loving, sun-seeking, freewheeling over-40s for so long.

Anyway, may it continue – chic, friendly, unspoiled and uncluttered.

So don’t go. It’s rubbish.