The stuff that makes up the bulk of the universe is a complete mystery to us...

... says Radio 4's quide liderally awesome the Cosmic Hunters. The programme talks about a new 'gaping hole in fundamental physics' that's given physicists and cosmologists a few more hundred year's scope for funding applications.

There's 'normal stuff' - about 4% of all the material in the universe, which we know a fair bit about. There's maybe 30% dark matter. It has gravitional effects we can observe, so we know its there... er, if our theory of gravity is right. And the rest is (possibly) dark energy. Roughly 70% of all the universe - and we have no idea what it is.

What's the visible evidence for this dark energy, then? Discovered by two separate teams who were looking for something completely different, its influence on the universe is dramatic.

They were using supernova explosions to detect the rate at which the universe is expanding - each wanted to be the first to measure the change in speed of the expansion of the universe. They used red shift to detect the change of size of the universe, and their brightness to detect distance, which tells us how long the light has been travelling. The two together indicate how much the universe has changed in size since that supernova exploded. So this means its possible to get a measure of the expansion history - the growth spurts - of the known universe.

Very large detectors - observing many many galaxies in one shot - made this possible, since there's only about one supernova explosion per galaxy per hundred years.

But! Dum Dum Duuuum... both teams discovered something they weren't expecting. Rather than slowing down, the universe's expansion is speeding up. Current models of gravity didn't fit this, so what does a scientist do in these circumstances?

Most people don't realise that what you do as a scientist is spend most of your time trying to figure out every possible way that you could be making a mistake. And so we spent a good part of a year looking through our data in many, many different ways and try to check what we were doing. And so finally we came to the conclusion - y'know, it looks really like this universe is speeding up, and we're finally ready to announce these results - this was towards the end of 1997 that we finally came to this conclusion.

Both teams reached the same conclusion. A nice idea, that: scientists mainly occupied by thinking 'what have I done wrong here?'

What kind of force in the universe would account for that speed? One team's model suggested - based on a universe of Einsteinian gravity - that 'negative matter' was one option. He thought this was impossible, until remembering Einstein's idea of the cosmological constant. Repulsive gravity, when put into the model, worked. It was a good fit. There was plenty of initial scepticism - but same results found again and again.

Still - sounds a bit ad hoc, don't it? So - two possibilies for physicists. They're a) looking for an explanation of 'dark energy'...

For the first half of the history of universe - where gravity was the strongest long-distance force - growth slowed, until a certain point after several billion years where gravity seems to have become weaker than this 'dark energy' - so that is now forcing things apart faster.

Dark energy, some say, may emanate from the quantum froth of the vacuum of space as virtual particles leap in and out of the fabric of spacetime.

Unfortunately, this quantum froth model leaves the modellers with 10 to the 120 times too much energy compared to observations. Oop. Which leads to another possibility... b)

The theoreticians are so wrong now that some astromers are suggesting a radical alternative explanation for the accelerating expansion of the universe. Dark energy may be an illusion - and perhaps instead long cherished theories of gravity need re-thinking.

Egghead 1:

Its a mystery that's beyond understanding the universe and how it evolved. It actually goes to the very fundamental laws of physics that we work with. The big two achievements in twentieth century was Einstein's theory of how gravity works - the general theory of relativity - and quantum mechanics. And we know that these two theories contradict each other at some level, and this dark energy problem is right at the nexus of this conflict.

Egghead 2, making patent-hungry claims that may or may not be true. A little hard to tell, since we don't have the theory, but they probably deserve to be allowed a little hyperbole:

So dark energy, and understanding it, may push us to deeper level of understanding in physics. And the spin-offs of that - the sky's the limit. There's almost no limit to the kinds of technologies and the kinds of problem that could be affected.

What happens next? Well, the teams are competing to get their instrument design to be taken up by NASA - so they can get much, much more detailed pictures, taking in huge quantities of extra supernova data. They reckon they'll be able to actually build a precisely grained picture of the universe's growth speed - slowing, speeding up, growing in waves?

This is potty:

when the big bang happens, think of the universe as a drum, which you've hit really hard. You get that first overtone - that 'boom!' Well that basically has put a wave in the universe that makes certain parts of the universe denser, other parts less dense. It's like a wave in a water-pond. Well, as that wave moves out, the spacing between the top and the bottom remains constant - so if you can just figure out a way to measure that, then you would have an absolute measuring rod for the size of the universe as a function of time. And that technique was just pioneered in the last year and a half.

Now, sadly, NASA is going to be choosing only one of the dark energy instrument projects. They've all been working on their projects for about a decade. Two out of three will lose. One guy says of this:

Along with developing one's scientific skills for a project, you also have to try and learn to become a zen master and I keep thinking that, at the end, you reach some sort of calm ability to take a complete catastrophe, and say, well - we did a good job. I somehow suspect it won't quite feel that way if that happened...

So - another example of just getting to the point of thinking we might, maybe, be getting a handle on the way things worked. The odd scientist foolishly saying, 'not much more to discover - just gotta tie up some loose ends... oh, bugger.'

Finally - hope for the merely diligent:

Like any student at university, I thought discoveries were only made by people in old black and white photos, y'know, with crazy hairstyles and platt pants from the 1940s - and discoveries aren't really made any more, because we've discovered everything. And when they are made they're made by people like Einstein, who are so smart that he has three discoveries before breakfast. So to me - I guess I think of myself as a fairly regular person. That regular scientists could make a really exciting discovery just seems to be really lucky to me.

The older I get, the crosser I become about the fact that I won't be here to see the end of it - whatever that might be. We only get to see the most wafer-thin sliver of the progress of human knowledge, and then we're wormfood. I shall have to write a stern letter to someone about this.