Superintelligence · sibolaget.se

The last invention.

Superintelligence may become the most important thing ever to happen to humanity. SI-bolaget exists to help make it the best, too.

01Vision

Our vision

For all of our history, we have been the smartest beings on Earth. That may soon change. Superintelligence is not the next app or the next gadget. It is the next chapter in the history of intelligence. SI-bolaget exists so that this chapter is written with open eyes, steady hands and all of humanity on board.

02The concept

What is super­intelligence?

An intellect that vastly outperforms the best human minds in practically every field.

Definition adapted from philosopher Nick Bostrom, 2014

It could take at least three forms

ISpeed

Faster

A mind that thinks like ours, only thousands of times faster. At 10,000× speed, a year of thought takes less than an hour.

IICollective

More

Millions of connected intellects that together far surpass any single mind. An entire civilisation of thought, housed in a data centre.

IIIQuality

Smarter

Not just faster, but fundamentally smarter. A mind that sees patterns we cannot even imagine, as far ahead of us as we are ahead of chimpanzees.

03The physics

The brain is a masterpiece. But it has limits.

The human brain is the most advanced thing we know of, shaped by evolution over hundreds of millions of years. But it is bound by the rules of biology. Machines are not.

25 000 000×

Faster clock

Neurons rarely fire more than about 200 times per second. A modern processor ticks around five billion times per second.

Neuron200 Hz
Processor5 GHz

1 700 000×

Faster signals

Nerve signals travel at up to 120 metres per second. Light in optical fibre: around 200,000 kilometres per second.

Nerve120 m/s
Fibre200,000 km/s

86 000 000 000

Neurons in your brain

Impressive, but the count is capped by a skull and a few litres of blood. A data centre has no skull.

Brain≈ 1.3 litres
Data centrescalable

20 W

Here, the brain wins

Your brain runs on about as much power as a dim light bulb. A single modern AI chip can draw 1,000 watts or more. Efficiency is biology’s trump card, for now.

Brain20 W
AI chip1 000+ W

Bars use a logarithmic scale. Values are rounded and indicate orders of magnitude.

04The intelligence explosion

When intelligence improves itself

In 1965, mathematician I. J. Good described a machine smart enough to design an even smarter machine, which in turn builds a smarter one still. The result is an intelligence explosion. The first such machine, he wrote, would be

“the last invention that man need ever make”.

I. J. Good, 1965

Provided, he added, “that the machine is docile enough to tell us how to keep it under control”.

1.10

How much each improvement makes the next one easier.

×20

How fast the system does research, compared with human researchers.

In the model: from human level to superintelligence

5.0 years

Moderate takeoff

Months to years. Little time to react, but some.

Toy model: dI/dt = k · Iα, from human level (1×) to 1,000×. Not a forecast, just a way to get a feel for the dynamics. On a logarithmic scale, exponential growth is a straight line.

05The scale

The intelligence scale

This is how we tend to picture intelligence: you on one side, Einstein on the other.

But zoom out, and the gap between the two of you shrinks to almost nothing.

Superintelligence need not stop just above Einstein.

It could end up here.

A conceptual illustration based on an idea by Eliezer Yudkowsky, not a measurement.

06History

The road here

More than seventy years from a simple question to one of the great projects of our time.

  1. 1950

    Can machines think?

    Alan Turing publishes “Computing Machinery and Intelligence” and proposes the imitation game, which we now call the Turing test.

  2. 1956

    The field gets its name

    The Dartmouth workshop gathers the pioneers and gives the field its name: artificial intelligence.

  3. 1965

    The intelligence explosion

    I. J. Good describes how a machine smarter than us could design ever smarter machines, again and again.

  4. 1997

    Deep Blue

    IBM’s chess computer defeats world champion Garry Kasparov in a six-game match.

  5. 2012

    Deep learning wakes up

    The neural network AlexNet crushes the competition in the ImageNet challenge and kicks off the deep learning breakthrough.

  6. 2014

    Superintelligence

    Nick Bostrom publishes the book that makes superintelligence a question for researchers, companies and policymakers.

  7. 2016

    AlphaGo

    DeepMind’s AlphaGo defeats Lee Sedol at Go 4-1, a feat many experts believed was a decade away.

  8. 2017

    The Transformer

    The paper “Attention Is All You Need” introduces the architecture behind today’s large language models.

  9. 2020

    AlphaFold

    AI effectively cracks the protein-folding problem, one of biology’s grand challenges for half a century.

  10. 2022

    ChatGPT

    A language model enters everyday life and, by some estimates, reaches 100 million users in two months.

  11. 2024

    Nobel Prizes in Stockholm

    Both the physics and the chemistry prize honour research on and with AI, from neural networks to AlphaFold.

  12. 2025

    Gold in mathematics

    AI systems reach gold-medal level at the International Mathematical Olympiad.

  13. 20??

    Superintelligence

    The next line has not yet been written.

07Sweden

The Swedish roots of super­intelligence

Sweden plays a surprisingly large part in the story of superintelligence, from philosophy to the Nobel Prize.

Helsingborg56°03′ N · 12°42′ E

b. 1973

Nick Bostrom

The philosopher from Helsingborg whose book Superintelligence (2014) put the concept on the map.

Stockholm59°20′ N · 18°04′ E

b. 1967

Max Tegmark

The physicist from Stockholm who asked in Life 3.0 (2017) what future we actually want with AI, and who co-founded the Future of Life Institute.

Stockholm59°20′ N · 18°04′ E

2024

The Nobel Prize

The Royal Swedish Academy of Sciences honoured research on and with AI in both physics and chemistry: neural networks, computational protein design and AlphaFold.

08Principles

Six principles for what comes next

Superintelligence is too important to leave to chance. This is what we stand for.

  1. 01

    Safety before speed

    A mind smarter than ours must share our values before it becomes powerful, not after.

  2. 02

    Everyone’s right to intelligence

    Just as Sweden’s allemansrätt opens nature to everyone, the benefits of superintelligence should reach everyone.

  3. 03

    Openness

    What affects everyone should be open to scrutiny by everyone.

  4. 04

    Humans at the centre

    Technology should serve people and strengthen their ability, freedom and dignity. Never the reverse.

  5. 05

    Humility

    No one knows everything about what is being built. That is exactly why we must tread carefully.

  6. 06

    The long view

    We think in centuries, not quarters.

09Questions

Straight answers to big questions

What we know, what we don’t, and why it matters.

Further reading

  • SuperintelligenceNick Bostrom · 2014
  • Life 3.0Max Tegmark · 2017
  • Human CompatibleStuart Russell · 2019
  • The Alignment ProblemBrian Christian · 2020
The glossary: AI and superintelligence from A to Z Article: What is AGI, really?
Does superintelligence exist yet?

No. Today’s AI systems beat us in specific areas, such as chess, Go and predicting the shapes of proteins, and they keep getting better at writing and programming. But no system outperforms the best humans in practically every field. That is where the line for superintelligence is drawn.

What is the difference between AI, AGI and superintelligence?

AI is an umbrella term for systems that handle tasks requiring intelligence. AGI, artificial general intelligence, is a system that can learn and do roughly any intellectual work a person can. Superintelligence is the step after that: an intellect that vastly outperforms us in almost every field. Whether that step takes decades or days is one of the big open questions. Try it yourself in the simulator. More about AI and about AGI.

When could superintelligence arrive?

Nobody knows. Estimates range from a few years to never. In one of the largest surveys of its kind, with 2,778 AI researchers in 2023, the combined estimate gave a 50% chance that unaided machines will outperform humans at every task by 2047. In a similar survey a year earlier, the same estimate was 13 years later.

Source: Grace et al., Thousands of AI Authors on the Future of AI, 2024

Is superintelligence dangerous?

It could become the best thing we ever build, or the most dangerous. The main worry is not malice but a system far smarter than us pursuing goals that differ from ours.

Researchers are divided. In a 2023 survey of 2,778 AI researchers, 68% thought a good outcome from superhuman AI more likely than a bad one. Yet between 38 and 51% gave at least a 10% chance of an outcome as bad as human extinction. That same year, hundreds of researchers and leaders, including Geoffrey Hinton and Yoshua Bengio, signed a statement that reducing this risk should be a global priority.

Sources: Grace et al., 2024 · Center for AI Safety, Statement on AI Risk, 2023

What does alignment mean?

Alignment is the research into making AI systems want what we actually want, even as they become smarter than us. It is harder than it sounds: describing human values so precisely that a machine cannot misread them has proved one of the field’s hardest problems. That is why safety comes first among our principles. We go through the research in the article Can we control something smarter than us?

What can I do?

More than you might think. Learn how the technology works, talk about it, and demand openness and safety from those who build it and from those who set the rules. The EU AI Act, which entered into force in 2024, shows that democracies can set limits even for the most advanced technology.

10Contact

The future is not yet written.

Want to explore, build or debate the intelligence of the future? We’d love to hear from you.

We’ll reply. With natural intelligence.