The Precipice
The Precipice by Toby Ord is a great birdseye view of the biggest challenges that we humans face in the future. Challenges that have the possibility of extinguishing our potential, threats that may make us go extinct. As could be predicted, these threats are currently mostly man-made.
I found this review by Scott Alexander very good. Do read it if you want to get a good overview of the whole book.
Also see this review by Theron Pummer on Notre Dame Philosophical Reviews.
The biggest things that stood out to me were:
- Risk from AI is one of the largest (1 in 10 change in next 100 years)
- I can agree on that point, but also think that there are some very good arguments that we will create benevolent AI or at least AI that has ‘good’ goals and that we can manage that
- See Human Compatible for more about AI
- Engineered pandemics seem very relevant at this moment (1 in 30)
- Killing everyone would seem difficult, but it could break down society by killing 99% (or even 70% I would guess)
- The knowledge seems difficult to get, but maybe would only be used one time
- Can we even prevent/cure something like this (here my knowledge is lacking most)
- Unforeseen anthropogenic (man-made) risks (1 in 30)
- At every step Toby Ord understands the ‘unknown unknowns’ and also here accounts for them
- There is 5/6 chance that we will make it through the next 100 years
- But making the 1/6 smaller (to 1/6,1) may be very valuable and we’re not doing enough at this moment
- The future potential of us humans is so vast (exploring the galaxy, making more art, discovering new things about nature, etc)
- In the last chapter Toby Ord does a great job of making you want to see the bright future that we need to protect