Lifeboat Foundation: Safeguarding Humanity
Toggle light / dark theme

Blog

Category: mathematics – Page 2

Generalization Dynamics of LM Pre-training

An AI has a limited amount of “capacity” (brainpower). Early in training, it develops quick, shallow circuits to memorize data because that’s the easiest way to get the right answer. Later, it develops complex circuits for actual reasoning. Because space is limited, these two internal systems are constantly competing for control. Whichever type of data the AI happens to be reading in a specific moment determines which circuit wins the battle.

People typically assume that LMs stably mature from pattern-matching parrots to generalizable intelligence during pre-training. We build a toy eval suite and show this mental model is wrong: throughout pre-training, LMs frequently and suddenly hop between parrot-like and intelligence-like modes, i.e. distinct algorithms implemented by distinct circuits. We call this mode-hopping. Across our suite, LMs can suddenly latch onto memorized or in-context patterns instead of in-context learning, use System 1 instead of System 2 thinking, pick up what sounds true instead of what is true, fail at multi-hop persona QA, out-of-context reasoning, and emergent misalignment — then just as suddenly revert and generalize. Mode-hopping is not explained by standard optimization dynamics: it is locally stable and can not be fixed by checkpoint averaging. We instead think of it as a capacity allocation problem: in a capacity-bounded model, generalizable circuits must compete with the shallow ones learned early in training, and the data in each pre-training window decides which circuits win. Our suite provides a cheap set of pre-training monitors and a new lens on generalization. Building upon our insights, we demonstrate three applications: (i) select intermediate pre-training checkpoints that strongly generalize reasoning and alignment, better than the final pre-or mid-training checkpoints, (ii) select pre-training data that controls and stabilizes generalization dynamics, and (iii) test prior generalization predictors, falsifying the monolithic belief that “simpler solutions generalize better”

Building general AI without generalization is doable but meh. We want an intelligence that learns deep, transferable structure, not a parrot that matches shallow patterns. Real generalization would unblock many today’s key open problems: data-efficient (online) learning, shortcut learning, transfer capabilities from verifiable domains (math, coding) to broader non-verifiable yet economically valuable domains, and maintain a coherent character that truly aligns with human values.

The distinction between parrots and intelligence is computational. Parrots repeat in-context patterns; intelligence infers in-context functions. Parrots encode a persona as bags of disconnected facts and traits; intelligence learns a shared persona representation that connects all. Parrots memorize reasoning steps; intelligence forms general reasoning circuits for entity tracking, backtracking, or even for highly abstract concepts like truth.

Liad Mudrik: Using Prediction Maps to Guide Theory Testing and the Search for the NCC

This talk is part of the “New Ideas in NCC Research” workshop of the Bamberg Mathematical Consciousness Science Initiative (BAMΞ). For more talks and details, see https://www.uni-bamberg.de/en/bamxi/r…

Abstract: In recent years, the search for the neural correlates of consciousness (NCC) has been complemented, and influenced, by the ongoing efforts to test neuroscientific theories of consciousness. A key insight from these efforts, though, is that many theories remain underdeveloped and not fully specified, making it harder to establish stringent tests for their predictions. In this talk, I will present a novel methodological approach that represents scientific theories as networks of beliefs structured in a core-periphery manner. These Prediction Maps visualize theoretical claims and empirical predictions, and illustrate their inferential relations. This framework further facilitates systematic theory testing by allowing researchers to evaluate the evidential weight of different components of a theory, and to identify which experimental results would constitute the most informative tests. To do so, we apply graph-theoretic and network analysis metrics, quantifying the centrality of specific predictions. I argue that this approach can advance efforts to arbitrate between theories of consciousness and to identify their most promising candidate mechanisms as NCCs.

String theory is uniquely derived from basic assumptions about the universe, physicists show

If you could take an apple and break it into smaller and smaller parts, you would find molecules, then atoms, followed by subatomic particles like protons and the quarks and gluons that make them up. You might think you hit the bottom, but, according to string theorists, if you keep going to even smaller scales—about a billion billion times smaller than a proton—you will find more: tiny vibrating strings.

Developed in the 1960s, string theory proposes that everything in the universe is made from invisible strings. The theory arose as a possible solution to the problem of “quantum gravity,” the quest to align quantum mechanics, which describes our world at the smallest scales, with the general theory of relativity, which explains how our universe works on the largest scales (and includes gravity). Researchers have tried to reconcile the two theories—asking, for example, how gravity behaves in the quantum realm—but their equations go berserk, or in mathematical terms, go to infinity.

String theory is a mathematical solution that tames the unruly infinities. It purports that all particles, including the graviton—the hypothetical particle believed to convey the force of gravity—are generated by very small vibrating strings. The math behind string theory requires the strings to vibrate in at least 10 dimensions, rather than the four we live in (three for space and one for time), which is one of the reasons some scientists are not convinced that string theory is correct. But perhaps the biggest challenge for the theory is the ultrahigh energies required for testing it: Such an experiment would require a particle collider the size of a galaxy.

String Theory Emerges from “Almost Nothing”

Developed in the 1960s, string theory proposes that everything in the universe is made from invisible strings. The theory arose as a possible solution to the problem of “quantum gravity,” the quest to align quantum mechanics, which describes our world at the smallest scales, with the general theory of relativity, which explains how our universe works on the largest scales (and includes gravity). Researchers have tried to reconcile the two theories—asking, for example, how gravity behaves in the quantum realm—but their equations go berserk, or in mathematical terms, go to infinity.

String theory is a mathematical solution that tames the unruly infinities. It purports that all particles, including the graviton—the hypothetical particle believed to convey the force of gravity—are generated by very small vibrating strings. The math behind string theory requires the strings to vibrate in at least 10 dimensions, rather than the four we live in (three for space and one for time), which is one of the reasons some scientists are not convinced that string theory is correct. But perhaps the biggest challenge for the theory is the ultrahigh energies required for testing it: Such an experiment would require a particle collider the size of a galaxy.

What is a physicist to do? One way they can probe the theory is to turn to a “bootstrap” approach, in which researchers start with certain assumptions they believe to be true about the universe, and then see what laws emerge out of those assumptions. In a new paper titled “Strings from Almost Nothing,” accepted for publication in Physical Review Letters, Caltech researchers, and their colleagues at New York University and Institut de Fisica d’Altes Energies in Barcelona, have done just that. From a couple of basic assumptions about how particles should scatter off one another at very high energies, they derived the elements of string theory.

Amino Acid Patterns Help Scientists Distinguish Alien Life

Dr. Fabian Klenner: “We’re showing that life does not only produce molecules. Life also produces an organizational principle that we can see by applying statistics.” [ https://www.labroots.com/trending/space/30534/amino-acid-pat…ien-life-2](https://www.labroots.com/trending/space/30534/amino-acid-pat…ien-life-2)

What methods can scientists use to correctly identify biosignatures, aka signs of life beyond Earth? This is what a recent study Nature Astronomy hopes to address as a team of researchers from the University of California, Riverside (UC Riverside) and Israel investigated a new pattern-based method for identifying biosignatures. This study has the potential to help scientists develop new methods for finding life beyond Earth, which could narrow the scope for both how and where to find life.

For the study, the researchers used mathematics to suggest that instead of looking for specific molecules when searching for biosignatures scientists should instead look for organizational patterns. The primary motivation behind the study was to challenge longstanding methods regarding how to search for biosignatures, which have traditionally been focused on finding individual and specific molecules. In the end, the researchers found that the amino acids in biological (biotic) samples exhibited a much larger range of diversity compared to non-biological (abiotic) samples. These could shape a new generation of astrobiology, which is the study of searching for life beyond Earth.

“We’re showing that life does not only produce molecules,” said Dr. Fabian Klenner, who is an assistant professor of planetary sciences at UC Riverside and a co-author on the study. “Life also produces an organizational principle that we can see by applying statistics.”

This Physicist (Unexpectedly) Derived Gravity from Information

Sponsors: Accelerate your efficiency. Sign up for your one-dollar-per-month trial today at http://shopify.com/theories Sign up for Claude today at http://claude.ai/theoriesofeverything and checkout Claude Pro — which includes access to all of the features mentioned in today’s episode. I personally subscribe to The Economist. FLASH SALE to May 18: 50% off annual (double the usual!) No other podcast has this! https://economist.com/TOE

What if gravity is just entropy in disguise? Professor Erik Verlinde joins me to argue that gravity isn’t a fundamental force—it’s thermodynamic, emerging from quantum information the way gas pressure emerges from molecules bouncing around. We explore why spacetime may be stitched together by entanglement, and how dark energy and dark matter both pop out automatically without extra particles or parameters. Verlinde explains why the cosmological constant problem is a red herring, and why there may be no final theory of physics. When asked where the universe comes from, his answer is one word: chaos.

SUPPORT: Support me on Substack: https://curtjaimungal.substack.com/su… me on Crypto: https://commerce.coinbase.com/checkou… Support me on PayPal: https://www.paypal.com/donate?hosted_… JOIN MY SUBSTACK (Personal Writings): https://curtjaimungal.substack.com LISTEN ON SPOTIFY: https://open.spotify.com/show/4gL14b9… TIMESTAMPS:

  • 00:00:00 — Thermodynamic Gravity and Information
  • 00:06:35 — Beyond Effective Field Theory
  • 00:13:08 — Turtles All The Way Down
  • 00:25:41 — Entropy as a Force
  • 00:36:31 — Entanglement and Spatial Connectivity
  • 00:47:31 — Deriving Inertia and F=ma
  • 00:56:41 — De Sitter Space Challenges
  • 01:02:01 — Dark Matter and Milgram
  • 01:11:51 — The Emergence of Time
  • 01:21:01 — Statistical Gravity Fluctuations
  • 01:27:01 — Quantum Computational Complexity
  • 01:36:01 — Physics Intuition and Mentorship
  • 01:47:31 — Beauty, Garbage, and Chaos

LINKS MENTIONED: Papers, books, websites:

Videos:

  • • A 2 Hour Deep Dive into Entropy
  • • The Mathematics of String Theory [Graduate…
  • • The Debate That Divides Physics: Is the Un…
  • • The Physicist Who Found Quantum Theory’s U…
  • • Retrocausality & The Transactional Interpr…
  • • The Physicist Who Proved Entropy = Gravity
  • • The Physicist Who Says Time Doesn’t Exist
  • • The Most Astonishing Theory of Black Holes
  • • The (Simple) Theory That Explains Everythi…
  • • The Crisis in String Theory is Worse Than…
  • • Dark Dimensions: NEW THEORY Unifying Dark…
  • • MIT Scientist’s Discovery: “Black Holes Mi…
  • • The Woman Who Broke Gravity | Claudia de Rham
  • • Solving the Problem of Consciousness | Ste…
  • • Frederic Schuller: The Physicist Who Deriv…
  • • The Loop Quantum Gravity Debacle: Carlo Ro…
  • • An (Elementary) Introduction to Quantum Co…
  • • Can Physics Explain Its Own Laws?
  • • The Nobel Laureate Who (Also) Says Quantum…
  • • This Cosmologist Discovered Something Stra…
  • • Michael Levin: Consciousness, Biology, Uni…

SOCIALS:

  • Twitter: / toewithcurt
  • Discord Invite: / discord

Guests do not pay to appear. Theories of Everything receives revenue solely from viewer donations, platform ads, and clearly labelled sponsors; no guest or associated entity has ever given compensation, directly or through intermediaries. #science.

JOIN MY SUBSTACK (Personal Writings): https://curtjaimungal.substack.com.

LISTEN ON SPOTIFY: https://open.spotify.com/show/4gL14b9…

TIMESTAMPS: 00:00:00 — Thermodynamic Gravity and Information 00:06:35 — Beyond Effective Field Theory 00:13:08 — Turtles All The Way Down 00:25:41 — Entropy as a Force 00:36:31 — Entanglement and Spatial Connectivity 00:47:31 — Deriving Inertia and F=ma 00:56:41 — De Sitter Space Challenges 01:02:01 — Dark Matter and Milgram 01:11:51 — The Emergence of Time 01:21:01 — Statistical Gravity Fluctuations 01:27:01 — Quantum Computational Complexity 01:36:01 — Physics Intuition and Mentorship 01:47:31 — Beauty, Garbage, and Chaos.

Continue reading “This Physicist (Unexpectedly) Derived Gravity from Information” | >

AI is the Great Filter

Artificial intelligence is now finding planets human astronomers missed and scanning for alien signals 600 times faster than ever before.
Yet the more powerful our search tools become, the louder the silence from the cosmos grows.

This video explores why the same technology helping us look for extraterrestrial life may also explain why we cannot find any.

We examine the Great Filter hypothesis, the mathematics of self-replicating probes, and the growing consensus that any aliens out there would be machines, not biological beings.

From Matrioshka brains to the aestivation hypothesis to the Dark Forest, the universe may be hiding minds we cannot recognise, or warning us about a test every civilisation faces.

Chapters.

00:00 — Intro.

Continue reading “AI is the Great Filter” | >

How Unknowable Math Can Help Hide Secrets

Perhaps the most famous example comes from a theorem by the logician Kurt Gödel’s celebrated result — one of two “incompleteness theorems” he published in 1931 — established that for any reasonable set of basic mathematical assumptions, called axioms, it’s impossible to prove that the axioms won’t eventually lead to contradictions. Though mathematicians continued their research much as they had before, they would never again be certain that their rules were self-consistent.

More than 50 years after Gödel’s theorem, cryptographers devised a radical new proof method in which unknowability played a very different role. Proofs based on this technique, called zero-knowledge proofs, can convince even the most skeptical audience that a statement is true without revealing why it’s true.

These two flavors of unknowability, which originated decades apart and in different fields, were long considered completely unrelated. Now the computer scientist Rahul Ilango (opens a new tab) has established a striking connection (opens a new tab) between them. While still a graduate student, he devised a new type of zero-knowledge proof in which secrecy stems from the fundamental limits of math. Ilango’s approach gets around limitations of zero-knowledge proofs that researchers have long thought insurmountable, pushing the boundaries of what such a proof can be. The work has also spurred researchers to explore other intriguing links between mathematical logic and cryptography.

/* */