CADS - Composable and distributed systems group

Mon, 9 June 2025

Sharing of our experimental call summaries: AI-generated digests that have demonstrated efficacy for weekly Yak Collective live study groups. Represents a step forward in our exploration of human-machine collaborative cognition and Yak Oracle and Yak Memory systems for enhancing collective intelligence capabilities.

---

Paper Discussion Overview

• Paper: “Harnessing the Universal Geometry of Embeddings” by Rishi Jha, Collin Zhang, Vitaly Shmatikov, John X. Morris, Department of Computer Science, Cornell University

• Links:

  • Main paper: https://arxiv.org/abs/2505.12540

  • PDF version: https://arxiv.org/pdf/2505.12540

  • HTML version: https://arxiv.org/html/2505.12540v2

• Core thesis: Language itself contains most of what we consider intelligence, not just in processing features

• Key concept: Platonic Representation Hypothesis suggests intelligence and knowledge are embedded within language structure

Key Technical Insights

• Models demonstrate universal geometry across different architectures and modalities

• Practical implications for vector databases and AI security

• Size comparisons: LLAMA 70B model is ~40GB in FP16 weights

• Compression vs. Encryption distinction:

  • Not pure compression but strategic storage

  • Mix of precise remembering, abstraction, and pattern recognition

  • Training data ratio: ~42 exabytes reduced to ~4 petabytes

Philosophical Implications

• Language as intelligence rather than just a carrier of intelligence

• Comparison to Hebbian learning: correlation through coincidence

  • "Neurons that fire together, wire together."

• Translation vs. Brain Scanning analogy:

  • Traditional translation allows choice in information sharing

  • This theory suggests complete knowledge transfer possible

• Historical perspective: Language structure evolution may have driven Enlightenment thinking

Discussion Points

• Multilingual implications for intelligence and complexity

• Limitations in three scenarios:

  • Early childhood development

  • Less expressive languages

  • Advanced scientific concepts

• Somatic/preverbal knowledge considerations

• Antimemetics discussion: brain’s natural filtering of information

• Photoshop analogy: mathematical transformations underlying complex operations

• Model interrogation analogy: similar to questioning a person rather than data recovery

---

Yak Collective Discord: https://discord.com/channels/692111190851059762/698566364595486720/1381493186521727106

---