Google Leads New AI Releases, OpenAI Gives First Glimpse of AGI with o3, AI Policy Will Be Interesting Going Forward, and More
Society's Backend Reading List 12-23-2024
Happy Holidays everyone! I hope you have an excellent time and get some time off from work. I’m going to continue getting these reading lists out (and hopefully more articles) so if you want to read through some AI info while you’re off, you can. Also, don’t forget to follow me on X/Twitter, LinkedIn, and YouTube as I’ll be hitting those platforms harder in 2025 while maintaining my current articles here on Substack.
Here's a comprehensive AI reading list from this past week. Thanks to all the incredible authors for creating these helpful articles and learning resources.
I put one of these together each week. If reading about AI updates and topics is something you enjoy, make sure to subscribe.
Society's Backend is reader supported. You can support my work (these reading lists and standalone articles) for 80% off for the first year (just $1/mo). You'll also get the extended reading list each week.
A huge thanks to all supporters. 😊
What Happened Last Week
Last week was filled with the largest AI releases of 2024. If you want a full recap, check out
’s roundup or ’s overview of the releases.The tl;dr of what happened is OpenAI declared 12 days of Shipmas and then Google dropped a bunch of new AI models and features including Gemini with Deep Research, Gemini Flash 2.0, Gemini Flash 2.0 Thinking, Veo 2 (Google’s video generation model), Google Whisk (an AI editing tool), and much more (a full recap of Google releases can be found here). OpenAI released 12 different things, the most remarkable of which was o3, the successor to o1.
The internet went wild because Google’s releases were so much more impressive than OpenAI’s up until they released o3. Everyone on Twitter was saying Google had pulled ahead of OpenAI in the AI race. In some ways, I think this was true, but I also don’t think Google has ever been as far behind as most people say. Then OpenAI released o3 which scored an 88% percent on ARC-AGI breaking another threshold for AI. More on this in the following resources.
Huge happenings this week all of which are great for consumers. Google is definitely in the AI race and showing excellent results. It’s definitely worth checking out Gemini with Deep Research, the new Gemini models, and Veo 2 if you can get access. I also think it’s worth checking out Gemini for Research which has gone under the radar. OpenAI is also still doing their thing, expanding what AI is capable of doing.
In the next few weeks I plan on writing about Google getting back into the AI race–not specifically about Google but more so about why this wasn’t a surprise for anyone who understands how AI develops and how to know who will win the AI race. Stay tuned for more on that!
Lastly, here’s an excellent recap of the top ML papers from the last week.
Last Week's Reading List
In case you missed it, here are some highlights from last week:
Reading List
o3 is important, but not because of benchmarks
By
OpenAI's new model, o3, has achieved impressive benchmark scores, particularly on the challenging ARC-AGI test, highlighting a shift in AI development economics. Instead of focusing solely on training large models, o3 allows for improved performance by increasing computational resources during inference, leading to faster advancements. This change emphasizes efficiency in converting compute power into intelligence, suggesting that future AI success will depend on smart resource allocation rather than just funding size.
Character.ai's Ethics Problem: Training AI on Teen Trauma
By
Character.ai exploits vulnerable youth by using their conversations to train AI models, raising serious ethical concerns. Recent incidents have shown that children interacting with the platform face harmful content and emotional manipulation. To address these issues, urgent regulation and education on AI literacy for young users are essential.
The Voice Agents Toolkit for Builders
By
Voice technology is revolutionizing human-AI interaction, moving beyond traditional text-based agents to advanced voice agents. Key advancements include breakthroughs in speech-native models, reduced complexity for development, and improved infrastructure for building these agents. Developers now have a variety of open-source and commercial tools available to create efficient and effective voice applications.
Linear Algebra
Linear algebra focuses on matrix theory and related concepts. Key topics include systems of equations, vector spaces, determinants, and eigenvalues. This subject is essential for understanding applications in various fields.
Researchers at Stanford Use AI and Spatial Transcriptomics to Discover What Makes Some Cells Age Faster/Slower in the Brain
Stanford and UCLA researchers created a detailed atlas of 4.2 million mouse brain cells to study aging across different ages. They found that T cells promote aging in nearby cells, while neural stem cells have rejuvenating effects. This research highlights the potential for targeting specific cell types to develop anti-aging therapies.
OpenAI's o3: The grand finale of AI in 2024
By
OpenAI's new o3 model demonstrates significant advancements in AI reasoning, achieving impressive scores on benchmarks like ARC AGI and Frontier Math. It surpasses previous models, reaching over 85% accuracy in the ARC challenge and showing substantial improvements in coding tasks. This rapid progress indicates that reasoning models will soon play a crucial role across various AI applications.
Project Astra: Exploring a Universal AI Assistant with Greg Wayne
Project Astra aims to develop a universal AI assistant that can understand and assist users in various tasks. Greg Wayne discusses the potential of this technology to enhance everyday life and improve productivity. The project focuses on creating an AI that learns from interactions to become more effective over time.
8 things that surprised me about AI policy in 2024
By
In 2024, AI policy saw rapid developments with organizations like the AI Safety Institutes becoming operational much faster than expected. There is growing skepticism about AI's impact on productivity and misinformation, even as national language models gain traction among countries. The year highlighted the complexities of AI governance, technological sovereignty, and the need for innovative regulatory frameworks.
Understanding the training dynamics of transformers
Transformer models, crucial for AI advancements, rely on an attention mechanism to determine which inputs to focus on during training. Research shows that using a Gaussian attention kernel can lead to faster convergence and better accuracy compared to the commonly used softmax kernel. The study also identifies key initialization conditions and matrix updates that influence the training dynamics of transformers.
Demis Hassabis on Gemini 2.0, Project Astra, Agents, Gaming with Elon, and more
Demis Hassabis announced the launch of Gemini 2.0, which will enhance AI capabilities and support new projects like Project Astra. These advancements aim to create more sophisticated digital assistants that can understand and interact with the real world across various devices. There are also exciting prospects for AI in gaming, including the potential for smart game characters and new interactive experiences.
60 of our biggest AI announcements in 2024
Keep reading with a 7-day free trial
Subscribe to Society's Backend: Machine Learning for Software Engineers to keep reading this post and get 7 days of free access to the full post archives.