No One Should Be GPU Poor

For everyone to have access to AGI, everyone must also have access to the compute to use it

I’ve been thinking recently about how to more optimally share information with all of you. There’s a lot of machine learning info to cover and even more resources and updates being created each week. I want to optimize your time spent reading my articles. My primary goal: make the information I share easier for you to parse.

This week I’m splitting my two articles differently. In the first (this one) I’ve written about an important machine learning topic and included a shorter, more easily readable list of machine learning updates and resources at the end. On Friday, I’m going to send a second article out with a reflection on the industry AI trends and available jobs within the realm of AI. This will be sent only to supporters.

This removes paid preview articles, only sends one email to free subs, and makes everything (resources and important ML topics) available to supporters. Let me know what you think of the format and I’ll keep updating it to make it better. Thanks! -Logan

---

We’ve had some interesting takes this past week about the journey toward AGI and how to do it safely. We’ve seen the head of superalignment at OpenAI resign amidst safety concerns regarding OpenAI’s AI development plan. Money had been moved from OpenAI’s alignment research teams toward the product development teams. From the head of superalignment, this was seen as a shift away from OpenAI’s mission to create responsible and safe AI for everyone. The main concern raised is that we need to put more resources into understanding AI that’s more intelligent than we are.

There’s been some pushback from many individuals regarding this stance. The fundamental disagreement is that LLMs don’t have the potential (or at least anytime soon) to be more intelligent than we are. It’s an interesting discussion that I’ll break down more for supporters this Friday.

AGI and its safety implications has been a huge topic since the advent of LLMs. Whether or not LLMs can achieve AGI is a discussion for another time, but the implications behind AGI are important to consider. There are two primary concerns I’ve seen regarding accelerating toward AGI:

• It’s an existential threat to our species.

• Those who will have access to AGI will have a huge advantage over those without. 

The second concern is what I want to discuss. In this second camp, everyone is all about pushing for AGI, but a democratized AGI. Within our current society, we’ve seen the advantage of being wealthy when it comes to accruing opportunities, more wealth, and raising quality of life. Uneven access to AGI will far surpass the advantage we see with regard to wealth today. This is why a large part of the AI community is pushing for open-sourcing all AI so everyone has access to it. They’re concerned that those with the resources to develop AI will continue to gain advantage over those who don’t. I agree, but just open-sourcing AI won’t be enough. Access to AI also requires the compute to use it. Compute is the new oil or the new gold—it’s just much more valuable.

This is a huge topic in the context of companies purchasing compute to develop AI further. What I don’t see discussed enough is that  individuals also need access to compute for AI to work for them. This means access to the compute to run AI for everyone will be a necessity for AI to be equally beneficial. I make a particularly large deal about machine learning accessibility improvements and what that means for all of us because they’re necessary for AGI to be truly beneficial. If compute isn’t low enough cost for consumers to use it, AI will become concentrated only amongst the wealthy.

Luckily for consumers, I don’t think there are currently any major worries regarding having access to compute outside of having devices connected to the internet. This access will likely come through large companies whose profit models require users to use that compute for the company to make money. This can already be seen with Gemini and ChatGPT. Users sending requests are using Google’s and OpenAI’s compute. The only concern here is the possibility of rising costs to price consumers out of AI tools, but a healthy competitive business environment should keep that in check.

Consumers also win out by purchasing devices that can run AI tools locally. Hardware advancements that bring the cost per compute down are hugely beneficial to consumers. The devices consumers purchase can also give them access to the compute needed for AI workflows if it’s made cheap and performant enough. There have been a lot of questions about Apple’s most recent iPad update and why a device like that needs as much compute as they put in it. AI is the answer. This is why I make a big deal about consumer hardware advancements and what they mean for machine learning. There’s no need to be concerned about access to compute resources when complex computational tasks can be run on the device already in your pocket.

For individuals developing AI, compute isn’t quite as guaranteed. There’s an overarching opinion on social media that to be an AI developer you need a rig with the latest Nvidia graphics card, but I’m going to push against that. Yes, it’s helpful if you can afford it, but the barrier to entry of building with AI is much lower. Just like consumers gaining access to compute, the best device to use for machine learning purposes is the one you already have. If you’re working with state-of-the-art AI, you’ll need a compute platform far more capable than what any local rig can provide. Your local compute just needs to run small machine learning workloads to validate ideas before you push it to the cloud.

Luckily, there are many inexpensive and even free cloud options. Check out the list I’ve created of the platforms offering free GPUs. It’s a work in progress, so if you have anything to add, let me know. Similarly to consumers, we need an open and competitive AI marketplace to ensure the costs of compute resources provided in the cloud stay low.

That’s all for this week. I feel like I was a bit all over the place, so here’s a quick recap:

• Access to compute is necessary for AI to provide an equal advantage to everyone.

• As a consumer, this will come through the devices and services you use.

• As a developer, your local rig should have enough compute to validate ideas at a small scale but cloud providers will be necessary to utilize state-of-the-art AI.

Thanks for reading! You can support Society’s Backend for just $1/mo and you’ll get the article I send out later this week:

Here are the machine learning updates and resources you should be aware of.

Stay Informed

• 100 thing announced at Google I/O - At I/O 2024, Google announced 100 new things, showcasing innovation and updates across various domains. The announcements covered improvements in technology, software updates, and new product launches. Key highlights included advancements in AI, mobile technology, and digital services. These updates reflect Google's commitment to enhancing user experience and pushing the boundaries of technology.

• OpenAI announces GPT-4o, a new flagship model - OpenAI introduces GPT-4o, a new model for audio, vision, and text tasks. Users can now input text and images in the API and ChatGPT. Voice and video input will be added soon. The new model aims to reason in real-time across multiple formats.

• Ilya leaves OpenAI - Ilya Sutskever is leaving OpenAI after almost a decade. He believes OpenAI will continue to build safe and beneficial AGI under new leadership. The company's trajectory has been impressive. Sutskever has confidence in the future of OpenAI.

• Machine learning enables safer and lower-power MRI - Machine learning makes low-power MRI more affordable and safer while maintaining accuracy. This advancement could lead to patient-friendly ULF MRI scanners in healthcare settings worldwide. Standard MRI machines are expensive and require specialized infrastructure, limiting accessibility, especially in low-income countries. A new low-power ULF MRI scanner shows promise in producing quality imaging without the need for costly equipment or high power consumption.

• Apple shows off controlling an iPad with just your eyes - Apple is showcasing eye-controlled iPad accessibility features. The technology is inspiring and innovative. The announcement was made ahead of a major keynote event. This demonstrates Apple's commitment to accessibility.

• Build a LLM From Scratch by Sebastian Raschka is almost complete - The book "Build an LLM from Scratch" by Sebastian Raschka is almost complete. The final stages include the classification-finetuning and instruction finetuning chapters. The estimated publication date is Summer 2024. Bonus material is available on GitHub for additional content.

• OpenAI rolls out interactive tables and charts in ChatGPT - OpenAI is introducing interactive tables and charts in ChatGPT. Users can now add files from Google Drive and Microsoft OneDrive directly into ChatGPT. This feature will be available to ChatGPT Plus, Team, and Enterprise users soon. 

• Controlling a Robotics Arm with Vision Pro - The author, Kai Junge, mentions working on the Vision Pro dexterous teleop system with the compliant ADAPT Hand. They are exploring different directions for sensing, learning, and bimanual capabilities.

• OpenAI head of superalignment resigns - Jan Leike reflects on his time as head of alignment at OpenAI, highlighting achievements in AI research and expressing gratitude for his team. He acknowledges the challenges of steering and controlling smarter AI systems, expressing concern about the company's priorities. Leike believes more focus is needed on areas like security, safety, and societal impact to prepare for future AI models.

• Salesforce releases studies about AI tools at work - Salesforce released studies about AI tools at work, showing their impact on productivity and trust. C-suite executives see trust as key for AI's business value. However, some find it challenging to utilize AI effectively. Desk workers have opportunities to leverage AI for more meaningful tasks.

Learning Resources

• How to set up your own Project Astra - The author was impressed by the Astra demo at Google I/O. They decided to build their own version using Gemini 1.5 Pro Flash. It can detect gates and stream content directly from the camera. Voice is provided by Eleven Labs.

• A lecture on aligning open language model Nathan Lambert gave at Stanford - Nathan Lambert gave a lecture at Stanford on aligning open language models. The lecture covered topics like llamas, alpaca, open assistant, qlora, and evaluation. 

• How to evaluate RAG applications - The text explains how to evaluate a RAG application using a step-by-step guide. It involves loading a knowledge base and required models, creating a test set generator with customizable options, and loading data into a Pandas DataFrame for analysis.

• LoRA explained - LoRA is a technique for fine-tuning Large Language Models (LLMs) efficiently. It helps reduce memory usage during adaptation by decomposing the weight matrix into smaller matrices. By using LoRA, fewer parameters need to be fine-tuned, making the process more efficient. Akshay explains LoRA and provides a hands-on coding tutorial to demonstrate its application in fine-tuning LLMs.

• How Diffusion Models Are Improving AI - Diffusion models are a type of AI that create high-quality, realistic data by refining random noise through a two-step process. They are versatile, capable of generating images, audio, and more, with the ability to control the outcome closely.

• How good are the latests open LLMs? Is DPO better than PPO? - Sebastian Raschka's article reviews four recent open-source, transformer-based large language models (LLMs) - Mixtral, Meta AI's Llama 3, Microsoft's Phi-3, and Apple's OpenELM - highlighting their features and advancements.

• Synthetic data generation doesn’t expand a model’s knowledge - Synthetic data generation is helpful but doesn't expand a model's knowledge. It can improve training data quality for curve fitting. Sampling synthetic data from the same distribution may not solve performance plateaus in models.

• Code and findings from experimentation with LoRA/QLoRA - Cameron conducted many experiments using LoRA/QLoRA and shared key findings for better model performance. Practical tips include using a low rank for LoRA, adding LoRA adapters to linear layers, and tuning learning rates carefully. Observing model outputs on various evaluation sets is crucial for understanding performance.

• Top ML Papers of the Week - Top ML papers of the week include AlphaFold 3 for predicting molecular structures, xLSTM for scaling LSTMs, DeepSeek-V2 for efficient inference, and AlphaMath Almost Zero for enhancing mathematical reasoning. These papers showcase advancements in AI models and techniques for various applications.

• Machine Learning Crash Course By Google - Google offers a fast-paced introduction to machine learning. The course includes video lectures and hands-on exercises. Learn key concepts and practice with real-world examples. Explore additional resources on Google's platform.

If you’d like more updates and learning resources, follow me on X.