Andrej Karpathy recently gave a talk on artificial intelligence and machine learning at the University of California, Berkeley AI hackathon, Andrej Karpathy believes that the current large language model (LLM) plays a central role in computing, changing the nature of computing, and also mentioned OpenAI's GPT-4.0 demonstration and the revelation of the movie "She" to modern AI, and predicted that AI models will solve a large number of problems in reality, This includes automating digital and physical infrastructure
Here is a passage from the speech:
Andar Karpathy,
Hello everyone, thank you very much for having me, I feel very honored to be here. I love hackathon so much because it's full of energy and creativity, where young people are doing cool things together, learning and creating together. I've been to a lot of hackathons, so I'm very happy to be here today.
One thing that surprised me was that the scale of this event was larger than I expected. When they invited me, I didn't expect it to be so big. It's really big here, and the scale of the hackathon is really big. I want to start by, in case you're curious, this isn't normal for AI. I've been doing AI research for about 15 years, so I can say with confidence that it's evolved a lot. For me, AI is a meeting of hundreds of scholars to discuss some mathematically esoteric details, and that's what I'm used to.
When I got into AI 15 years ago, when I trained a neural network, you would train a Restricted Boltzmann Machine with small numbers on the MNIST dataset, train the network with contrastive divergence, and then double-check that on the first layer to make sure the network was trained correctly. I know it sounds a little complicated because it was a long time ago, but it felt different back then and wasn't as crazy as it is now. I feel like things are getting a little out of hand, but it's really wonderful to see that energy. Today, 15 years later, things are completely different, and the AI field is now more like this. That's why this event is so grand.
Nvidia, as the company that makes GPUs, for all the heavy computing of neural networks, is now the most valuable company in the United States and has taken over. That's the era we live in today, and it explains why there are so many hackathons like this. I think it's remarkable, but it's really unprecedented. It's a very unique moment, and it's not common for many people to be just getting into AI. Right now this field is very interesting, hyper-human, and a lot is happening.
I think the root cause is that the nature of computing is changing. We are entering a new computing paradigm, which is very rare. I almost felt like I was back in the 1980s when computers were no longer processing instructions and bytes, but large language models (LLMs) processing tokens. Tokens are small string fragments. We have a context window for the Token, not bytes in RAM, and have equivalents for disks and the like. It's a bit like a computer, but now large language models are at the core, which is why I call it the large language model operating system LLM OS.
I've talked about this at length on X, and I see it as a new computer that we're learning to program, understanding what it's good at, what it's not good at, how it fits into the product, how to get the most out of it. I find it very exciting.
Probably many people have seen OpenAI's recently released GPT-4.0 demo, about two or three weeks ago. You guys start to feel that this is something that you can talk to, that it responds in your natural language, like an audio interface. It can see, hear, and draw, and it can do many things. A lot of people have seen the movie Her, and if not, I highly recommend it because it is very enlightening for us today. The main character in the movie talks to the AI, which is called the operating system (OS), which is very prescient. It's a beautiful movie, and I encourage everyone to watch it.
The focus of the film is on emotional intelligence, but these models will solve a lot of problems in the digital realm in the real world. It's not just a single digital entity, a bit like a human, that you can talk to. It's not just a single digital entity, there may be many of them. We can give them tasks, they can communicate with each other, they can collaborate, they have virtual Slack threads, they can get a lot of work done in the digital space. They automate a lot of digital infrastructure, not just digital infrastructure, but maybe physical infrastructure as well. This is an early stage and can be a little behind a lot of digital innovation, because it's much easier to deal with bits than with atoms.
I would like to recommend another movie, which is one of my favorites, called I, Robot, starring Will Smith in 2004. It's a great movie that explores the scenario of a future where human robots accomplish a large number of tasks in society. Spoiler alert, the humans in the movie don't go too well, and the robots kind of take over, but I found it funny.