Working with machine learning today feels like inching towards the edge of a very tall cliff. Peering over the edge towards our future feels vast, and rapid change feels imminent. Not only do we not know what’s at the bottom of the cliff, but any day now, we’ll be cast into the speed and lack of control of the freefall of exponential acceleration. Anyone who is pushing the boundary of their field experiences these “ah-ha” moments about the future from time to time, and this edition is dedicated to sharing ours.

 

Last month, a massive language generating AI called GPT-3 was released to the public. We got a beta version for researchers in March and have been playing with it ever since. This model was trained on the entire internet, many times over. The learning process was executed on millions of dollars worth of hardware, allowing the program to consume lifetimes of human writing in seconds. It was likely trained in such a manner for years, undergoing countless self-correcting iterations before being released. Even with all this training, the text it generates is at times redundant and at other times, wildly off-base. It’s far from perfect and it’s pretty slow. But it's capable of flexibly picking up on patterns and writing text that’s often indistinguishable from human-generated writing, which forced us to take pause. Even with all of its flaws, it puts our current reality into the perspective of where we’ve come from and where we’re going. 

What’s most exciting for us is the realization that GPT-3 is only just the beginning of a new era of rapid innovation. For about 40 years, the state of the art in language modeling was a set of manually programmed rules. You may share our nostalgia for Zork, a crude text based adventure game that exemplifies such a system. Language generation used to consist of gigantic files containing handwritten phrases and sentences, triggered when certain words were found in the input. From 1975 to 2007, we were essentially stuck building more complex versions of Zork. AI language bots like GPT-3 represent a quantum leap in language generation technology in that they rely on no hardcoded logic and instead learn observationally and through trial-and-error, much like a human. Like biological evolution, human innovation seems to exhibit punctuated equilibria: long periods of stasis interrupted by rapid growth before returning to another period of stasis. This is what innovation looks like on an organic scale.

 

However, as we develop AI that builds the computers it runs on, innovation is no longer confined by organic scales of change. With the ubiquity of machine learning, we are permanently divorcing from this pace of growth. Machine learning is capable of finding patterns and generating content within moments that used to take so many hundreds of monkeys on typewriters. It’s plain to see we are approaching an exponential evolution: one year of progress just won’t mean the same thing in a decade.

 

GPT-3 was the first AI to prove that language models can be “few-shot learners”, which means they can pick up on patterns and apply them to the outside world with very few training examples. It’s one step closer to the distant ideal of “general artificial intelligence” as opposed to the narrow intelligence that’s currently ubiquitous in AI.

 

 

This concept of general artificial intelligence even leads many experts in the field to pontificate about the possible emergence of an “artificial superintelligence” (ASI) in our lifetimes: machines inconceivably smarter than us in many ways. 

While there is not perfect consensus on what that would look like and when (or if) it may happen, our current technology can offer glimpses, and it is especially interesting when it appears capable of passing the famous Turing Test, in which AI passes off as a human without being detected. For example, would you have guessed the paragraph in italics above was written entirely by GPT-3, when asked about artificial general intelligence? 

We always talk about the singularity as if it’s out there—some dystopia or utopia waiting for us, but not yet a reality. But let’s look around us. I spent 9 hours today staring into an artificially illuminated world that I can access through a thin film of liquid crystals. This world was created entirely by us and holds most of society’s power, communication and wealth. It’s responsible for medical innovations that save and improve countless lives a year and is also responsible for absolutely unprecedented levels of depression, isolation and anxiety. And it is just barely approaching the precipice of the power it has the potential to wield.

 

These tools we've built have become a fundamental part of what it means to be human in terms of our experience of life, and soon enough, in terms of our physical bodies. If you could swallow a pill of nanobots that programmatically excise any cancer or inject genes to keep you young, wouldn't you? Examples of this mind-bending progress and exponential change are all around us. We are witnessing advances in brain machine interfaces: technically living microscopic robots capable of reproduction. Nearly each year we see explosive growth in personal medicine, fusion energy, quantum computing, decentralized systems, AI, space exploration, and many other nascent fields.

 

Bearing witness to the cutting edge of technology is like staring up at a tidal wave that is mounting in front of you. It’s massive, powerful, inevitable and it’s coming straight at you—all the good and all the bad. But there’s a funny thing that happens staring up at this wave. You realize that it is composed of billions of droplets of water, and you are one of those droplets. Staring into the vastness ahead makes us feel at times like we have absolutely no power against whatever’s about to happen. But we are what’s happening. 

The chances of existing at a time in which these changes are occurring, having the space to think about these problems, and being in a position to affect them are infinitesimally small. We represent an extremely unique inflection point. We are privileged with the opportunity to think carefully about how we are impacting the world, to ensure that every action and word that we generate be a reflection of our values, and to be a part of the current that is not just attempting to stay ahead of the masses but is actually guiding society toward the future we want to see.

 

Every year, the last generation’s bleeding edge becomes obsolete. Technology becomes more integrated into our lives as time passes. We are actively creating a future that will either champion humanity or will squelch its light. If bots take all of our jobs, where does that leave us? Destitute, broke and purposeless, left behind by a world we created only for it to trample us? Or will we create a society in which the dirty work of maintaining the systems that sustain us are automated, liberating humans from robotic jobs and freeing us up to be human again? When's the last time you painted on a cave wall? In a phrase coined by a friend of mine, the best-case scenario is that we are heading into a "primitive future", in which bots take care of the drudgery of maintaining our society and humans are released back into the expanses of this stunning world, to think freely, live freely, to love and breath and truly experience our existence while we have it.

 

So, what do you value? What are your dreams? Stand behind them. We can help. With all of that power, what will you build?

Enjoy your humans, your homes, your families, your evening walks and your morning coffees this holiday season. Technology can wait. It’s not going anywhere.

I (Ariana) used to run a wine blog. It has a pretty specific voice. We trained a separate GPT-3 model on my wine blog and then let it start generating its own wine reviews. Not only did it very successfully mimic my voice, its reviews are also startling accurate, especially given their impressionistic nature. This also happens to be a damn good wine…

Wine: Arianna Occhipinti Frappato

Review: I’m not sure if it was the wine or me but this Frappato made me feel like a teenager again. It reminded me of my first time, when I didn’t know what to expect and everything felt new and exciting. The feeling that you can do anything because nothing is impossible. And then there was the wine itself, which tasted like a warm summer day in Sicily with all of its sunshine and promise.This Frappato from Sicily makes you want to run through fields of wildflowers until your clothes are soaked with dew and your hair is filled with pollen, before coming home at dusk for dinner on the patio as your mother cooks up fresh pasta while singing along to Italian pop music playing on the radio…and then maybe getting busy under the tablecloth after everyone goes to bed! But even though it tastes so good now, don’t wait too long because it won’t be around forever (or will it?).