The Singularity Approaches!

GPT-2 As Step Toward General Intelligence
A machine learning researcher writes me in response to yesterday’s post, saying:

I resisted the urge to answer “Yeah, well, your mom is a brute-force statistical pattern matcher which blends up the internet and gives you back a slightly unappetizing slurry of it when asked.”

But I think it would have been true.

A very careless plagiarist takes someone else’s work and copies it verbatim: “The mitochondria is the powerhouse of the cell”. A more careful plagiarist takes the work and changes a few words around: “The mitochondria is the energy dynamo of the cell”. A plagiarist who is more careful still changes the entire sentence structure: “In cells, mitochondria are the energy dynamos”. The most careful plagiarists change everything except the underlying concept, which they grasp at so deep a level that they can put it in whatever words they want – at which point it is no longer called plagiarism.

GPT-2 writes fantasy battle scenes by reading a million human-written fantasy battle scenes, distilling them down to the concept of a fantasy battle scene, and then building it back up from there. I think this is how your mom (and everyone else) does it too. GPT-2 is worse at this, because it’s not as powerful as your mom’s brain. But I don’t think it’s doing a different thing. We’re all blending experience into a slurry; the difference is how finely we blend it.

“But don’t humans also have genuinely original ideas?” Come on, read a fantasy book. It’s either a Tolkien clone, or it’s A Song Of Ice And Fire. Tolkien was a professor of Anglo-Saxon language and culture; no secret where he got his inspiration. A Song Of Ice And Fire is just War Of The Roses with dragons. Lannister and Stark are just Lancaster and York, the map of Westeros is just Britain (minus Scotland) with an upside down-Ireland stuck to the bottom of it – wake up, sheeple! Dullards blend Tolkien into a slurry and shape it into another Tolkien-clone. Tolkien-level artistic geniuses blend human experience, history, and the artistic corpus into a slurry and form it into an entirely new genre. Again, the difference is how finely you blend and what spices you add to the slurry.

“But don’t scientists have geniunely original ideas?” Scientists are just finding patterns in reality nobody has ever seen before. You say “just a pattern-matcher”, I say “fine, but you need to recognize patterns in order to copy them, so it’s necessarily a pattern-recognizer too”. And Einstein was just a very good pattern-recognizer.

“But don’t humans have some kind of deep understanding that pattern-recognition AIs don’t?”

Here’s a fun question: the human brain is undoubtedly the most powerful computer in the known universe. In order to do something as simple as scratch an itch it needs to solve exquisitely complex calculus problems that would give the average supercomputer a run for its money. So how come I have trouble multiplying two-digit numbers in my head?

The brain isn’t directly doing math, it’s creating a model that includes math and somehow doing the math in the model. This is hilariously perverse. It’s like every time you want to add 3 + 3, you have to create an entire imaginary world with its own continents and ecology, evolve sentient life, shepherd the sentient life into a civilization with its own mathematical tradition, and get one of its scholars to add 3 + 3 for you. That we do this at all is ridiculous. But I think GPT-2 can do it too.




Read on. It convinced me GPT-2 is the closest thing to AGI ever created.

 

WellSaid creates life-like synthetic voice for next generation experiences, products, and content | Woah. It's scary and incredible how realistic it is!

 

StyleGAN trained on Portrait Art

 

UMMS scientists develop technology to give night vision to mammals

 

MS Paint doodle of a dragon turned into something more substantial via style transfer

 

^
Hold that thought.

 

AI, Alphastar, beats Human Starcraft champions.

 

I'm worried that it might set an unintended precedent: just a few months ago I felt it was unreasonable to create an AI that could replace human writers. I figured out this was the wrong way to look at the problem.

More to the point: rather I want more truly human like androids like Erika. I have no problem with writing their own novels. Because she's a synthetic human.

With the EU precedent, I feel like depending on who you elect, such guidelines could actually make the problem worse.

At issue isn't with AI in itself, but the people that control it.

For those unfamiliar with Erika:



To me, because they don't yet have sentience, and make their own decisions, it almost feels like slavery.

 

There's a fine line between "Kill these humans" and "Kill all humans". Sometimes I wonder if we're in a race to see what kills us all off first - AI or some engineered genetic disease. I wonder if, in some lab in Shanghai or somesuch, there's work being done on creating a disease that will kill off everyone who's not genetically Chinese. Of course, then I wonder "How can I turn that into a novel?" I've already done the whole "AI" bit.

 

This website generates text. Try prompting OpenAI's medium sized GPT-2 model

Prompt: After the revolution, the Chocolate Party distributed free candy to everyone
Result:
After the revolution, the Chocolate Party distributed free candy to everyone in the city. People would drink their own milk, and buy their own sweets. But one day, the whole city was drunk and hungry; children in class B and C couldn't even manage to wash off the sticky substance on their hands. As the kids looked at the city of blood in the garbage, everybody said, 'We are all children. In one way or another, we have to go.' "This experience made people lose hope that revolution would be able to help everyone. For those kids who felt they didn't have much to lose, the candy gave them some hope. "A couple weeks later, the students started eating less candy," says Kowalski, who left Tsergyeva's group in December 2013 and joined the anti-government Maidan movement. "The more children tried to live in a less socialized way, the less candy they bought."

Photo

As the Maidan movement grew