Co-DMing with an AI

[NichG]

So, neural text generation now has multi-paragraph coherency and can reuse and refer to proper nouns and other specific things which have occurred earlier in the text. It's not quite at the level that it'd make a good DM on its own, but as a source for prompts, setting details, even new mechanics I think there's a lot of potential.

Here's an example of some prompts generated using 60k words of campaign journals from one campaign, written in an over-the-top style (PCs are always, in the campaign journals, referred to as 'the gods'):

The gods showed him their powers, and with his knowledge of the library he knew the powers of the Library of the Unseen Force. The Library was a place where the sacred writings of the gods lay buried in the mortal realm's knowledge, protected by mortal guardians. Much as the Wielder of the Deadly Metalstorm had guarded the secrets of the Dark Sign, the Master of the Unseen Force had guarded the unleased powers of the God-Cleaving Blade, and so had he protected the Blade's descent.

The gods found themselves in a chamber of fire and flowing water. Walking calmly about, they perceived by now that this creature was a manufactured being, a puppet of the Author. And so they tried to cut it free, as the Author had decreed, yet the gods were exceeding themselves in their hubris. And so a time passed, and the Author grew impatient. And yet the thing remained, and the Author grew impatient. And yet the thing grew in strength, and in moments the Author grew weak.

The Starfish' Domain was a prison of sorts for the insane, held until they left this world. The gods found themselves in a vast submerged pit, some sort of prison. The Lord of All Nature led the gods to a large, circular structure, where he had placed a large bed of earth. Upon the circular bed, a metal wheel spun endlessly, increasing in diameter until it eventually became too heavy for the gods to carry around. The Lord of All Nature restrained the mad scientist Rotimi from destroying the world, but the mad man fought everly and struck out at the Lord of All Nature. The gods then realized that the anger of the Lord Rotimi was directed at him not for the destruction of the world, but for the amusement of the beings of this realm. And so they agreed to sit and discuss the threats of this life form, to depart that way and yet remain as a precaution.

Incidentally, there's no character named' Rotimi' anywhere in the campaign journals - this is something it cribbed from random stuff on the internet somewhere. Same with 'the Starfish' Domain', though 'the God-Cleaving Blade' was a thing. The model does explain elsewhere: "The Starfish, a sphere of destruction larger than a thumbnail
that would eat the light from the Sun in a thousand years."

So anyhow, any interest in collecting a big dataset of campaign journals and making a prompt generator? This thing (https://github.com/minimaxir/gpt-2-simple) isn't too hard to use, though it does require a Tensorflow install and some level of comfort with Python/Jupyter notebooks. For generation, I'm not even sure you'd need much in the way of a good GPU or anything like that - I think it might take a second or so per prompt if you ran it on CPU in the worst case.

[Bohandas]

Now that it cool

[ti'esar]

The gods found themselves in a chamber of fire and flowing water. Walking calmly about, they perceived by now that this creature was a manufactured being, a puppet of the Author. And so they tried to cut it free, as the Author had decreed, yet the gods were exceeding themselves in their hubris. And so a time passed, and the Author grew impatient. And yet the thing remained, and the Author grew impatient. And yet the thing grew in strength, and in moments the Author grew weak.

I honestly really like how this one sounds.

[Strigon]

I honestly really like how this one sounds.

Agreed. I might have to start working on some of these ideas for my next campaign...

As an aside, though, what I'd be really interested in as a DM is an AI that can run some of the tedium for me. Automating groups of weaker monsters, for example, would be fantastic.

[Bohandas]

Agreed. I might have to start working on some of these ideas for my next campaign...

As an aside, though, what I'd be really interested in as a DM is an AI that can run some of the tedium for me. Automating groups of weaker monsters, for example, would be fantastic.

Depending on the monsters and if you're playing mostly core rules, you could download the Temple+ Mod for the Temple of Elemental Evil computer game and set up the encounter using the mod's DM tool

This thing (https://github.com/minimaxir/gpt-2-simple) isn't too hard to use, though it does require a Tensorflow install and some level of comfort with Python/Jupyter notebooks. For generation, I'm not even sure you'd need much in the way of a good GPU or anything like that - I think it might take a second or so per prompt if you ran it on CPU in the worst case.

How would one go about actually installing this? I don't see any executable files or installer files there.

[NichG]

Depending on the monsters and if you're playing mostly core rules, you could download the Temple+ Mod for the Temple of Elemental Evil computer game and set up the encounter using the mod's DM tool

How would one go about actually installing this? I don't see any executable files or installer files there.

You need to have a Python installation. I think you probably need to have Tensorflow installed as well. Then, you run setup.py which registers this as a library module that you can just call inside of a script (or, more conveniently, a Jupyter Notebook, which is a sort of browser-based IDE that lets you modify and run Python code interactively, and displays the outputs inline).

This is the code I used to call it once installed:

import gpt_2_simple as gpt2

TEXTFILE = "" # Put your campaign notes/etc filename here
model_name = "345M"
gpt2.download_gpt2(model_name=model_name)

sess = gpt2.start_tf_sess()
gpt2.finetune(sess, TEXTFILE, model_name=model_name, steps=1000) # steps is max number of training steps

gpt2.generate(sess)

Note that this includes about half an hour to an hour of training to fine-tune the network to your data (on a decent GPU). There's also a model_name="117M" model which is a bit lighter. Once the model is trained, though, you can use it to generate text quite quickly.

[Bohandas]

You need to have a Python installation. I think you probably need to have Tensorflow installed as well. Then, you run setup.py which registers this as a library module that you can just call inside of a script (or, more conveniently, a Jupyter Notebook, which is a sort of browser-based IDE that lets you modify and run Python code interactively, and displays the outputs inline).

This is the code I used to call it once installed:

import gpt_2_simple as gpt2

TEXTFILE = "" # Put your campaign notes/etc filename here
model_name = "345M"
gpt2.download_gpt2(model_name=model_name)

sess = gpt2.start_tf_sess()
gpt2.finetune(sess, TEXTFILE, model_name=model_name, steps=1000) # steps is max number of training steps

gpt2.generate(sess)

Note that this includes about half an hour to an hour of training to fine-tune the network to your data (on a decent GPU). There's also a model_name="117M" model which is a bit lighter. Once the model is trained, though, you can use it to generate text quite quickly.

OK. Other than Python (which I am merely aware of) and GPUs, I have no idea what any of that is. Can you dumb it down a bit?

[NichG]

OK. Other than Python (which I am merely aware of) and GPUs, I have no idea what any of that is. Can you dumb it down a bit?

Jupyter is a server that creates a web interface for Python code, allowing you to do things like embed plots or outputs in the middle of code blocks and run different bits of code interactively. If you're on Windows, it comes with the basic 'Anaconda' Python distribution. Otherwise you can install it using pip or a package manager. There's also Google Colaboratory, which basically looks like a Jupyter notebook (and you could probably do these GPT-2 experiments on the free GPU they provide for 12 hours at a time).

Tensorflow is a neural network/tensor math/automatic differentiation library. It contains all the basic primitives to compose and train the neural network. Again, if you go the Colaboratory route, it'll be pre-installed and you don't have to worry about it too much. You would have to figure out how to download the code repository inside the Colaboratory virtual machine and install it there, but I think that's possible (it might take some figuring out though).

The actual code is just the syntax of the gpt2 distribution, so it isn't really too important how it works - its just boilerplate to download a model, train it, etc. The important thing is the text file you provide it.

Edit: This is a Colaboratory notebook for GPT-2 which you might find easier to work with to see what generating samples is like (but it doesn't have training). On the other hand, this one seems to be designed for training a custom variant. Click on the ipynb file, and there should be an 'open in Colab' button you can use.

[Frogsama]

This reminds me of an AI back when i was in university. It could write a paper with charts and everything. It sounded nice but was basically nonsense.

This seems way more useful especially if you could insert some parameters of your world

[NichG]

This reminds me of an AI back when i was in university. It could write a paper with charts and everything. It sounded nice but was basically nonsense.

This seems way more useful especially if you could insert some parameters of your world

You can use prompts (give it a few paragraphs of context), but directly injecting prior information is still an open research question (some stuff from Facebook from a few years ago, but I don't know the current status). If we could do that, it feels like we're pretty close to the level where we could use this for NPC conversations in computer games and have it be sufficient quality to be at least climbing the far side of the uncanny valley.

[jayem]

You can use prompts (give it a few paragraphs of context), but directly injecting prior information is still an open research question (some stuff from Facebook from a few years ago, but I don't know the current status). If we could do that, it feels like we're pretty close to the level where we could use this for NPC conversations in computer games and have it be sufficient quality to be at least climbing the far side of the uncanny valley.

Presumably as a half way house you could submit say Dumas to it having replaced "France" with "[My Country]", "England" with "[Enemy Country]".
Then after you let it go through most of the neural net, and then substitute back in from your games table.
Then if necessarily have something afterwards to do a bit more playing with it.

You'd have some oddities. "I'm named after the Beatles Song, Peter in the sky with Diamonds"

Alternatively you could have a separate program write the prompt texts from a known table (which is basically doing both the substitutions above before the neural net) and well designed templates (trial and error, but you only have to get it right once).

[NichG]

Presumably as a half way house you could submit say Dumas to it having replaced "France" with "[My Country]", "England" with "[Enemy Country]".
Then after you let it go through most of the neural net, and then substitute back in from your games table.
Then if necessarily have something afterwards to do a bit more playing with it.

You'd have some oddities. "I'm named after the Beatles Song, Peter in the sky with Diamonds"

Alternatively you could have a separate program write the prompt texts from a known table (which is basically doing both the substitutions above before the neural net) and well designed templates (trial and error, but you only have to get it right once).

I'm thinking more like, I want this NPC to be trying to convince the player to buy things, while that NPC is worried about events going on around their homestead, and that one is administering an empire and is suspicious of there being a foreign spy in their court and wants to find out who it is.

Presumably you could fine-tune a separate model for each NPC, but then you're looking at something like a 1gb per NPC download, much of which is redundant...

[Bucky]

the gods were exceeding themselves in their hubris

This phrase shouldn't ever exist. The generator probably doesn't understand what "hubris" means.

Or the point could be that the real hubris is calling themselves "the gods". But I'm not giving the generator that much credit.

[Knaight]

This phrase shouldn't ever exist. The generator probably doesn't understand what "hubris" means.

There's a bit of a difference between the definition of hubris and the etymology of hubris - and while the latter pretty cleanly curtails that phrase the former absolutely doesn't.

[Grinner]

Crazy thought, but what about using fairy tales? Not sure if there are particular formatting requirements, but there's plenty of plaintext files containing fairy tales floating about.

[Bohandas]

This phrase shouldn't ever exist. The generator probably doesn't understand what "hubris" means.

I think it understands it just fine. It merely doesn't understand literary convention that gods are typically given a pass on the matter despite frequently being the worst offenders.