Ultima Ratio Regum 0.11 Update #29: Riddle Generation 3: Poems

Hello everyone! Welcome back. This week we’re going to be delving into the generation of one of the clue types, which is going to be the main clue type for the 0.11 release. In the screenshots I’ve shown off previously I just used some relatively simple placeholder generators in order to give a sense of what these clues should look like when generated, but it’s not time to build the complete generators, and ensure that they can successfully produce a massive number of different permutations. What’s becoming clear as I work on this, interestingly enough, is that the more abstract clue types are actually going to be simpler to generate clues for, because the more abstract the clue type, the fewer formal rules it has to follow. And, as part of this, I’ve firstly come to “rank” clue types according to how abstract or how specific they are, and I’ve decided that this is going to influence when in the game the player encounters them. For 0.11 we’re focusing right now on just one clue type (more on this shortly) but in the longer term, I would anticipate the early game and easier clue threads emphasising certain kinds of clues, and then later game and harder clue threads emphasising other kinds of clues. So, firstly, let’s recap briefly the types of clues I’ve so far created the basic outlines for – sentences (a long sentence or two containing information), letters (a series of letters or numbers to be deciphered), “pattern” (a bunch of small shapes that describe a spatial relationship), “mural” (a number of symbols depicting / explaining something), world map and local map (pretty self-explanatory), “formula” (a number of things which combine into another thing), “composition” (a more literal picture of something), “shapes” (a number of connected shapes in which meaning could be hidden), “eldritch” (lots of weird connected symbols), “grid” (a set of boxes containing symbols and colours) and “focused information” (one symbol with a lot of data around it). Now, however, I think we can reasonably rank these into several groups, based on when I want to give them to the player:

So in the early game, or for easier quests, we have clues whose nature is more immediately apparent to the player. It’s pretty clear what a world map or a local map is showing, even if it might take the player a little bit to brain power to actually hunt down the location in question. The “sentence” type, again, is very clear, even if it needs deciphering – it’s describing a place, or an event, or an action, or something like that, and you need to follow the information it’s presenting to get to what it wants. The two images – the more abstract image (the middle one) showing in this case a number of features that might only be jointly found in a single place, and the more specific image (the right-most one) showing an exact location whose place needs to be figured out by the information the picture shows, are again both pretty clear in nature, I think, and comparatively simple to understand. The new clue type also fits in this category, being similar to the sentence type. Then, in the middle bar, things get more complicated. The “letters” archetype involves the first letters of things spelling out instructions, so “capital city of Nation of Urrequ” becomes “CCONOU”, and so forth. The middle-right one is a formula, which might be pointing towards an actual formula, or a more abstract comment about how various things are related to or constitute something else. The right-most one implies spatial relationships between things, e.g. if you’re trying to activate a hidden staircase, then maybe doing something where the squares are relative to that staircase will make it happen. There’s also another clue type that will go in here later. Then, the final row makes things extremely abstract. I’ll talk more about these clue types in the future, but the key point is that all of these are contingent on the player building up, over many hours, a deep understanding of the game world, translations and meanings for large numbers of symbols, and a large body of information in books and elsewhere that can be referred to. These therefore have to come last.

The other advantage to this system, however, is in terms of player onboarding. Imagine you know nothing about this game, but you’ve seen a post on r/roguelikes or wherever, and you download 0.11. It gives you a bunch of clues to be getting on with, and tells you that the goal is to decipher them. You open your inventory, glance at your clues… and you see stuff from the bottom row. Cryptic riddle games are novel anyway, but a cryptic riddle roguelike is wholly novel, and that’s already a potential boundary or obstacle to a new player. Give them those straight off? I think a lot of players will bounce off. But, by contrast, imagine you open your notes inventory and you see a bunch of things from the first row. Even if you don’t know how to solve them, you instantly understand what they are, and what they’re doing. They’re showing places! Or they’re giving written information! We all understand what maps are, and we understand what pictures are, and (I hope) we understand what sentences are. This is an infinitely simpler and kinder on-boarding process not just because these clues are easier to solve, but also because the nature of the clue is just so much easier to understand. As such, we’re opening clue generation properly by starting with the top row, and in particular, in fact, the question mark clue there. I’m only showing one of these three new clue types off this week – the one that’s going to be the focus for 0.11, and which goes in the first group – but the other two are definitely interesting as well, and I’m looking forward to showing those off in some near future blog post.

But, with all that said, this week we’re going to focus on the new “poetry” archetype, which is going to the first one I work on to completion, and will form the foundation of the 0.11 release (though more might also find their way in, but I’ll talk about that in a minute). The poetry archetype is what you’d expect, i.e. a poem with several stanzas that gives you some cryptic information, and rhymes – this last part is very important. Here’s a mock-up I made of the sort of thing I want to get generating:

Where hunters search prairies and white boars dash,

Where the Fish-King’s slavers strike the lash,

Three pillars stand firm where black drips below,

In ancient worship to the god always slow…

And this is indeed the first generator I now intend to complete as the basis for 0.11. There are a number of reasons I’ve chosen this one. The first reason is because I think this is a riddle style that people are very familiar with, whether it’s from television and film, from books and literature, or from a handful of other games as well. Some of the others have been essentially created from scratch for URR, even if they might indeed have been lightly inspired by clues or riddles or puzzles of other sorts or found in other games, whereas the riddle-as-poem is something I think everyone is familiar with (although with that said, I don’t know whether this idea extends beyond English – though I assume it does, if anyone could give me information on this, I’d be very interested to hear it). As such, I think this’ll make it immediately clear to (new) players, even if they don’t read any of the game’s info, what they’re supposed to be doing and what they might want to investigate. The second reason is that it’s immediately accessible in a way that some of the more “advanced” or “involved” clue archetypes aren’t, i.e. some of them you can’t engage with in-game until you’ve already gained a substantial amount of information about what represents what, the use of colour and shape in that particular generated world, and so on. A poem clue, by contrast, requires no grounding and no prior knowledge on the part of the player, and can be given out and engaged with as soon as the game starts. The third reason, frankly, is that I think this will be one of the more “impressive” generators of the bunch to have fully implemented. Given that we’ve been fourteen years in development, and 0.11 will finally be bringing with it a (basic, initial) win condition, I’m anticipating and hoping to generate a lot more interest in 0.11 than some recent releases. Given download numbers and the like, a couple thousand players is not a totally unreasonable expectation here – what that means is that, to be crude about it, I want to make a good impression. If lots of people are going to be coming in for the first time, and I definitely intend to actually spend a bit of cash on advertising to help bring some attention to this forthcoming release, then I want one of the most impressive types to be front and centre and really grabbing attention as a completely novel and completely original game mechanic that simply hasn’t been seen before. I think all the clue archetypes are cool, of course, but I think this one is immediately appreciable in a way that some of the others might not be? I don’t expect every player to be hugely familiar with procedural content generation, of course, but this should still be clear and novel.

So: we’re going with poems for starters! My intention is therefore as follows: 0.11 will have a fully implemented poetry clue generator which leads to a win condition, and if I have time this year to implement other clue types fully, I’ll do so, but the priority to develop this one completely. 

But where to start? Yikes. So, the first thing was to establish a set of things that a poetry clue might be pointing to (and I’m sure this will be similar for others, but we’re starting from the foundations here and working up), and this pondering gave me the following: ‘Location’, ‘Instructions’, ‘Person’, ‘Item’, ‘Connections’, ‘Formula’, ‘Event’. Location is pretty obvious – a civilization, a settlement, a building, a specific location (e.g. “four steps south of the door”), or a cave, a place in the ocean, an island, and so forth. Instructions would be a set of directions to follow, e.g. that dropping a weapon one tile above and one tile below a statue of a certain god will cause some mechanism to activate. Person, again, is pretty clear, this would be describing either an NPC who actually exists in the world (and when they die of old age would get replaced by an equivalent NPC who still fits the definition), or a historical figure (or a god too, I suppose). Item is pretty clear, talking about a unique item (e.g. a relic, a crown), or perhaps a type of item one can do something with (so what kind of item do you want to use in a mechanism somewhere). Connections I’m still working on exactly, but would be giving information about things which are linked together, so many one clue would tell you which three other clues are talking about the same thing, for example. Formula is pretty self-explanatory, since I want the player to be able to discover secret formulae for secret concoctions with all manner of strange effects. And then, event, again, could be historical, or could be current, e.g. arriving at a certain place, in a certain month, during a certain phase of the moon, and a certain kind of weather, will reveal something. Each of these is huge, though, so I decided to start with location, and I broke it down into every possible type of location I could want a clue stanza to point towards:

Then I picked one of them to start with, and got to work. My thinking was to create one functioning example first at each level – so poem is the first kind of clue, then location is the first kind of poem clue, then civilization is the first kind of location, and then fully exhaust that and add in all the relevant parts of the generator and all relevant connections to other new aspects of code too (more on this shortly), make sure it all works, and then expand it upwards – so then expand to other locations to complete the “location” category for poems, and then expand it to other kind of information category to complete the “poem” category for clues. So for civilizations as a location that we want poems to be able to tell the player about, the first thing was to work out what kind of information would go into this calculation. The first thing were the policy / cultural choices for each civ, so things like being internationalist or isolationist, or punishing people with torture vs punishing people with jail time, and so on. There are eight categories and each has at least five possible variations, so this is already a lot of information that could point quite specifically towards a particular nation. Onto that I’ve also added the one “unique feature” that each civilization is going to get – this is another background thing I’ve been working on, but whichever one of those applies to a given nation also goes into the mix. That then gives nine data points (eight policies / cultures, one unique feature) to create a two-line stanza to describe a civilization. Then we can add the nature of their coinage (a tenth data point), what kinds of weapons they like (an eleventh data point), whether they use bows or firearms (so potentially an twelfth data point), any unique cultural thing the people do (e.g. scarification of faces, wearing turbans, etc, so potentially a twelfth or thirteenth data point). Then the question was: okay, how do I turn these into poems?

Well, after some pondering I decided to settle on a model where the first line would always be of the form “Where [one or two info] and [one more info with a rhyming final word]”, and the second would be “And [one more info with a rhyming final word]”. This was because it became immediately apparent that the truly challenging part of generating these poems was going to be making sure every stanza rhymed, and in order to do this I would need to know what the final word was going to be of the first line – which would differ a lot based on which of the nine features was being referenced – and then it would select a bunch of possible words for the rhyme, and try to find a fit. So actually, it’s not the information or the civilization or whatever that determines the poetry generation – at the deepest level, it’s the rhyme! So I then went through all of the possible features a civilization might have to be included in here, which was around a hundred things, and for each of them decided on a set of end-words for lines talking about that feature. It might be a bit hard to understand what I mean by this, so here are examples of end words selected for religious choices:

So if it’s trying to pair a description of the “collective faith” culture / policy / national feature, it can only produce sentences for that first line which end in “divines” or “shared”, e.g. “…and all worship the same divines” or “…and all belief is shared”, or variations on those. Once it has those for the first line, a first line might then look like “Where [one or two info] and all belief is shared”, and that then enables us to start thinking about a suitable rhyme for the second half. This then brings us to the creation of a huge rhyming database, of which this screenshot (which has since been alphabetised, don’t worry) is only the smallest portion:

You’ll see the first stanza word is at the start – “divines” is around fifteen down from the top – and then a selection of words that could usefully rhyme with it, and which I can imagine finding a way to end a sentence with, when that other sentence is describing another thing the poem needs to describe. For each of these – yes, every single one, on this list, which extends far beyond the size of the list I’ve shown in this screenshot – we then add a generator whose goal it is to see whether or not it can find something logical and interesting and relevant and distinctive it can use to attach to that final word. So, for instance, if we take a look at the rhyming words for “divines”, we’ll be able to see what I mean. It has five in there at the moment (again, as a reminder, this is just for describing a civilization, as a subset of describing a location, as a subset of all possible things a poem might be trying to describe), which are ‘Mines’, ‘Signs’, ‘Pines’, ‘Wines’, and ‘Vines’. For ‘Mines’, then, a possible rhyming sentence might be of the sort “And [people] [work] in [number] [descriptor] mines”, so this might yield a sentences like:

• “And slaves toil hard in four deepest mines” (for a civ with slavery and then noting that maybe those mines are particularly extensive)
• “And many work long in two deep diamond mines” (a bit more info might or might not appear about the specific mines, in this case both diamond)
• “And people now strive hard in cold emerald mines” (so this is noting two emerald mines, and they’re below colder terrain, so one might be able to find that info somewhere else)
• “And vassals use pickaxes in three almost connected mines” (so this would be a vassalage nation, with three mines that are very close to each other)

Here’s an example, where the rhyming word is “forgot”, and then the game looks to see whether any of the viable traits with a rhyming sentence are in the civilization trait list we’re trying to pair here, and then it puts in the sentence – each sentence is hand-written (and there are many thousands of these across many hundreds of rhyming words) but the game also generates the sentence too, so the actual wording will be differentiated further each time by the use of synonyms, even while the chance of seeing the same base sentence twice, even across multiple games, will be incredibly low (again – thousands of base rhyme sentence pairs). The “other” category will be for more advanced rhymes, e.g. those involving historical events, or real people, and so the game will have to seek those out – but we’re handling the top-level stuff at the moment. The goal of course is not just to ensure the seeing the same, or even similar sentences, is an incredibly rare event, but also – in the future – to enable players to share and discuss with each other what a particular set of clues might mean, and hopefully to have an overall database that is just far too large for anyone to singlehandedly conquer, thereby guaranteeing that genuine thought will be required on each playthrough, and/or people discuss with other players – both of which I would consider excellent outcomes.

But back to our example: most times you get the divines-mines pairing, these are the sorts of second sentence you’d be getting. You’ll note that some of these do refer to some of the core nine bits of info a civilization submits to the generator for a stanza, but others – like locations or types of mines – do not. The first sentence will always then reference just those nine core parts (2/3/4 of them) while the second sentence can be more varied, as I need to find ways to make the rhyme come together in a logical manner. However, this generator (like many others) can also produce other variations, like:

• “And the nation imports much from others’ gold mines” (a civ with gold coins but no gold mines)
• “And many work down the red and grey mines” (a civ with, perhaps, a ruby mine and an iron or silver mine, but that’s all)

And so on. So, to recap, those are the rhymes for mines, which is one of the rhymes for “divines”, which is one of the potential sentence ends for the collective faith religious style, which is one of the traits for civilizations, which is one of the things locations can point towards, which is one of the things poems can point towards… and you get the idea. Happily (!) some (all?!) of the other generators will actually, interestingly, be much simpler than this one, as I mentioned earlier in this point – I have chosen a hard one to start with (possibly the hardest, apart from the image generator one), but this is still nowhere near a full year’s work, and I think all the justifications and rationales I described above for starting with this one are very sound, so this is the first one we’re implementing, after which beyond 0.11 we’ll then be branching out into more and more types.

But this still hasn’t given us a finished stanza! We now have something like…

“Where [one or two info] and all share the same divines”

“And many work long in two deep diamond mines”

…yet that last part is missing. For this part, the game looks over the elements which have not yet been used – so in this example it couldn’t use the collective faith element – and finds a few short words to put in here. For example, for a nation really big into spying, you’d get something like this being added to a database of potential short descriptors:

While for a nation with the slavery policy, it’ll combine it with other policies to give a short couple of words that are usefully descriptive; in all of these you’ll see relevant policies listed in the []s in each tuple being added to the set from which a random selection is taken, and that is used to ensure we don’t get duplication, as you don’t want “Where many are slaves and many serve as slaves”, or whatever.

Once these are all added to a list (“shorties”) the game then cycles through that list, comparing them against used traits, until it finds ones its happy with, and adds one of them into the stanza. That then takes…

“Where [one or two info] and all share the same divines”

“And many work long in two deep diamond mines”

…and turns it into…

“Where many walk in shadow and all share the same divines”

“And many work long in two deep diamond mines”

…and suddenly, we have a civilization’s description! You’ve got two policy pieces of data in the first line –interventionist and collective faith – and then a geographical or informational element in the second lines (two mines, both diamond). The second is considered to be a harder piece of information than the first to acquire, and so it would only trigger for a poem beyond the most basic difficulty level, as I talked about these difficulty rankings of information last time (although now I think we need three rankings, for surface-level information, mid-level information such as how many mines a civilization has, and then specific information which can only be gleaned from certain books, certain NPCs, in certain places, and so on). With those pieces of data we’ve then successfully procedurally generated a two-line stanza, which rhymes, and which describes some element of a world which is itself generated, and does so in a riddle-y manner! So now, we just attach these onto the notes, and let’s just add a second pair as well for extra detail, and here are three entirely procedurally generated riddles, each describing a nation:

And here’s a breakdown of what each of these would be telling the player:

Now these are good… but missing something. What they’re missing is twofold – more specific information, and visuals that make the notes look a little more organic. So, here’s a mockup of where I want this style of clue to finish, including other paragraphs discussing a specific tile on the world map within that nation, and then the specific place within that world map tile where something can be found, and also some organic-looking damage to the note, which is going to be important to add to make things look a little more natural and less artificial:

Now that, I think, is pretty awesome! This screenshot above is basically what we’re aiming for, and it’s well along the way to being finished. The first two blocks are fully generated at present, with the game now able to find (by my count) several billion ways to discuss a nation in a poetic and obscure manner, and the latter two blocks I’ll be working on over the coming weeks within the broader “location” poetry archetype, before moving onto other poetry aspects. So this is where I’m up to at the point of writing this blog post – working through all the massive number of pieces that need to go into where for the location category of the poetry clue type. I’m extremely proud to say that I have indeed finished the civilization subset of the location category, but gosh, there’s still so much to do. It’s really exciting, though, and incredibly gratifying and satisfying, and I’m really cracking along in the terms of the speed of progress here. Soon I should be able to show off other aspects of the poetry generator (i.e. beyond the “civilization” category), though rest assured that they will all be plenty detailed. I don’t yet know whether we’ll get to other clue types in 0.11 or not, but I feel (not unreasonably, I think?) that a full completion of this clue archetype would itself be a huge milestone, and plenty to implement a first core quest around, so that’s where we’re aiming for right now.

What next?

Thanks for reading this entry, friends! I think this is incredibly exciting stuff, especially as this is only the start. Over the coming weeks I’ll be continuing to work on the riddle-poem generation, and working on all the other stuff for 0.11 as well. The next update will either be a discussion of the tutorial system I’m implementing in the game for 0.11, more discussion of continued progress on the riddle-poem generator, more progress on the appearance and generation of notes (e.g. some of that damage shown in the mockup above), or a major bug-fix entry, as those are the four things I’m concurrently working on at the moment. 

And…

…this next entry will be in two weeks, rather than three! Yes, yes, I know it’s bold, but I’m going to try the two-week update cycle for a little while now. I’m working great, things are cracking along, and even if the entries are a little bit shorter, I think I’d like to try the two-week update cycle for a little while again, on the basis of first and third weekends of each month. Let’s see how this goes, friends. As ever, if you liked this post, please do think about sharing it around on some online place where people are interested in roguelikes or game development more generally, and I’d love to read any comments or thoughts in the comments section below :). See you all in two weeks!