2019 MIT Mystery Hunt, Part 3: Metapuzzles

(This is a recap/review of the 2019 MIT Mystery Hunt, which happened last month. Puzzles and solutions can currently be found here. These posts will contain many spoilers.)

While I very slowly answer questions you don’t have, it might be useful to know that my Setec teammates and I answered questions you actually did have on Reddit last week. You can read the discussion here.

In the last post, I talked a lot about how we developed an intricate web of metapuzzles that all fit together into a nice closed map. One thing I didn’t talk about, which people have asked about here and there, is how we tested for the one element of our Hunt that was arguably most variable: solvers’ ability to determine which answers went with which puzzles.

The short answer is “not that much.” The Mystery Hunt is unique, and there’s no way to tell how 100 teams of varying sizes and skill levels will grapple with things until you actually find out in real time. And as noted earlier, we had to test metas as we went in order to know what fit on the map and what other metas still needed to be written; so when testing Halloween-Thanksgiving, we couldn’t show our testers all the other Halloween and Thanksgiving answers because we didn’t know what they were yet. One thing we did do for some tests, usually a second test after things had gone okay the first time, was introduce decoys (fake answers) in both rounds as we revealed answers to testers. But it’s one thing to have noise when you’re focused on a single meta.

The second thing we did was much later; a few weeks before Hunt, Tanis tracked down a handful of puzzlehunt-savvy solvers in the Seattle area who were not planning to participate in Mystery Hunt. (This is hard to do, as most of the solvers with the level of experience to succeed at Mystery Hunt are excited enough that they find a way to participate for real!) I’ll reference this below as the “Seattle Test”… We basically fed solvers a steady diet of new answers and had them focus on metapuzzle solving. At that late stage, we had no ability to cut metas or change puzzle answers, but we could use the results to decide if presentation should be tweaked, and predict how people would use solvent (as well as testing the website’s ability to process solvent).

The group had about four or five solvers present on average for one afternoon, which is not a good gauge for how the Hunt is going to go; as it happens, they solved three metapuzzles (and backsolved a few answers) but didn’t get very far beyond working on the metas that were in the “Town Square” (the five initial holidays) or connected to it. We did get enough data to conclude that (a) our early metas might be a little bit harder than we thought, and (b) figuring out which answers go where might be as well (though to be fair, this was something Chris Cieslik always thought would be more challenging than I did, so hat tip to him).

As a result of this, we tweaked the flavortext on a few of the metas to make them slightly more approachable, and we also decided to give out solvent a bit faster than we initially intended (probably about twice as fast). The Seattle Test group were initially skeptical of the usefulness of solvent, but once they started playing with it, it definitely seemed to help them make progress when they were otherwise stuck, which is what we hoped.

Metapuzzles Primarily Written By Me

Halloween/Thanksgiving: This was the first meta I wrote, and quite possibly the first meta anyone wrote; I came up with the mechanic in February and came up with a way to fit it thematically to either of the two Hunt themes I was advocating for! The combination of ABO and ternary seemed natural enough that I figured it must have been used somewhere else, but I couldn’t find a previous instance. The initial version had two key differences; the ordering was alphabetical with answers starting with letters from C to L, and the answer was A NICE STAKE, suggesting that the resolution was actually a way for the Thanksgiving Townspeople to ward off the vampires. The ordering changed due to overlap with the Halloween/Valentine’s mechanism, and after the Seattle Testers had difficulty noticing the new length-based ordering, we added a sentence hinting it. As for the answer, it was too guessable with few answers in testing, and we also decided our resolutions should be more peaceful… I believe Todd McClary came up with the excellent pun we used instead.

Valentine’s/Arbor: I had been interested in using the names of trees in a meta, and when one of our finalist maps needed a Valentine-Arbor connection, I came up with the idea of carving initials into the tree names. Initially I was only going to add two letters to each tree, but that turned out to be too constrained. It was also hard to make good bigrams out of the letters that formed good answers, so eventually I decided to put in shell elements (the X’s and O’s) that explicitly reordered the letters. The Seattle Testers didn’t make progress on this meta, but when I explained the answer to them I got worried that the mechanism might be too much of a leap, so I suggested adding a plus sign in the middles of the carved hearts to suggest the missing initials. TK didn’t think this was necessary, and ultimately I think he was right.

Holi/Pi: I’m not convinced the Trivial Pursuit connection here was my own; I think there was a test solve of a different Pi Day meta where testers were brainstorming connections to pi/pie, and someone mentioned Trivial Pursuit, but it didn’t lead anywhere. I kept thinking about it, and given that our meta interpretations of Holi were color-based, “changing” the colors of special edition TP categories seemed like a fun mechanic. (I also like both word-association metas and metas where a bunch of normal looking answers actually belong to the same data set, and this allowed me to play with some of each.) There were a lot of more unusual editions I hoped to use in this meta, such as World of Warcraft and Lord of the Rings, but it turns out TP changed orange to purple at some point, and so to use the “classic” set of six colors, I was restricted to fairly early editions. The exact form of indexing is dangerously close to being a “what am I thinking” puzzle… we tweaked the flavortext a lot to try to make it clear how the solver was meant to interpret things.

Some Comments on Metapuzzles Written By Others

Halloween/Christmas: I won’t get into as much detail about the metapuzzles other folks wrote (or even mention all of them), but I’ll share a few anecdotes/opinions here and there. The authors of this meta found that it required either ROBERT EDWARD CROZIER LONG (who is, basically, some dude) or IN THE SHADOW OF ZEKROM! (the name of a Pokemon episode) to conceal a TWO ZERO. We settled on the second, but nobody noticed until after a Halloween Pokemon puzzle was written with the answer NURSE JOY that we had stuck two Pokemon-themed answers in Halloween. Since that was the first round where metas would split, we wanted to avoid red herrings there as much as possible, so I insisted on switching back to Mr. Long, and agreed to write a puzzle with that answer myself.

Halloween/Valentine’s: Great mechanism, glorious pun answer. As Brian has said elsewhere, this was initially a Halloween/President’s meta with the answer HAIRIEST TRUE MAN (and using two separate grids, I believe).

Thanksgiving/Presidents: We saw in the Seattle Test that this had major red herring potential… Many of the turkey names contain food items, and once you convince yourself that it’s a food meta, how do you convince yourself that MAY THE FORCE BE WITH YOU goes here and FRANCIS BACON doesn’t? After that test, we made a subtle flavortext tweak to refer to “their” annual tradition, in the hopes of getting solvers to think about what presidents do for Thanksgiving. As noted at the wrap-up, the original answer proposed here was simply GEORGE WASHINGTON CARVER, and our testers guessed it purely from the flavortext (and implied answer length), so we changed it.

Valentine’s/Presidents: This was initially a Valentine’s/Thanksgiving meta (with references to “stuffing” and an initial answer of CANDY CORN BREAD) but that pair didn’t fit into our final map. But we liked the mechanic for an early meta, so this got rejiggered for Presidents’ Day. The mechanism stayed mostly the same from conception to delivery, but we did change the presentation of the extraction hearts from a line to a circle to make it harder to wheel-of-fortune the answer. The Seattle Testers struggled with this, causing me to propose changing “romantics” in the flavortext to either “couples” or “pairs of lovers”… I advocated for the latter, but the authors thought that was too much of a giveaway, and we compromised. This played slightly harder than intended, so I still wish we’d used the stronger hint, but ultimately it was fine.

Bloomsday/Arbor and Bloomsday/Pi: Those of us keeping an eye on the big picture got very nervous about the Bloomsday metas, particularly because the answers in Bloomsday were arguably the hardest to divvy up. The answers that went with Arbor were only notable in that they’d need a lot of Scrabbly letters to form a pangram; the others were odd-length, but in the initial version, there was no grid to even suggest that odd lengths would be reasonable. Leading up to Seattle, I was trying to convince the authors that we should give solvers the shape of the first grid… the Seattle Test didn’t get close to Bloomsday, but given the difficulty of the other metas, I thought this was a necessary change. We still thought Bloomsday’s metas would be a stumbling block, and our focus on that left most of us blind to the real bottleneck…

New Year’s/Patriots and Holi/Patriots: Although Patriots’ Day opened before Bloomsday, New Year’s/Patriots was the last solve for both contender teams. This was not because teams had trouble deciding which answers went where or even how to solve the meta; at some point Sunday afternoon we visited Left Out and Palindrome, and both teams were able to tell us exactly how the metapuzzle was meant to be solved. It was just very hard to do so with incomplete information. (There’s been some speculation about whether we intentionally wrote metas that required all or close to all the answers… we did try to avoid metas that could be short-circuited with less than half the inputs, but requiring completes was never a goal.) For a while, Left Out abandoned traditional solving and tried calling in various clever puns… little did they know that this was the one meta where we didn’t push very hard in this direction. The answer we used was a play on Lexington, MA vs Lexington Ave, but the rest of the phrase was kind of arbitrary. Guy Jacobson did say at some point that he thought they could make any 11-letter phrase work in the Game of Life setup… We just didn’t come up with anything of that length that worked better. Much of the Hunt was a dead heat between Left Out and Palindrome (though Left Out more frequently led on metas solved), and a big part of the final difference between them was the ability to work out where the answer should go and reverse-engineer an input that would complete the answer.

As for Holi/Patriots, I was an early advocate of this meta, as I really liked the concept and how it worked. We used Ben’s rings with minimal changes (apart from getting rid of a pair that clued the red ring with “OXBLOOD”), but Ben kindly let me take a second pass at a path that used the rings more extensively and spread out the letters used. I intentionally added some checksum statements that would help confirm the numbers of letters between intersection points, and I hope that helped solvers build the rings more confidently.

April Fool’s: We brainstormed this meta after all other metas were written, and while we tried a few different mechanics, I’m glad we went with the “holiday pranks,” which tied the various rounds together nicely and allowed us to draw solvers’ attention to the art for a while. As for options we didn’t use, if you find yourself trying to write an April Fool meta for your own holiday-themed puzzlehunt, I can confirm that there are a whole lot of pop songs with “fool” in the title, and a whole lot of Shakespeare characters identified as “fools.” FYI.

Metas, yeah! Next time, let’s chat about some individual puzzles.