2020 MIT Mystery Hunt, Part 3: Points of Contention

(This is a recap/review of the 2020 MIT Mystery Hunt, which happened this last month. Puzzles and solutions can currently be found here. This recap may contain spoilers.)

I haven’t managed to find time to blog recently, which is unfortunate because when I envisioned this third post, the main purpose was to encourage discussion, and I’m not sure people are going to be that excited to opine a month after Hunt. But better late than never, I hope.

[Since I also haven’t had time to post about other puzzlehunts, let me mention that I thought My Little Pony: Puzzles Are Magic was quite good! It had too much MLP-specific content for my tastes (you’d think that’s to be expected, but the FAQ was misleading in that respect), but the puzzles were fun, and the story took some twists and turns in a “GPH Lite” sort of way. Also be on the lookout for Cryptex Hunt 2020, which starts on Leap Day (much like the Revs season) and is apparently a novel.]

Before I totally run out of mental fuel to talk about this year’s Mystery Hunt (and the state of Hunt moving forward), I want to ask some questions that I think I have subjective answers. I have my own answers to them, which I’ll provide in some cases, and some of you may have answered them in your own way in the post-Hunt feedback form, but I think it’s often useful for solving teams to know what other solving teams think, and for teams to know what teams think about what teams think, etc. (Though everybody should keep in mind that the vast majority of Mystery Hunt teams are fairly casual and may not be reading a sparsely updated puzzlehunt blog. Hunt is for them too, so don’t assume any consensus reached here reflects the community ethos!)

I’ll number questions below so that it’s easier to refer to them in the comments.

Q1: Are there too many puzzles?

I loved most of the puzzles and metapuzzles I encountered in Hunt this year and thought the quality and polish was really solid on all of them… but there sure were a lot of them. It felt like a big jump in quantity, although comparing the number of puzzles to 2019, my back of the envelope calculation from a few weeks ago showed there wasn’t actually that much of a difference. Of course, I didn’t solve 2019, so maybe people felt the spike then, and I didn’t notice it from constructing HQ. I did think the average puzzle this year was harder/more involved than the average 2019 puzzle, but again, I don’t have the solver perspective. More teams finished “on time” this year than in 2019 by infinity percent, though teams in 2019 seemed more logjammed by metas, and I feel like in 2020 our team was never stuck for long on metapuzzles.

The traditional discussion in this realm is “when should the coin be found?” People have varying opinions about this: an earlier first finish means more teams will get to see the entire Hunt, but some will have a truncated experience, and a later finish means fewer teams will get a satisfying conclusion. But even if you agreed on a target, anyone who’s constructed Hunt can tell you this is something that isn’t purely a function of number of puzzles, and also something you’ll never be able to completely control. Actual Hunt teams are impossible to accurately replicate/simulate in testing, and I’ve helped write Hunts were the coin was found much earlier than intended and much later.

In any case, you can probably guess from my Part 1 post that I would have liked Hunt to be a round or two shorter so that my team could finish. I probably would have endorsed aiming for fewer puzzles in 2019 too, except that the structure we chose needed a lot of puzzles to support it… having metas split between rounds isn’t interesting unless you have a lot of connectivity between the rounds. Structural innovations are something valuable that set Mystery Hunt apart from other puzzlehunts for me, and sometimes those innovations require a large size. But at the same time, a long stretch of Hunts with many many puzzles is inevitably going to lead to an increase in average team size. And that’s something that (a) may not be sustainable on a campus of fixed size, and (b) makes it hard for casual student teams to get involved, and given that Hunt lives off of MIT resources, they’re the teams we owe the most to. More on that in a bit.

Q2: Should every team see every puzzle?

When we designed the “Santa curve” for 2019, one of our goals was that every team, regardless of progress, should have every puzzle unlocked at least an hour or two before close of HQ. We made pretty steady progress this year, and as a result I didn’t realize until afterward that (as far as I’m aware) Left Out didn’t time-release rounds*. This meant a team that didn’t solve a lot of puzzles early probably never saw the Outer Lands and missed a large chunk of story and innovation.

*[Edited to correct myself: As Wei-Hwa points out in the comments, my “as far as I was aware” statement was incorrect, and rounds were in fact time released. I think the main reason I gathered they weren’t is that three of the four largest rounds were batch-opened at once right around when the coin was found, so the isolated bits of information I heard from teams well behind the curve was that they got an impression of “Okay, folks, competitition’s over, now the rest of the Hunt is open.” I would still argue that given the size of these rounds, opening them at 1pm on Sunday makes it impossible for those teams to see all of the content in them, and opening them all at once diminishes the discovery effect, so the Q2 debate is still relevant to this Hunt. But I apologize for my false assumption above.]

There’s certainly a debate to be had about this, and the big unknown variable is trying to predict what will make casual teams happiest: focusing on early meaningful goals, or getting to solve whatever they want. There’s an easy argument that if you give teams all the puzzles eventually, they can make this decision for themselves. But there’s a counterargument that teams that are only solving a few puzzles may be overwhelmed/dispirited to have a pool of 100+ puzzles to wade through. Some constructors would probably like to encourage those teams to solve a meta (maybe their first ever!) and keep them in the shallow end so they can do that. On the other hand, some constructors probably want as many teams as possible to see what they’ve written, especially if they wrote the last puzzle released in the last round…

My own opinion on this probably varies based on the Hunt structure. When a new round or concept is revealed on the Hunt website, I get an endorphin rush, and I want every team to get that rush. So for this year, my preference would have been to have the round openings time-released to teams. (Maybe this did happen, but I gather it either didn’t, or it happened very late.) Within each round, you had to solve puzzles to unlock other puzzles, which means even as a strong team, some puzzles were never opened to us. I don’t object to that. I’d like teams to be able to see what the whole structure looks like by the end of the weekend, but I’m not fixed on everybody seeing every single puzzle. We did that in 2019, but due to the linear release structure, getting everybody every round wasn’t that different from getting everybody every puzzle.

Q3: Where do you put the story?

This is a nitpick, but 2016 and 2020 were the two Hunts I can remember that conveyed major story elements through videos isolated on the website. I take a lot of interest in the Hunt theme and story, but in both of these years, I never felt particularly driven toward the video page. As a result, these were probably the years that I felt least engaged in the story. This is, of course, my own fault, since I could have watched the videos if I wanted to (and the 2020 videos in particular, I discovered later, were adorable). But my interactions with the website and puzzles never really encouraged me to do that.

I also get the sense that a lot of story/theme/aura was conveyed to the people who went to squish pennies after each meta solve. I didn’t go to any of these, and the people who did didn’t tell us much else than that they got pennies. On a large team, a lot of members won’t end up going on any of those trips, especially since some team members like to volunteer for multiple pick-up missions. There was a nice skit when we opened the Outer Lands, and I appreciated that that skit happened in our HQ so that as many people could see it as possible. But it was something we watched rather than something we did, and the plot didn’t have a whole lot of urgency (especially once we established that the park was no longer closing).

I don’t want to send the message that I didn’t like the story or theme this year… I thought the kickoff (even after the wedding) was creative and well-written, and the theme park “lands” structure was intuitive enough that you didn’t need to follow the story to understand how the puzzles fit together. If you don’t care about story, the website and puzzle design made it easy to ignore the story and solve the Hunt. But as someone who does care, it was too easy for me to ignore it too. To paraphrase an argument I remember making to someone on another team I constructed with, Mystery Hunt is a set of puzzles with a story layered onto it, not a story with some puzzles, and so the story will (and should) always take a back seat. But I think it’s worth discussing narrative strategies to immerse solvers in that story as much as possible (without obscuring the puzzles).

Q4: Is phoneless answer confirmation the way of the future?

I dropped a question about this at the end of my Part 1 entry, and a lot of people already commented (as they did on Reddit). I think the question is fairly well-understood, so I won’t restate the problem, but it’s certainly an open question, so I’m including it again here.

I will say this: Virtually all of the people/teams I’ve heard who didn’t like losing the phones are Hunt veterans… I was concerned with the effect on newer teams, and they don’t seem to have had a problem, though by definition they wouldn’t have anything to compare to. And a lot of the veteran complaints have been along the lines of, “We expected to do X and have done X in the past, and we were bummed that we didn’t get to do X,” whether X is using the phone calls to keep team members aware when progress is made, or to check in with HQ, or to make wisecracks. Maybe some more advance notice might have been useful to smooth the transition, and I would encourage PPPGTPPP to let teams know in advance what to plan for next year.

I might get kicked off Setec for saying this, but I actually think the online submission is mostly better (and I especially loved the sound tags on solves). But there was one related issue that might be a dealbreaker. I’ll meet you at Q5.

Q5: How much guessing is too much guessing?

If you’re a regular reader (at least as much as I’m a “regular” poster), you probably remember my ranting about backsolving and THE WOLF’S HOUR in 2019. (I hope THE WOLF’S HOUR will join BE NOISY and RECTION in the die-hard Hunter’s inside-joke file.) I am by no means the most opposed person to backsolving/guessing, but I’m certainly not the least. That said, the captain of teammate wrote a great blog post about their Hunt experience that contained this little nugget: “during the minutes before the hunt started I encouraged everyone to submit a guess as long as they were >10% sure that it could be the right answer.”

The next paragraph begins, “I realize that this aggressive guessing strategy seems horrifying to some teams”… And I would like to confirm that YES, IT DOES. I realize this isn’t how it’ll happen in practice, but “more than ten percent sure” suggests that a team may expect to call in nine or ten answers per puzzle. With no live phones, that’s no longer something that’s going to jam up the lines and prevent other people from confirming answers. But the same post also includes this very relevant graph:

>=D=> >=D=> >=D=> GT >=D=> >=D=> >=D=> (yes, I am going to write that differently every time instead of looking up how to type a damn airplane symbol) is certainly a team to be reckoned with… They have great solvers, and I’ve loved all three Galactic Puzzle Hunts, and I was honestly rooting for them to win (sorry, Palindrome, I know you’ve been waiting) this year. But I find it very troubling that the two teams that guessed answers the most aggressively were two of the first three teams to finish, and in particular, in the first year without phones (and thus without a construction team potentially getting annoyed about answer spamming), the two most aggressive guessing teams had a notable increase in performance. I’m not sure that’s entirely a coincidence.

So here’s the thing. I think it’s up to the constructors to decide what parameters they think are reasonable for solving teams. On one hand, I think those standards should be communicated better to teams, and on the other hand, if teams know the precise standards, it could encourage some teams to engage in “unacceptable behavior minus epsilon.” And I suspect Trendsetters will have a pretty loose attitude about this given their own solving style. But I encourage all Hunt writers to consider that allowing teams to guess aggressively will encourage any team that wants to win to guess aggressively. And that will cause a lot of puzzles to fall more quickly. Which may cause constructors to feel obligated to write more puzzles. At which point I refer you to Q1.

Q6: How should non-student participants (and teams) approach Mystery Hunt?

This question is here for two reasons. The first is that the late-night-visitors-on-campus policy this year was a major event in Hunt history, and it’s worth discussing. The second is that I spent a lot of time before and after Hunt ranting at people on various social media platforms, and I want to collect those thoughts here for the record.

I did not state the question as something like, “What resources should MIT provide to non-students?” because no one creating or participating in Mystery Hunt ultimately has control of that (though the evolution of Puzzle Club is probably the best thing that has happened in years in terms of giving Hunt a stable voice on campus). MIT doesn’t have any obligation to support or continue Mystery Hunt, and they certainly aren’t required to host a large number of visitors on campus for the event. Yet they allow Hunt to use a large number of rooms, both for solving teams and the constructing team, and they also partially fund the event through various avenues.

Based on some of the posts I saw elsewhere and flipped out about, this relationship between Hunt and MIT is not fully understood. People suggested that Mystery Hunt is a vital part of MIT’s educational mission. It’s not. People suggested that Mystery Hunt is one of the biggest events of the year on campus. It’s not. People suggested that Mystery Hunt is a large enough event that Boston and Cambridge should be aware of it and provide support. It’s not. I think it’s a wonderful part of MIT culture, and it’s one of the most important events of the year to me (and has played an outsized role in my development as a human being). But it serves a niche audience, and we should not let ourselves believe the world revolves around that audience.

People also suggested that MIT is paying attention to the event design throughout the year, and so any policy changes they make are through some fault of the construction team. That’s not how this works. If anyone still believes any rule changes for late night HQs this year were because of Left Out, please understand that, in fact, the reason so many rules eventually were left unchanged were because of diligent work by Left Out and the Puzzle Club. I am very grateful for everything they did, especially in the last week when there are a million other things to worry about.

If you are not an MIT student–and I’m including myself in this even though I paid MIT four years of tuition back around the turn of the century–any resources, classrooms or otherwise, that MIT provides you are a gift. The Institute is not perfect (I refer you to a lot of other blogs and news outlets for more information on that), but they are doing the Mystery Hunt community a big favor. The best way to ensure that this favor continues to be done is to remember that, whether you donate to Hunt or not (and please consider doing so), you are a guest on campus. Enjoy the event within whatever constraints are put into place, don’t be entitled, and trust that the people who interact with MIT are doing what they can to include as many solvers as possible.

Sorry to end this post on such a ranty note, but Q6 has been a touchy subject for me this year. With that, I’m going to close the book on 2020 Mystery Hunt posting. If you have thoughts on any of the questions above (or other Hunt thoughts that you don’t have a convenient place to deposit), please respond below, and I’ll see you in the comments.