(This is a recap/review of the 2020 MIT Mystery Hunt, which happened this month. Puzzles and solutions can currently be found here. This recap may contain spoilers.)
As usual, I expect to devote multiple blog posts to the premier puzzlehunt of the year. But for the second year in a row, the Brown schedule is such that there’s no buffer week between Hunt and my teaching responsibilities, so I’m about to become quite busy. As a result, Hunt posts may be spread out, though I’ll try to keep things a bit more together than I did with Miskatonic. I’ll open with a post about what I’m feeling right now a day after Hunt and address content/structure/length/policies/what-have-you later, and if there are particular topics you think should be discussed here, feel free to bring them up in the comments.
Regardless of how good the content of this Hunt was (and I think it was very very good), it will be hard not to remember this year as both the first Mystery Hunt where I explicitly didn’t want to win, and the first Mystery Hunt where my team didn’t solve all of the metas. I realize the latter might be surprising, since the vast majority of Hunt participants don’t see the end of the Hunt before HQ shuts down, and so expecting to finish is admittedly somewhat entitled. But here’s a brief summary of my Mystery Hunt experience:
- 1998: Hunted with Setec, reached the coin location at the same time as Iliaphay, who found it during their turn in alternating shifts in the elevator
- 1999/2000: Won with Setec, wrote with Setec
- 2001/2002: Won with Setec, wrote with Setec
- 2003: Hunted with ACRONYM, and we were informed during the first phase of endgame that the coin had been found
- 2004/2005: Won with Setec, wrote with Setec
- 2006/2007: Won with Evil Midnight, wrote with Evil Midnight
- 2008/2009: Won with Evil Midnight, wrote with Evil Midnight
- 2010: Hunted with Attorney (A Team That Obstinately Replaces Names Every Year), solved the last meta while HQ was open, but they didn’t have resources to run endgame
- 2011: Hunted with Attorney, finished the Hunt
- 2012: Hunted with Attorney, finished the Hunt
- 2013/2014: Hunted with Attorney, wrote with Attorney [Alice Shrugged]
- 2015: Hunted with Setec, finished the Hunt (though I slept through the endgame)
- 2016/2017: Won with Setec, wrote with Setec
- 2018/2019: Won with Setec, wrote with Setec
Which brings us to 2020. After writing in 2017 and 2019, the leaders of Setec Astronomy polled the team, and we pretty much established we didn’t have the time/energy/interest to run a third (well, sixth) Hunt in 2021. So we basically said we would approach the Hunt in a more casual way. We said it would be okay to solve puzzles in parallel, people should feel free to get as much sleep as they wanted to, and people brought board games, though I’m not sure any of them actually got played.
In practice, the extent to which people took their foot off the gas probably varied from solver to solver. Early on when the first logic puzzle came out, Jackie and I solved it independently even knowing another group was working on it, but other than that I thought I was going pretty much my normal speed, especially once I started to suspect how big the Hunt was, as I really didn’t want to miss out on completing the endgame. After talking to Jackie, there are still ways in which I gave less than 100%… I slept a few hours more than normal per night (especially from Saturday into Sunday), and there were a few occasions where I sunk a lot of time into one puzzle, and in a more competitive environment, I probably would have abandoned ship and looked for things that were closer to an answer. (I am glad I got to see Star Maps through to its conclusion, but man was it a time sink, and I didn’t even help people assemble the jigsaw.)
As time went on, I became more sure we weren’t likely to finish by 5pm, but I didn’t think we were doing much more slowly than usual, so I was worried we were going to enter an awkward stage; if Hunt passed 5, we couldn’t finish without winning, and I wanted to finish but we were not willing to win. This became a moot point when PNGTOPN (plane noise Galactic Trendsetters other plane noise] officially won, but now we had a race against time to try to finish our last two rounds.
We didn’t, and at the end of the Hunt, we still had 4-6 puzzles in each of those rounds we hadn’t even unlocked. I spent most of the last hour of Hunt desperately trying to contribute to Cascade Bay by backsolving one of two missing answers that would have to fit into both Lazy River and Coast Guard, apart from about 25 minutes where we had a teamwork time open and I had to learn to speak Beale. At the time, I didn’t think we had a chance at finishing on time, though I thought we definitely could wrap things up given another five hours or so. But having been to wrapup, I actually think we should have been able to get in under the wire, given three things I discovered today:
- Our theories on how to solve the two remaining metas were pretty much exactly correct, but we just needed more inputs (in particular, for the Cascade meta, most of our answers had different lengths and didn’t pair up, and we had almost all of the PH halves and very few of the answers that actually provided meta letters.
- My Cascade backsolving was handicapped by the fact that we hadn’t put HELICOPTER DROP into the grid yet, and so I had been trying to fill an answer in where two went.
- At breakfast I discovered that one of my teammates had guessed a pun answer for Cactus Canyon based purely on the flavortext, but never called it in. It was correct. (Given our goal of not winning, I wouldn’t have wanted to skip an entire round while the coin was still active, but once Galactic found the coin, I desperately wish I’d known we had a guess, because if we’d been able to focus on Cascade Bay for three hours, I think we could have wrapped things up.)
I shouldn’t be frustrated about missing endgame; I had a great weekend solving well-written puzzles with people I like. But I have a perhaps unhealthy habit of characterizing myself based on accomplishments over time, and it’s hard to cope with the fact that I’ll never again be able to say I have a perfect record of finishing Mystery Hunts. But all things must come to an end, and so maybe it’s good to rip off this band-aid and not have to worry so much about getting to endgame next year, though I certainly hope we will.
In any case, the end of the 2020 Hunt was very anticlimactic for me, and I’m still strangely uneasy, but I suspect I’ll get over it in a few days, and then I can evaluate the Hunt from a more rational perspective. So I’ll check in again when I’ve chilled out.
In the meantime… answer checking without phones! Loved it? Hated it? Did the world end? Setec had a lot of members expressing displeasure about it, but for me, it was actually fine. I did wonder (once others brought it up) whether it made Hunt less welcoming for newb teams.
Puzzlvaria 
i thought this hunt was outstanding, top to bottom. mostly this is based on good, fun, clean, unbroken puzzle content. but in terms of the innovations of this hunt:
1. i thought answer checking without phones was fine. i appreciated that it allowed left out team members to do more interaction with the solving teams, and it did seem like there were more puzzles that involved interaction. that’s neat.
2. the “teamwork time” puzzles were awesome. no matter what puzzle i was working on or how much fun i was having, i greatly enjoyed every instance of dropping everything for 10-15 minutes to do those puzzles with my whole team. that’s a terrific innovation that i hope will become a hunt mainstay.
LikeLike
At first I was somewhat disappointed by the news of automated submissions, because I really do enjoy the callbacks and the fun phone gimmicks and the interaction that provides, but I think I grew to accept the automated answer checking. Specifically, I agreed with what was mentioned at wrapup that the sound jingles for correct submissions were awesome! I also enjoyed the accompanying popups for what things happened.
But I think the big boon from moving to automated submissions was how I’m guessing it freed up the time of people in HQ. In particular, I suspect the hint system they used this year was only possible without needing people for answer callbacks, and I thought the hint system was absolutely outstanding. I cannot overemphasize how much this hint system increased my amount of fun in the hunt. Coming from a mid-level team, the hints were mostly used on puzzles that already had a lot of work, and had been open for a while. In my opinion, the worst feeling in any hunt is when a lot of time is spent on a puzzle, but it languishes in extraction hell, etc. The freedom to use hints on these puzzles without feeling worried about wasting a valuable hint was amazing, and led to significantly less of this feeling over the course of the hunt.
I also think a great feature of this hint system was the ability for HQ to give “gentle nudges” as they called it, but with an invitation to ask for a more explicit, traditional hint. In a hunt where the hint system is some finite number of hints based on a resource, there is pressure on the hinter to make sure the hint is actually useful, so it tends to be more explicit on a next step, etc. But I really appreciated still being able to get ahas after only a gentle nudge, and the few times when we were still stuck, we were able to ask for another hint.
I think this hint system solved the problem of being helpful for smaller teams, while not getting in the way of competitiveness for the top teams. Normally the problem is that giving more hints to smaller teams encourages not submitting answers in order to hoard more resources. While I don’t know exactly how exactly Left Out decided to give out hints, it seemed like we were generally getting them on puzzles that we had open for a while, which was both giving hints to where they need them without creating weird incentives.
Clearly the downside of this hint system is that it probably required a lot of human resources to give hints, but this happened to work because of moving away from answer callbacks. I think this hint system provided a level of interaction for smaller and newer teams that also made the experience way more fun.
I think losing phone callbacks is ten times worth it if we get this sort of hint system every year.
LikeLike
I didn’t get to interact with the hint system (we weren’t eligible for hints on any puzzles that were useful to us until very very late in the Hunt, which is as it should be since we weren’t far behind the lead pack), but from the description, it sounds like a formalization of what’s actually happened with lots of teams informally in previous years. For the last couple of Setec hunts, we were pretty open to hinting any team that wasn’t likely to win to whatever extent they wanted, either by phone or during team visits. But if a team didn’t ask, or the right people didn’t check in on them, we might not have conveyed that effectively to teams. In 2017 especially, because we had a gold-for-hints mechanism for all teams to ensure fairness, we probably implied to a lot of non-contending teams that hints weren’t avaiable to them outside that system, even though they were.
LO can correct me, but it sounds like their approach was essentially to make puzzles hintable over time rather than deciding when teams were hintable, and that seems like a good metric. In my opinion, the primary goals of any Mystery Hunt hinting system or policy is that hints should not affect which team wins the Hunt, and that aside from that, teams can get sufficient (and appropriate) hinting to keep them enjoying the Hunt. From everything I’ve heard, Left Out hit it out of the park in both respects. But I also think it’s possible to do that and answer phones for answer confirmation. The latter doesn’t eat up as much energy as you might think. So while I don’t necessarily think phone confirmation should come back, I don’t think there’s necessarily a choice between that and good hinting.
LikeLike
I play on a small team and I can say that I loved both the no-phone system and the hint system. For no-phone – it’s faster (it’s very annoying having to wait around 5-10 minutes sometimes for a call back when we’re excited to know if we’re right). It lets us be less self-conscious about a dumb guess. It avoids the problems of the person with the main phone number being off at an event or whatever. We get the fun celebratory sound when we’re right. And it had a handy list of our previous guesses so somebody could come into a puzzle late and see what’s been tried. I also assumed that it would let us submit “guesses” to the website after the event so we could see if we were right without having to look at the Solution page, but that doesn’t seem to be the case?
As for the hints, I thought this was the best hint system ever. We never ask for hints unless it’s explicitly a Thing – it seems like cheating to just call and ask for help. Or if there’s a currency that you have to spend for hints – we never do it. But here – explicit “you’re allowed to ask for a hint now on Puzzle X”, with quick and helpful responses – I loved it.
In fact I thought everything about the website of this hunt was excellent.
LikeLike
Hints unlocked over time. Someone should be more precise, but generally hints were unlocked when at least 30 teams had solved a puzzle and one of the remaining teams had had it open for longer than either the average or maximum solve time (I forget) of the other 30 teams.
And yeah, we liked hint opportunities being structured. Structure allows for equity, as S is saying below.
And I disagree that phone confirmation wouldn’t have been a big deal. For the phone interactions we did have, it was a royal pain contacting teams- not the top teams, but the newbs and the small ones. And in any case, phones in general are a ruder and less reliable way of contacting people than methods based on electronic text.
LikeLike
“And I disagree that phone confirmation wouldn’t have been a big deal. For the phone interactions we did have, it was a royal pain contacting teams- not the top teams, but the newbs and the small ones.”
I mean, I’ve done it, so I know it can be done. On one hand, I think the growth of Hunt (especially allowing all-remote teams, which was not a thing until a few years ago) has increased the number of teams who may not know what they’re getting into and may not pay attention to Hunt dispatches. On the other hand, if a team knows in advance that they’re going to be called every time they want to make progress, they’re going to take answering their phone more seriously. I suspect that in a more traditional format, fewer teams would have frustrated you in this way, but the ones that did would have frustrated you much more frequently.
For my part, I strongly disagree that phones are objectively “ruder” than text communication. You can prefer not to talk on phones (and many people do), but that doesn’t make calling someone rude, especially if it’s as part of the event you’re running that they signed up for.
LikeLiked by 1 person
Human resources was definitely part of the no-callback decision, and we were able to more fully staff other areas of hunt. At least the way this year played out there were some peak load times when we only barely kept up with all the competing demands, and those would have also been peak times for answer callbacks – something would have had to give.
I think Dan is right though, if we felt callbacks were very important and meaningfully improved hunt, we’d have found a way to make them work. And we did hear from some people who feel that way, but we also heard from others who saw callbacks as annoying and archaic and were glad to see them gone. I genuinely don’t know how it balances out and if we made the right decision, and I’m sure Galactic will be paying close attention to feedback that comes in from people on both sides. That’s the process by which hunt improves over time – teams try new things, the good ideas stick, and the bad ideas get rolled back the next year. The lack of callbacks I’m pretty sure didn’t *ruin* hunt for anyone, but whether it was on balance good or bad, I dunno. I’m glad it got tried at least once though, and I’m proud that our implementation was slick enough that we won over at least some people who wouldn’t have expected to find it to be a change for the better.
LikeLike
Hints: I was on LeftOut and did a few “wellness checks,” as we called them, on teams. Most of them were smaller teams. Of the teams I talked to, one said they disliked the switch to automated checking because they enjoy saying witty things on the phones. Others said that they either liked it or were okay with it because:
– They have a lot of remotes, so the person who answered the call may not have any idea what puzzle it was far or who needed this information. (This was a perennial problem for LeftOut, actually; somehow our West Coast was always getting the answers for East and vice versa).
– The person who was supposed to be answering their phones had it on silent/ kept losing it under stuff/ etc. One team thanked us so hard because last year, they missed so many answers because they didn’t realize it was on silent. (I will say this year when we did try to call people, for like Concierge Services or for scheduling visits, sometimes it was extremely hard reaching them! Plenty of young people don’t actually talk to people on phones any more.)
– It frees up a person from being on phone duty, so they can solve, which is more fun than waiting on a phone.
LikeLike
Er, that should be “answer checking” above.
Hints: I was on hint duty too. Typed hints allowed people to paste in their data or link to their spreadsheets, which was super helpful in hinting them correctly.
LikeLike
“It frees up a person from being on phone duty, so they can solve, which is more fun than waiting on a phone.”
There are a handful of people on our team who disagree. For the past N years, we’ve chosen a team name that allows for a different comedic interaction when responding to the phone calls. (This year we were going to ask you a different obnoxious census survey question like “How many VCRs are currently in your household?” every time you called.) It’s also a thing people can do for a bit when they feel burned out by puzzles they’ve been stuck on for hours. The lack of phone calls really reduced the amount we felt connected to HQ and kept us from submitting our usual jokingly-trolling answers to try and get a laugh out of y’all. (Not sure if any human saw my “Go Blue!” and “Buck the Fuckeyes” submission for the Ohio State marching band puzzle that this Michigan alum simultaneously loved (as far as puzzle construction goes) and hated (as far as puzzle content goes — 35 minutes of watching the Ohio State marching band troll Michigan hurt!).) I’m getting from the comments here that this kind of interaction was supposed to be done via the hinting system, but since, like Setec, we were far enough ahead to never be eligible for hints, we didn’t get to experience that.
I loved most things that Left Out did this year, but taking away the phone calls was a strong negative for me.
On the short list for next year’s team name is definitely “Call Us Maybe.”
LikeLike
Looking at what some of the people from small/new teams are saying, I think we really got stuck at an unsweet spot in terms of interaction. It does sound like the hinting system provided a lot of the missing interaction – but if it’s going to do that, I think more attention needs to be paid to the teams that are too far up to get hints but not far enough up to get extra attention because they might trigger the runaround soon. Not to mention teams that might not want to use hints for whatever reason but would still appreciate/benefit from more interaction with HQ.
There are a lot of places where a few phone callbacks or similar could be designed in, depending on the structure – having to talk to HQ to unlock a new round, for example, or having meta answers (but not non-meta puzzles) on a callback queue instead of automated submissions. I get that not every writing team will have the manpower we did that allowed us to have a full phone bank plus run near-constant character interactions for every round, but there must be many ways to have a happy medium.
LikeLike
Yes! I saw your “Buck the Fuckeyes” submission. My first reaction was “that’s a really funny submission!” My second reaction was “but I have no idea what it means!” I then told the whole hints team about it, and someone pointed out you were almost certainly from UMichigan. So yes, it connected with us, just not in the real-time way that a phone call would have.
LikeLiked by 1 person
DAMN IT, DAN, WE ARE NOT THE “TETAZOO TEAM.”
Signed,
All of the putzen who out number tetazooans on the team, plus all the people who are neither.
LikeLike
firstputzazoo
LikeLike
Am I wrong in thinking/remembering the team was *started* by Tetazoo, though? That’s certainly what we were referring to you as back in 2000, though that could just have been an Ann/Ray influence.
You could easily disambiguate this by picking a damn team name. 😉
LikeLike
Anyway, fixed it. Congrats on your law degree.
LikeLike
Technically, you are correct. But starting in about 2003 or so, the response to calling us the Tetazoo team causes all Putzen to put up arms and be indignant. I was just following suit. Your acronym is compelling.
LikeLike
It was definitely started on Tetazoo. As a non-Tetazoo member I’m not offended by anyone calling it that for simplicity’s sake, but maybe if I were from a hall with higher representation like putz I would feel differently. Nobody will ever refer to us as Team Pi e Stickman so I have no horse in the race…
LikeLike
I thought I’d hate the automated answer checking, but I actually loved it. Mainly this is because the celebration sounds were just so delightful. I think it was also nice for whoever was on “quartermaster” duty to be able to solve puzzles and not just answer the phone. I was worried that automatic rate limiting would be problematic, but it was finefor us. I still worry that it might be a pain on big student teams with lots of newbies, so I’d love to hear if there were problems on other teams.
LikeLike
On your main topic, this is a bit different, but this year Plant is having to come to grips with the likelihood that we’ll never finish another hunt. We had a good run of over a dozen years where we typically finished or came close, but both this year and last year we were nowhere close to finishing. And looking at the graph we’re well outside the top 10 and a full day off the pace of the leading teams. It’s a different experience being in the middle of the pack and I don’t enjoy it nearly as much. Especially in a hunt like this one where most of the “innovative” structure was late enough that we barely saw it. It was a great hunt, but I kinda wish things like the animal round had been moved earlier. For the video game hunt we put Megaman as the second world because we thought it was the most memorable structure so we wanted as many teams as possible to see it. That said, of course it’s not LO’s fault that my team has slipped from five years ago. If we can’t stay within 12 hours of the leaders then we’re just going to have to get used to having a lot we don’t see.
LikeLike
There is a balance. But we wanted to put the simpler metastructures earlier because there are a bunch of teams that never solve a meta, and we wanted to give them a chance to finish at least one meta, or if they’re a little better all the inner lands.
LikeLike
Thanks for taking the time to reply. I agree with what you say in general terms, and I certainly am not trying to argue to put those complicated structures *first*, for exactly the reasons you gave. I just would have preferred for the “showcase innovations” to start closer to a third of the way into hunt. Similarly if the “inner lands” were a little smaller then both more teams would get the accomplishment of finishing them *and* more teams would get to see the emoji round and the animal round. I can’t speak for other people, but I certainly would have felt more accomplishment finishing either of those outer lands than I felt finishing the inner lands (which didn’t feel all that different to me from just finishing another round).
LikeLike
Yeah, that explains a lot about our progress at the Transplantations (we finished one spot ahead of the Plant and I’m still trying to wrap my head around not being behind the Plant). We got 7 metas by Saturday night, I think? But then we never got any additional metas, and the last rounds were very big. Our rate of progress through Saturday – 7 metas down before the coin was found – looked like we were going to be top 10, when in reality we were never there.
LikeLike
I really hope that “[will] never finish another hunt” is not true, either for Plant or for Central Services (my team). I was kinda shocked at how we just kept opening more content, and then shocked again at how far we were behind the leading curve: a very similar position to Plant. None of which is bad except for the reasons Noah elucidates: a ton of stuff we never saw, and no hope at all of finishing. I will still *hugely* enjoy Hunt even if that remains the case ad infinitum… but it would be nice to be able to hope :-).
LikeLiked by 1 person
I really enjoyed the bulk of this hunt! Many thanks to Left Out and Puzzle Club for putting on such a delightful event, and to Mark and Gaby for asking me to be in their wedding!
I’m commenting because I strongly disliked the elimination of phone callbacks for answer submission. (I don’t think sharing my thoughts on the benefits of phone callbacks for constructors is useful here; suffice it to say that they exist.)
First and foremost, the change removed the whole-team feedback step for correct answer submission. In previous years, when an answer was submitted, the whole team had to get quiet so we could hear Hunt HQ on the speakerphone, and we all got to celebrate together when an answer was confirmed.
That phone break also helped me have a sense of how we were doing, and provided a natural launch point for grouping up on whatever puzzles get opened. Much of our team did not get to participate in Teamwork Time puzzles, because it wasn’t obvious that they’d opened and by the time folks noticed the puzzles would already be mostly done. And Teamwork Time puzzles are a great idea! Microsoft Puzzle Hunt has done them in the past and they are an excellent palate cleanser and re-energizer.
I also appreciate the chance to talk frequently with the constructors, even on a quick call. I like feeling like this event is interactive, and talking to Hunt HQ lets solvers share their joy (and their frustration.) I didn’t feel particularly connected with the constructing team during this event.
Fundamentally, though, this answer submission system is functionally similar to that used for most online puzzle hunts, and I am protective of Mystery Hunt’s uniqueness. This event is the Super Bowl of puzzle hunts, and I want attending it to give you as many things you can’t get elsewhere as possible.
LikeLiked by 1 person
Ditto everything Tanis said.
Add to it the thing I said about how our team name incorporates the phone calling as part of the event and that’s a big part of our team culture. Then you have pretty much all of my thoughts on the matter.
LikeLiked by 1 person
It sounds like there are two main types of reasons that people like answer callbacks.
One type boils down to a team enjoys them and finds them useful. The other is that they are part of the culture of Mystery Hunt.
If the running team allowed per team opt-in (or opt-out) answer callbacks, I wonder how far that would go towards satisfying the folks who want to keep them. It seems likely to help with the “our team culture and process work better with callbacks” but perhaps not at all with the concern that “Hunt culture includes callbacks”.
Thoughts?
LikeLike
I think then you would get into the slight time difference between automated answers and callback answers, which could theoretically be just enough to give one top team an edge over another one, which is generally one of the main things you’re trying to avoid when running the hunt. Yes, you could argue that the team who opts into callbacks has to accept the consequences of that, but generally anything that might introduce that kind of inequality between top teams, especially late in the hunt, is going to be more trouble than it’s worth.
LikeLike
I think an opt-out system would be great for solvers. It might solve some issues we’ve had in our more recent constructing years with all-remote teams (contacting them can be a challenge) but there are fairness concerns for competitive teams.
LikeLike
In addition to Tanis’ points, I think it’s worth discussing how automated vs. callback-based answer checking affects the “cost” to the solving team of checking an answer, and the impact that may have on solving strategy. I’m thinking about the time cost to the solving team of waiting for a callback response, and the psychological cost of knowing there’s a human taking time to consider your submission.
My theory (which could be wrong!) is that callback-based answer checking discourages incorrect guessing (more so than automated time-based lockouts), and as a result encourages actual puzzle-solving, which I posit is a desirable thing for solver satisfaction.
It feels like the answer checking process is evolving more from a way to simply confirm an answer that one already believes is correct, to yet another provider of constraints (in addition to the puzzle content, knowledge of other answers in the round/meta, etc.) used to narrow down the set of possible answers to a puzzle. Sure, this happens to an extent with callback-based answer checking as well, but I think automated answer checking pushes teams to rely on this more.
By codifying the amount of allowed guessing via automated answer checking lockout times, we make the submission of (some number of) incorrect answers totally free, and implicitly “grant permission” for teams to guess answers up to that limit. Approaches such as the “20 total guesses per puzzle” system employed by the online Galactic puzzle hunts have a similar effect. Introducing more severe limits or lockouts has been mentioned as an alternative to discourage incorrect guessing, but is too punitive for “good-faith” incorrect guesses (e.g. due to ambiguity in the puzzle).
All that said, if teams find getting immediate feedback for guessing answers to be more fun than waiting for callbacks, that’s what matters at the end of the day, but personally, I find the suspense, interactivity, and “whole-team feedback” aspect of callbacks that Tanis mentioned to be more fun than a sound effect, and I feel that the “costs” of callbacks encourage me to solve puzzles more fully.
LikeLiked by 1 person
Thought this was a wonderful hunt in pretty much every aspect. Though I was in one of the teams that finished so I likely have a different perspective than many others; I can’t even imagine how to make a hunt to please everyone. Maybe in addition to hinting, opening up rounds on a timely basis for all teams to at least allow them to see the variety of puzzles before the hunt closes?
And I loved the automatic answer system as well as the sound effects. Though I’m not positive how the rate limiter worked – it seemed to me like we submitted several logical forward-solve attempts on a few of the emoji puzzles and got locked out for a substantial amount of time (15-20 minutes?). I do think there should be a submission rate limit, but also someone closely monitoring violations of the limit to check how much a team deserves a penalty.
Curious – has it at all been decided how many backsolve attempts a team should be able to submit within a given period of time? How about forward solve attempts when you’re missing some amount of letters but have a reasonable guess?
LikeLike
“Though I’m not positive how the rate limiter worked – it seemed to me like we submitted several logical forward-solve attempts on a few of the emoji puzzles and got locked out for a substantial amount of time (15-20 minutes?).”
Our experience on Setec was that we got rate-limited much faster in the emoji round than in any other rounds. (For example, when we submitted VITRUVIAN MAN, which was a natural wrong answer for Teamwork Time, we immediately got a 10 minute penalty… Meanwhile, I remember submitting two wrong answers in a short period of time for Wolf and not being penalized.)
LikeLike
That sounds similar to our experience. Not sure if it was by accident or design. Am also curious whether the penalty applied per puzzle or per overall guesses (was there a penalty for spamming a backsolve try to 10 different puzzles?).
LikeLike
I think it makes sense to not tell teams the exact rate limiting rules in advance, but now that hunt is over I do think it’d be really helpful for future writing teams to hear exactly what rules LO used.
LikeLike
The rate limiter worked by giving you a “free guess” when you unlocked the puzzle, and then another free guess after 6 minutes (or 20 minutes for the emoji round, which Dan brought up). So after an hour, you would have 10 guesses before you started getting limited (and we capped the maximum number of guesses that you could ‘bank’ at 10). Our goals were to limit dictionary-attack guessing (MAJESTIC PIGEON? MAJESTIC PANDAS? MAJESTIC PHLEGM?) and to mitigate the case where a new puzzle unlocks, you know the three possible answers, and you try all three without even looking at the puzzle. We were also hoping that teams that were honestly trying to forward-solve a puzzle would very rarely run into any sort of lockout. When we pulled data for previous years, it seemed like instances where teams submitted more than 10 reasonable guesses for a puzzle were rare (though we miscalculated a bit with the emoji round).
– We had the ability to manually reset the lockout. During the Hunt, when it became clear the emoji a-ha was challenging, we had a script go through and continually auto-reset anyone who hadn’t gotten the a-ha and had only so far submitted text answers. We also monitored when people were being locked out and tried to reset them as best we could manually.
– We also had a concept of “honest guesses”, which were reasonable guesses that meant to us that you were trying to forward-solve the puzzle. These didn’t count against your lockout.
The thing I would change for next time would be to add more control to edit the rules on the fly. 20 minutes was too long for the emoji lockout, it would have been nice to have per-puzzle control on the rules, and I would have liked the ability to “add” honest guesses like ☮️to the Masked Images puzzle. Other than that I think it worked reasonably well from our side.
Thank you for playing and for opening up a good forum for long-form discussion on things like this 🙂 We had a lot of fun writing this Hunt!
LikeLiked by 1 person
Out of curiosity, why did the emoji round have a longer delay? Just because there are fewer emoji than words in the world?
LikeLiked by 1 person
The emoji round had a longer delay because it’s a finite set of data and we feared it would be too easy to just start blindly backsolving puzzles once they appeared (assuming you knew how the mechanism worked). So if you knew Cast Away was going to be in there because of “package” and “island,” then you’d be able to easily guess “volleyball” on all open puzzles without too much of a penalty. It wasn’t ideal as we knew we might be hurting teams who were legitimately trying to solve puzzles, so we put a clause in the rules to tell teams to contact us and we also were manually unlocking teams that we felt were legitimately trying to solve the puzzle. In retrospect, it probably should have been a smaller time penalty.
LikeLike
I was in favor of the automated answer checking. I have always found the phone callbacks to be an annoying archaism, maybe because I never enjoyed participating in them from either end. If the point is to have the constructing team interact more personally with the solving teams (which I agree is very valuable), I think LO made the absolutely correct calculation that you can have more and better interaction by devoting your human capital to character visits, events, and things like that, versus the relatively rote “Hello, is this foo? Your answer of bar is incorrect.” “OK thanks.”
I actually was expecting (and would have liked) the rate limiting to be even more aggressive. I was thinking that you should be locked out for a short time even after the first wrong answer on a puzzle, increasing the length of the delay with every additional wrong answer. In general, I like for it to feel like you’re incurring a penalty for every wrong answer.
I solved on a small team this year, which meant we saw almost none of the later rounds, which was where the more interesting mechanisms lay. I am torn about this – I am a bit disappointed that we didn’t get to see a lot of the puzzles, especially given there were stretches where we went several hours without getting any new puzzles. On the other hand, keeping us confined to the “central” regions of the park almost certainly allowed us to solve more metas than we otherwise would have, by keeping our solving focused on the more tractable areas.
LikeLike
Re: “In general, I like for it to feel like you’re incurring a penalty for every wrong answer.”
I feel very strongly the opposite. You should be penalized for bad-faith abuses of the system, sure. But it’s not always the solver’s fault that they’re calling in a wrong answer, and it’s doubly frustrating to be penalized in that circumstance.
LikeLike
“In general, I like for it to feel like you’re incurring a penalty for every wrong answer.”
I definitely disagree. If I’m really struggling on a puzzle, and I’m getting frustrated, but I have something that could maybe be an answer if I just had a couple more letters but I have no way of knowing if it will be or if I’m on another wild goose chase, please at least let me get some closure on that bad idea, y’know? Sometimes a guess or two is part of the solving process, or even just a pressure release valve (sometimes when you’ve been wrestling with one puzzle for hours, “I know this isn’t right, but hey I bet it’ll make someone at HQ laugh and who knows maybe there’s a tiny chance that it IS the answer” is just something you need for mental health purposes).
Even if the penalty is fairly small, I would feel as a solver that the writing team was sort of setting up an adversarial relationship there. “We don’t want to deal with your silliness or your stupidity; we only care about your right answers.” That may or may not be at all what they’re thinking, but I would feel very put off. Especially since it’s not like there’s even a call queue that’s trying to balance all teams – literally the only reason to punish a team for every single wrong guess is to send the message that wrong answers and guesses are not okay.
LikeLiked by 1 person
While you may be upset to have lost your streak, I’m happy to learn 17 years later that ACRONYM solved all the Matrix metas — I’ve spent think time thinking “we never solved the real world meta” but now that I look back, it was entirely part of the runaround and not really a standalone meta!
I thought I would miss answer checking (and I’ve enjoyed doing the random greeting thing on other teams as well) but I didn’t as much as I thought I would. And having more staffed puzzles seems like a good tradeoff (though it didn’t benefit me as a remote solver, I don’t really think Hunt needs to benefit remote solvers at all).
Speaking of remote solvers, I was actually surprised to learn (from seeing a few puzzles that gave workarounds) that all-remote teams are supported at all! (The team I joined this year was a primarily Oakland-based MIT-alum team with a much smaller group on campus.) I simultaneously appreciate the ability to work on puzzles like Penny Park Guide and Catenoid from far away while still being skeptical that more focus on those of us who aren’t there is a net positive for the event.
I agree with Roger and others that, on a smaller team, not getting to experience the emoji and Safari rounds was a shame, but I can’t fault the choice of giving us a chance to make it through more straightforward rounds.
All in all, I had a wonderful time. The theme was great and consistently enticing, the puzzles were fun, the hint system was perfect, and from the snippets I gathered, all of the interactive on-campus puzzles and events were fun. Looking forward to seeing what ✈✈✈GT✈✈✈ pulls off next year… which I’m especially looking forward to after the last three Galactic Puzzle Hunts. (And if I understand correctly, they’re a primarily MIT alum team that’s a bit younger than most of the recent MIT alum team winners, which is exciting!)
LikeLiked by 1 person
I wonder how much of the discussion of phone calls has to do with individual teams’ cultures, because while this was only my second hunt, last year I never noticed at all that phone calls provided interaction with HQ– the QM got the callback, whether it was right or wrong was entered into the team slack by the QM, and… nobody ever saw whatever happened on the phone besides the QM. From my perspective it’s functionally identical to what happened this year except with immediate feedback, which is substantially better. Meanwhile the character visits everyone *did* notice and enjoy; most of the team saw them.
We also had a few periods this year where we didn’t have a QM on duty for a few hours at a stretch or in the very early hours of the morning, which would have been significantly more difficult to pull off if someone had had to be Officially By The Phone.
LikeLiked by 1 person
The ethics of guessing/backsolving is still interesting to me. From our work in progress on the emoji meta, we were fairly positive that there had to be a house somewhere. And that confirmation would have helped. So I submitted the house emoji to the two puzzles that seemed the most thematic, but then refrained from the rest of the round so as not to be spamming. Turns out it would have been right on one of the other puzzles, so spamming would have saved us some time and effort.
I’m sure what we did here is more than some other teams are willing to do, but less than others. And certainly spamming to the extent allowed gives a team an advantage, not that we were willing to find where that limit was.
LikeLike
I’m curious to hear peoples’ thoughts on the theme. For my part, and what I understand from my team, the theme *really* fell flat this year. A few thoughts on why that is:
1) lack of a midway runaround. I understand from wrap-up that LO wanted to take that effort and put it into a bunch of ‘mini’ runarounds at the end of the hunt, and maybe that’s ok, but what about all the teams that never finish hunt? Apparently there was a whole thing with ‘the heart of the park’ that none of us ever knew or heard about. A much better solution, IMO, would have been to have a midway runaround where teams are introduced to the crazy machine, try to turn it on, and then find out it doesn’t work because the new zones and old zones are out of sync, or whatever the explanation was, and then they’re charged to come back again once they’ve finished everything.
2) lack of a tangible/clear in-story goal. Yes, I understand we were trying to generate ‘buzz’ (and later ‘wonder’). But that’s so intangible it’s hard to care about. Whereas with holidays you can say, ‘oh, these towns have a tangible problem, we need to fix all the problems’ or with Inside Out you can say ‘oh, we need to find all the core memories’. In this case we’re told, oh, you need to make people care about the park. By solving puzzles. It’s a bit more of a stretch.
2.5) lack of understanding of what the coin is. Some hunts have done this better than others, but it’s cool if you know when you leave kickoff what, exactly, the coin you’re trying to find it.
3) the wedding. I really don’t want to disparage two people’s really special day, so all I’ll say is that for us watching the livestream, it was super unclear that what we were watching *wasn’t* part of the theme (I have no idea if people in Kresge were supposed to be confused by this, but everybody on the livestream was super confused) and was, in fact, and honest-to-goodness ceremony, and that watching a wedding for 30 minutes before a sudden turn into theme parks was extremely jarring.
I also want to throw in a shout-out to Setec for last years unlock structure, and IMO every hunt for the rest of forever should have all puzzles unlocked for all teams by ~noon Sunday.
LikeLike
I liked the theme well enough as a theme, and I enjoyed a lot of what LO did with the theme, but I think it was very much just a theme without enough story – I particularly agree on #2. Especially after we got the midpoint interaction from Penny – we were left going, wait, so we kept the park open, but now we need to open more parks? Because…. they’re there?
LikeLike
Remote solver here, Duck Soup member and former captain. Opinions are my own. For reference, we narrowly missed the mid-mark, as we solved the Big Top meta a little too late. We basically saw none of the outer lands.
N.B.: Opinions are my own, not my team’s.
I was part of the crowd that speculated about the theme for fun, and my personal bet was someone’s guess of Miles Morales & Gwen Stacy, because — this being the 40th Hunt — we could have an “Into the Hunt-verse” theme, possibly in a similar vein as MH30(0) ten years ago. Though someone else later guessed Morticia and Gomez and that seemed to work even better. Then someone told me that this was likely a real wedding between these two actual people, and that seemed to make a lot of sense.
I wasn’t impressed by the theming from kickoff. I appreciated the wedding, though I was expecting a little more fanfare to it, honestly. The fakeout of a fakeout (Cupid’s scene) was amusing I guess.
The actual theme kinda underwhelming at kickoff, particularly the skit. To be fair, however, I’m generally not really into the opening skits anyway. Though, 2018’s opening skit did manage to impress me. But I was a little disappointed that it wasn’t a special meta-theme for the 40th Hunt (a la Hunt-verse or whatever).
As we progressed through the Hunt, though, I enjoyed the theme more and more. I didn’t really think of the puzzles as theme park attractions, but I did feel that the Hunt was progressively telling a story — which it was, particularly through those park history videos, and I felt that that matched the flavor element of us going through and helping to solve the problems and bring the park back to life. And it was so wholesome and beautiful too, neither a murder mystery or heist, nor based on some movie I hadn’t watched.
(It probably helped that I had recently watched Kemono Friends, which has a vaguely similar setting premise.)
I think that the theming was followed very well in the metas and individual puzzles and solutions themselves. Balloon Vendor’s puzzle is especially hilarious and awesome, but I remember most (maybe all?) the puzzles having answers that were in some way related to the puzzle and to the meta thematically. And I really appreciated this — one of my first points of confusion when I started out doing Mystery Hunt was how the themes didn’t tie together, like you could have a puzzle about sweet foods and the solution could be STEREO SET or something. On the other hand, having everything fit together thematically just makes things really elegant and enjoyable.
LikeLike
“MH30(0) ten years ago”…ouch, really? (checks calendar). yep, really…
LikeLike
One of our team members had a brilliant sounding, although probably not actually good, idea for the theme: The wedding was Mario and GLADOS, making a sequel to the 2011 videogames hunt, and each round would be a sequel to one of the 10 previous hunts. The overall theme would be “Hindsight is 2020”.
LikeLike
“Hindsight is 2020” was one of the few dozen themes we considered.
LikeLike
The absence of a mid-hunt ‘first stage endgame’ event/process was a negative for me also; see above discussion of catering to mid-pack teams rather than mostly-lead teams. But I really, really want all of this to be taken by LO in the context of ‘excellent puzzles, fun time, thankyou all!’ :-).
LikeLike
We seriously debated whether to have all puzzles unlocked for all teams at some point during the Hunt. There’s basically two competing factors here. Some solvers, like Daniel here, want the fountain of all the puzzles to choose from. Some solvers can’t handle that amount and will drown. If you’ve got 5 hours left, and suddenly 120 puzzles are dumped on you, and it takes you 3 minutes just to *look* at a puzzle and decide if it seems fun, you don’t even have time to look at all the puzzles in the time you have left. It’s overwhelming and demoralizing. And that is from some of the genuine feedback we got regarding the 2019 Hunt.
We tried to thread the needle on that this year. We put some of the more accessible and entertaining puzzles at the start of the lands when we could, and at 1:30pm Sunday, we unlocked all the lands for all teams. This effectively let us give a curated list of “hey, we think here are some 15 or so fun puzzles for you to try in the last 200 minutes of Hunt.” I think it’s pretty impossible that there would be a team in that position who could realistically solve 15 puzzles in 3.5 hours anyway — and if they did, more puzzles would unlock.
I certainly would like to understand the psychology of why you want all puzzles to unlock for you by noon on Sunday. Consider that, in 28 more hours, all puzzles will be unlocked anyway, because of wrap-up. What does having all puzzles unlock on Sunday noon get you that you can’t get by having 15 or so of the fun puzzles unlock instead?
I suppose maybe the difference is that you want more agency in choosing the puzzles you work on — but you’ve already gone through 46 hours of giving up that agency, trusting us to choose puzzle releases to maximize your fun, so why this sudden mistrust and wanting all that control back? What do you plan to do when you see those 100 new puzzles suddenly show up?
LikeLike
I understand that some people have complained about feeling overwhelmed by having too many puzzles open. And, I don’t disagree that they feel that way, or believe that their feelings are invalid, or believe that their feeling should be ignored. But at the same time, I can’t even begin to understand those feelings? Are some people like the dog in Up who goes ‘Squirrel!?!?!’ that they can’t focus on one puzzle when another opens? Are people frustrated by the feeling that they have so many puzzles that they know they’ll never finish, when otherwise they might actually think that the end of the hunt is right around the corner? (hint: it’s obviously not) None of these really make any sense to me. To be perfectly frank, the Mystery Hunt is such an insanely complicated megalithic behemoth of a puzzle hunt that I’d be sort of concerned if people *weren’t* overwhelmed by it. To me that’s sort of the point.
Personally (and I may well be biased here), I feel that the problems that arise from having too many puzzles open are far worse that the puzzles that arise from having too few puzzles open. If too many puzzles are open (a concept I still can’t understand), the worst thing that happens is that a team feels demoralized. But, they can still progress, they can still solve metas, they are still fundamentally capable of finding approachable puzzles and solving them. If too few puzzles are open, teams feel demoralized, but none of the above is true. Teams can’t progress, people look at the 4 available puzzles and realize they don’t possess the skills to solve the puzzles or can’t find the ‘a-ha’ to even begin to understand what the puzzle is even about. In both cases, teams may be demoralized, but if too few puzzles are open teams can become quite literally paralyzed, whereas if too many are open at least teams can still function.
From my point of view, the only function of limiting access to puzzles is to make sure that top teams don’t parallelize the entire hunt and finish in a half a day, and to disincentivize growing teams past some arbitrary size. As somebody on a non-competitive team (well, we’re 14th, which I consider non-competitive. I’d be curious if LO/other top teams disagree with that assessment), I actually think our enjoyment would be maximized if the entire hunt were opened at noon Friday. Or at least, perhaps all puzzles in the inner worlds, and then after the midway point all puzzles in the outer worlds opened, or something like that.
In terms of all the puzzles opening on Sunday — I set aside around ~72 hours of my life per year to do nothing but solve puzzles, do runarounds, etc. Once I get back to my daily routine of life the appeal of spending 6+ hours on a logic puzzle or whatever is drastically lower. I’d rather be able to spend those 72 hours doing puzzles that I enjoy, rather than staring at the 4 open puzzles that have been stuck on extraction for over 12 hours, which is not something I find particularly fun.
To be clear, the puzzle unlock structure I favor the most is the 2019 structure, which, as I understand it is something like ‘after N minutes, another puzzle unlocks, unless that puzzle was unlocked by traditional means already’. This has the advantage of being less likely to overwhelm teams, because puzzles aren’t coming all at once, but if teams are stuck then at least a new puzzle will come in under an hour or so. It shouldn’t affect the competitive nature of the hunt because top teams will probably be ahead of that curve anyway, and if they aren’t, this unlock structure is probably helpful to ensure the hunt doesn’t drag into Monday.
LikeLike
“And, I don’t disagree that they feel that way, or believe that their feelings are invalid, or believe that their feeling should be ignored. But at the same time, I can’t even begin to understand those feelings?” The paradox of choice is a well-studied phenomenon. It doesn’t hold true in every situation, but it does in many. When I was a novice and there were lots of puzzles open, I would open a puzzle, work on it, get stuck, go to another puzzle, get stuck, etc. etc. every five minutes and never get anywhere near to solving anything. With Mystery Hunt puzzles, you never know whether a given puzzle is broken or takes 50 hours or not, so it encourages moving on the first time you encounter something weird. It was very frustrating and not something we wanted to inflict on all teams.
At some point, LO discussed a system where solvers would rate a puzzle right after solving, so later solvers could be directed to puzzles that were easier or more fun to solve. I think we discarded that system for being too unwieldy, in favor of focusing on the core hunt. In the future, it would be nifty to be able to direct later solvers to the more fun puzzles for when there are 100 puzzles open and there’s too much choice.
LikeLike
I agree with a lot of what Daniel has to say here. I tend to think that in many cases, teams know what’s best for themselves in terms of enjoyment (as long as it doesn’t affect standings) and are perhaps better off being able to see more stuff before Hunt ends, even if it means that they might not focus on what the construction team thinks they should care about that that stage.
I am also the type who will probably not attempt to solve many untouched puzzles once Hunt is over; if I did that I would probably want to solve all of them, and given I don’t have the time for that, I feel my time is better spent reading solutions and marveling at the constructions. (During Hunt, I know I won’t see most of the puzzles firsthand, but I know my team is solving them, and when the team accomplishes something, I feel like a part of that, and I’m content to just see what happened afterward.)
One place I disagree is in the appeal of having the entire Hunt open at once. Speaking as someone whose team will usually solve most of the puzzles by the end, if everything was open at once, we probably would do the equivalent of hitting the dessert cart first, and annihilate most of the puzzles we think we’d have the most fun with. Which would leave nothing but vegetables for the remainder of Hunt. The staggered round release innovated in 1998 (and then staggered individual puzzle release a few years later) is one of the most crucial innovations of Hunt to me… I like the feeling of earning new stuff throughout the weekend, as opposed to just being able to say, “Yeah, now we’ve solved 39 out of 170 puzzles!”
In short, if I’m renting a video game for the weekend (or if I were back when that were a thing in my life) I would not want to start from a superpowered save game where all the levels are accessible… I want to open up content as intended. But if I have to return the game in six hours and know I’m not going to play it again, I wouldn’t mind being granted access to the spiffy levels near the end.
LikeLiked by 1 person
Heh, I’ve actually been wondering lately what the history of the online hunt experience was. I started in 2003 and my memory is that puzzles were already fully online by then, mostly rolled out a round at a time? And I think answer call-ins didn’t involve typing the answer into a webapp at the point (which allowed for the round where the answers were objects rather than words).
LikeLike
I believe that in 1999 (Enigmatology) puzzles were fully on paper, and in 2000 (Oz) and all future Hunts they were accessed on a website. I don’t remember whether the 2000 website actually tracked your progress by team; the only relevant permissions would have been to view new round pages, and I forget whether we handled that with a team account, or simply password protected the round pages. (The metapuzzle answers that year were username/password combinations, but they wouldn’t have been very secure.)
LikeLike
Daniel, you say you can’t begin to understand the psychology. I can’t tell if you *want* to understand it but are failing to do so, or if you don’t actually want to understand it. If the latter, ignore the rest of this comment.
Let me try several different explanations and analogies and see if any of them work for you. (Keep in mind that I myself only have a bit of this and am more like you as far as unlocking puzzles are concerned, so my understanding won’t be perfect.)
“…the worst thing that happens is that a team feels demoralized. But, they can still progress…” I think this is probably the crux of your misunderstanding here. The sort of demoralization we’re talking about is one when you can’t progress because you *no longer have the desire to progress*. The team looks at all the puzzles they have available, and decides there’s no point in continuing. Maybe they’ll have more fun going to dinner or playing a board game or something. And yes, this is psychological. It’s like the the old trick of climbing a mountain — if you look at how far up the peak is, you don’t want to continue, but if you look just at what’s in front of you and put one foot in front of the other, you’ll get a lot farther. Rationally, they know that the end of the hunt is not right around the corner, but having the fantasy that it is — that’s a source of motivation.
The question of whether 14th-place is competitive is a good one. I think it’s hard to answer because “competitive” is not that well-defined. Certainly, the 14th-place team is not going to win the event, so if you define “competitive” as “in contention to win”, then no, 14th-place is not competitive. But let’s say the 14th-place team and the 13th-place team have a rivalry going on. They’re going to be pretty grumpy if the organizers decided to disturb the pristine-ness of that competition by offering different hints, unlocks or whatnot. To make a football analogy, the Army Black Knights and the Navy Midshipmen might not be good enough to get an NCAA rank every year, but they take their game against each other pretty darned seriously.
“I’d rather be able to spend those 72 hours doing puzzles that I enjoy, rather than staring at the 4 open puzzles that have been stuck on extraction for over 12 hours, which is not something I find particularly fun.” Agreed, but not really the intended comparison here. We’re talking more like 20 open puzzles, of which the team hasn’t seen 12 of them, versus having 50 of them all open at once.
LikeLike
Oh boo boo, I didnt get to finish Hunt this year. Cry me a river Dan, and join the rest of us who barely get midway each year.
LikeLike
Don’t be a shithead.
LikeLiked by 5 people
For the viewers at home, my only previous interaction with Giovanni was seeing him post to a puzzlers’ group on Facebook on January 14, “Hey, who wants to see a tire fire happen in Boston next weekend?” I thought this was pretty hostile and classless, so I responded asking what he had contributed to the puzzle community (legitimately not knowing the answer since I don’t know who he is, but suspecting it was not much). I guess this is his mode of contribution?
In any case, I could delete his comment since it adds nothing productive to the conversation, but if this is how he wants people to see him, far be it from me to get in the way.
LikeLike
Could you point me to this Facebook group? I’d be interested in joining.
LikeLike
More crosswords than anything else, but https://www.facebook.com/groups/114491658630189/
LikeLike
Giovanni, given that your contributions to Mystery Hunt discourse seem to consist exclusively of shooting your mouth off, maybe give it a rest for a while.
LikeLiked by 2 people
Putting in my two cents as a co-captain of a new and inexperienced team, with several solvers who had been on hunt-completing teams in past years (me included). I can’t speak for the rest of my team, but I was really fine with not getting callbacks from HQ. I also felt like we had plenty of interaction with HQ anyways (3 character visits, all the events, a visit from HQ for in-person hint help on a meta, another check in that we were having fun, and the pretty good responsiveness to hints on Saturday) — even though we only solved the first 2 metas and by Saturday lands unlocked for us. For people concerned newb teams would feel unsupported with this new structure — that was absolutely not the case for my team. And the hint system allowed us to make real progress on puzzles we would have given up on extraction for. Great job, Left Out.
LikeLiked by 1 person
This was only my second year on-site after years of remote solving, and I have to say, this was much better with the automated checker – I remember being a remote solver needing to wait a long while to get confirmation, and just really sitting there no doing much. It was compounded with the fact that the dude on phone duty can’t really rest, in addition to needing to worry about the events and on-site stuff that the remote corpus needed. I think this would be especially true for teams with a small on-site presence, since I remember a few years ago, our team’s solving basically needed to be put on hold because we can’t keep letting our one on-site person stay awake just for phone calls. The automated system definitely is much better, since it freed up both the HQ and on-site solvers.
I think this year was extremely high quality in terms of the art and theme – and due to the events and other interactions (workshop reveal, penny’s visit, and misc ones like the fortune teller and balloon vendor) it gave a very fun theme park feeling. Puzzles like Weakest Carouselink and Concierge Services could have just been text on a page and the latter extraction stuff would have been similar – but it would have made for a very boring and forgettable experience. By putting so much effort into enjoyable interactions, it really made this hunt into a memorable event. I didn’t really think “buzz” and “wonder” worked as well, but I feel like Left Out really nailed the theme park feeling (esp. during Midway), and the art is gorgeous. Kudos to the folks that played the mascots – Kobo and Luma at breakfast were very funny!
The hint system was great, but I imagine HQ was very flooded on Sunday. I think it was a bit strange some puzzles never got hints the entire hunt (or at least, unlocked very late). Probably due to limiting 1 hint per team it made it much less of a deluge than opening up hints to every puzzle (and requires teams to be tactical about choosing what to ask about). The hints were almost always very useful, so thank you again for this! (I think we were one of the only teams to frontsolve Arts and Witchcrafts after the hints pointed us in how to extract)
One small gripe our team had was that we expected something in the mid-hunt, after we finish the main park – so when the outer lands unlocked (thank you for Penny for still visiting us a few hours later, even though by that time we already knew Galactic won!), the majority of our team decided to ignore it so we may at least finish the center. Our team regularly finishes middle of the pack (around 20th-40th range) and we typically try to aim for at least getting through half the hunt since there’s no way with our current team size we would come close to finish. When we learned that there really isn’t any mid-hunt interaction/runaround, I think it took a lot of excitement out or our team – and we only looked at the outerlands after the hunt even though we have had most of Sunday to look at it, because we focused fire on the middle so hard. It’s a big shame because I think the outer layers were very cool experimental stuff, but we only really looked at it post-hunt. This is partially our own fault for an incorrect assumption, but it was still quite a disappointment in an otherwise extremely enjoyable hunt.
I think the production value into the puzzles were very high this year – this is definitely one of my favorite years in my relatively short hunt career, if not the best!
LikeLike
> since I remember a few years ago, our team’s solving basically needed to be put on hold because we can’t keep letting our one on-site person stay awake just for phone calls.
There was at least one year when solvers were allowed to basically input their choice of phone number, and at least another year when solvers could pick which phone number to use out of a list that could be like seven items long. This would be the ideal accommodation if any future writing teams intend to reintroduce the phone-based answer verification system — in other words, allowing solvers to choose what number gets the call, so they can pick either a designated team number or even their own.
LikeLike
The existence of a mid-game success event (above and beyond a character interaction) was definitely one of the things on our list. And I wish we had done it. But whenever we talked about adding something like this, we went with the conservative approach. We weren’t really sure how overextended we were going to be during hunt and we didn’t want to commit to something that might require even more staffing or maintenance. And from a story perspective, we didn’t want to have a huge plot twist/switching of gears because we wanted the whole hunt to be about the full park.
But there are plenty of ways to have that event without a lot of the costs, so yeah, my apologies for that. It’s good to hear that teams value mid-game interactions and victories, and we’ll be sure to address something like that if we’re ever stupid enough to win again! Thanks for the feedback.
LikeLike
Yeah, C and I honestly thought that the character visit would scratch that itch for a mid-game success event. In the light of what happened the week before Hunt, I can’t help but think that it was just as well that we didn’t have a huge runaround that many teams would likely have encountered in the middle of the night.
LikeLiked by 1 person
Every team makes decisions about how they want to use their time to interact with teams, and I think any answer other than “not at all” is valid. Setec in 2017 and 2019 knew we wanted to make the midgame highly interactive, but that meant running it for many teams was going to be a large time suck, and that likely affected how much interaction we could do in other contexts.
In contrast, this year I think the midgame, being just a skit, was a little underwhelming (especially if you hadn’t been following the plot videos closely… more on that in a future post), but on the other hand, there were a lot more involved individual interactions (karaoke and Weakest Link both leap to mind since I participated in them, and I know there were others), and I think teams really appreciated those.
LikeLiked by 1 person
Just for clarity: there was a specific character visit associated with making it about halfway through? That was based on the solution of… the six rounds associated with the Loonie Toonies? (And also the Balloon Vendor?)
My team had a goal of “get to the midgame” (I like the terminology, Dan!), and we were a bit disappointed when we thought at the end that there wasn’t actually a midgame at all (though we only got 4 of them anyway). But it sounds like there actually was one (even if it wasn’t a “meta meta” puzzle), which actually does feel good! I think there’s something to be said for having the “midgame” be somewhat clearly defined — eg, we still had a guess that there was some sort of midgame but we weren’t sure if Balloon Vendor was going to be required for it or not.
LikeLike
When you got 62? buzz, Penny and a Loonie Toonie would come and act out a skit opening the Outer Lands. You would then go from collecting Buzz to collecting Wonder. However these visits often got delayed if this transition happened overnight and the team either didn’t respond to scheduling, was too far away, or didn’t want a visit.
LikeLike
Ah, OK. Was this number achievable basically only by solving a certain number of metas, or was it just basically the next round unlock after Big Top Carnival?
LikeLike
Metas were worth 1 buzz just like any other puzzle. So yes, just another round. We didn’t want this to be a bottle neck but weren’t considering the anticipation factor.
LikeLike
For me the issue with midgame was that it wasn’t a very “puzzle-y” accomplishment. You finish a bunch of rounds, each of which is a genuine puzzle accomplishment, but the act of finishing the inner worlds didn’t involve anything interesting from a puzzle standpoint. More plot or a bigger interaction wouldn’t have really changed that for me personally. By contrast finishing each individual outer round was a large interesting puzzle accomplishment. This is why I’m a little disappointed that we finished inner worlds, but got stuck an hour or two short of finishing the first outer world, and barely saw what was happening in the second outer world (I’m not sure we even opened any trainers?). I think it’s a very valid choice not to make midgame a big deal, but I think that decision makes a case for having a shorter inner park. That said, this is all pretty minor in the grand scheme of things.
LikeLike
I liked this Hunt. It’s not 2015, but it’s probably the second best I’ve played (and I started in 2012)? A few points:
1. The individual puzzles were excellent, rather like in 2015 and 2017. There are a few puzzle types that are so common, like crosswords and logic puzzles, that every year people have to come up with new twists on them. The things I saw this year were good twists, like the Obscure Movie House cryptic.
2. The unlock structure was pretty good, albeit not amazing; consensus on my team is that 2019 should set the standard for that. But non-remote solvers complained about the lack of a good story, which I guess relates to Noah’s complaint that finishing the inner areas of the Hunt didn’t feel like a good midgame runaround.
3. I didn’t personally mind the lack of phone calls for answers; I don’t think the on-site people minded, either. I still think back to 2013, when the Hunt took so long that I had to be answering the phone on Sunday night from Vancouver, much to AT&T’s delight. That said, if answers are done via the website, why not let people keep trying out answers after the Coin was found even after 5 pm on Sunday? (Or was it 7? I don’t remember. Either way.)
LikeLike
I was a dissenting voice on Left Out who felt that the answer checking should be kept open. But the arguments for closing it are quite reasonable, the main one being that having the submissions close is a clear signal for “end of Hunt”. You need such a signal, otherwise there will be die-hards who will probably continue to solve until July, and we’d really like to be able to close out HQ some time before MIT gets angry at us for squatting. Having a way to end Hunt gives us time to work on the wrap-up videos, migrate the server, all sorts of things that we can’t do if teams are still submitting answers. There are other ways to signal “end of Hunt” — for many years, it was “The Coin has been found!” — but closing answer checking when the coin was found would be even meaner. We’d also like to put up solutions soon after “end of Hunt”, because we know that’s something a lot of solvers want to see, and it would be really strange to have teams be able to make progress after solutions are public.
Although, what we should have done, in retrospect, is to have a static answer-checker on the site released at the same time as the solutions. But we didn’t quite have the personpower to make that happen.
I was able to push enough to at least make it so that even after “end of Hunt”, there was still a venue for hint-requesting, via the hints email address.
LikeLike
Yeah, that makes sense. I find it strange that a static answer-checker would require so much work, though – could it not just be the same mechanism used for answer-checking during the Hunt itself, just without the rate limitation? Then again, I suck at coding, so maybe I don’t get that there’s a real issue with coding difficulty/server resources/babysitting the site/whatever?
LikeLike
What Dan and Matt say are all true. Getting the teams to clean their room was something that was mentioned to me as a pro-“turn off” argument, I just forgot about it. Since the organizers are responsible for cleaning up anything the teams don’t, that’s actually ends up saving more work than you’d expect.
Our static answer checker was actually written two days after the Hunt by Eric Prestemon, who notably did NOT work on writing the answer checker on the server. In addition to having to change answer responses like “Correct! Please come to room XX-XXX” or “Correct! Please send us a video of the answer in flight”, he also had to get the emoji checking working and have special messages for the puzzles with multiple answers. None of which is hard, but it’s still work.
LikeLike
LO can answer more authoritatively than I can, but if I remember correctly, some answer submissions prompted a “We will contact you about this soon” message, and if the system was going to keep running on a team-by-team basis, they’d probably want to disable those so as not to mislead teams, which would require an alternate set of code. And I don’t know if this was the case this year, but in Setec hunts there were some answer submissions that prompted in-person interactions that then prompted website behavior once someone physically checked a box in HQ; if anything was built this way, even with the answer-checker active, teams might still hit an impassible bottleneck.
Honestly, I also figured that one reason to stop verifying answers was to make sure people with classroom HQs actually stopped solving puzzles early enough to clean their rooms on time!
LikeLike
Oh, and I suspect the answer to, “Why is it hard to make a static answer checker compared to the real Hunt checker,” is, “It’s easier, but it’s different, so someone still has to sit down and do it, and we were understandably kinda busy.”
LikeLiked by 1 person
Yep, I set up the static answer checkers in 2017 and 2019, and “easy but different” sums it up pretty well. The 2017 one wasn’t added until many days after Hunt, but the 2019 one was activated as soon as we stopped making calls… I actually had not anticipated that it would allow folks who wanted to keep solving right then and there to continue to do so, but I was very happy to learn that many teams took advantage of it immediately! Definitely a feature I’d encourage for future constructing teams, if they have the time/resources (and as long as we’re not too concerned about the potential room cleanup consequences).
LikeLike
Pingback: Two hundred puzzles, fifty weeks later | MIT Admissions