Collecting Feedback

1. Pre-Ship Feedback Sources

1.1. Team and QA Feedback

The first source of feedback comes from the development team itself. Team members play the game constantly and can flag issues early. This is both a strength and a weakness — they're deeply invested and close to the game, which means they may be too familiar to notice problems, or they may dismiss issues that would bother a fresh player.

QA feedback is especially valuable. No one plays the game as frequently or as thoroughly as QA testers — they're using the UIs all day, fighting the AI all day, and they surface issues that others miss. Tim emphasizes: listen to them.

Publisher feedback is another part of the pre-ship team equation. Tim found some publisher feedback invaluable — they'd point out things the team wasn't seeing, or suggest small changes that could broaden the game's appeal. However, publisher feedback frequently leans toward "cut this, not add this" and tends to be more CYA-oriented (e.g., flagging content concerns rather than suggesting creative additions).

1.2. Focus Testing

Tim has a love-hate relationship with focus testing. The upside: you get raw, honest reactions from people encountering the game fresh. The downside: you sometimes get bizarre, unactionable feedback — like someone who only plays first-person shooters rating your RPG poorly.

The key is not to feel beholden to every focus test comment. Tim recommends starting focus testing as soon as you hit vertical slice and doing it again later in development. Focus testers give you valuable vibes — how the game is presenting itself to fresh eyes — even if their specific suggestions aren't always useful.

1.3. Mock Reviews

Companies or individuals you can contract to play your game and write a pretend review as if they worked for a game review site, often including a score. They'll note crashes, story problems, progression blockers, and highlights. Tim considers mock reviews more actionable than focus testing — mock reviewers provide structured critique similar to what real reviewers will write post-launch. He recommends listening to both, but for different reasons: mock reviewers for actionable specifics, focus testers for overall feel and presentation.

2. Post-Ship Feedback Sources

2.1. Reviews

After shipping, you'll receive reviews from journalists, bloggers, influencers, and vloggers. Tim highlights what his Outer Worlds UI designer Chris Stewart did: he collected every review into a spreadsheet, analyzing what features were mentioned, how frequently, and whether the sentiment was positive or negative. This produced powerful data — for example, "90% of reviewers mentioned this feature and liked it" versus "30% mentioned this other feature and all disliked it." The 70% who didn't mention the disliked feature tells you something too.

Tim recommends checking reviewers' past work to understand their biases. If a reviewer hates turn-based games and you made a turn-based game, weight their criticism accordingly — but pay attention to reviewers who do like your genre.

2.2. Online Comments

Online comments are harder to parse because the signal-to-noise ratio is low. There's massive volume and massive disagreement. Tim notes that on almost any feature, you'll find people who love it and people who hate it. Increasingly, online comments skew negative — people list what they didn't like and rarely mention what they enjoyed. Developers must account for this negativity bias when interpreting community feedback.

Despite the noise, the team still tracks online comments the same way: cataloging features mentioned, sentiment, and frequency.

3. What To Do With Feedback

3.1. Assess Actionability

First, determine if feedback is actionable. "The UI should work this way" is actionable — you can test the change immediately. "The AI is dumb" from 90% of players is clearly a problem even if non-specific — you need to investigate. Non-actionable feedback goes into a pile where you track how many people mention the same issue.

3.2. Consider Your Audience

Some companies go purely by popularity — if 90% don't like something, it goes. Others filter by target demographic. If you're making a dating sim and non-dating-sim players don't like it, you can probably ignore that. Know your audience and weight feedback from your target players more heavily.

3.3. Evaluate Consistency and Polarization

If a feature is consistently loved or hated, the path is clear. But when feedback approaches 50/50, you've hit something polarizing. This isn't necessarily bad — some developers want their game to provoke discussion and leave interpretation to the player. Others want to avoid polarization entirely and may remove the contentious feature. Your creative vision should guide the decision.

3.4. Do a Cost Analysis

For every identified problem, weigh the cost to fix against the cost of not fixing. A pernicious problem requiring major rework may be expensive to fix, but if it's causing people to stop buying your game, leaving it unfixed is expensive too. This analysis depends on your game, budget, and publisher relationship.

3.5. Decide When and How to Act

Even after deciding something needs to change, you have options for when:

• Patch the existing game — relatively cheap now, though publishers may control whether you can patch
• Fix it in DLC — address gaps or improve systems in downloadable content
• Save it for the sequel — if the change is too dramatic for the current game, redesign the system in the next one
• Apply it to future games — sometimes feedback is best absorbed as a lesson for entirely different projects rather than retrofitted into the current one

4. References

• Tim Cain. YouTube video. https://www.youtube.com/watch?v=AHiwa1nruUU