Playtesting The Outer Worlds

Lessons from Past QA Failures

Tim opens by contrasting The Outer Worlds with two previous disasters:

• Temple of Elemental Evil — External QA decided they didn't like playing bards, so they simply never tested them. The game shipped with a major crash bug affecting bard players. Tim had played an all-bard group himself but never hit that specific bug.
• Vampire: Bloodlines — Almost identically, external QA ignored the test plan and refused to play Nosferatu because the clan was "ugly." The game shipped with an unavoidable crash bug for Nosferatu players — one that would have been found in a single playthrough.

Both cases shared the same root cause: external QA self-selecting what to test rather than following the plan.

What Made Outer Worlds QA Different

The Outer Worlds had an internal QA group, which Tim credits as the critical difference. Key practices:

• Weekly one-on-ones with QA lead Taylor Swope — Tim would flag new features needing testing; Taylor would report what felt wrong, where bugs were clustering, and what unfixed bugs were blocking further testing.
• Casual drop-ins — Tim would walk down the hall to watch QA play and chat about what they were finding.
• Smart bug deduplication — When QA saw a recurring bug, they filed new instances under the original report. This meant fixing one bug instantly resolved all 90+ duplicates, unlike external QA which would flood the database with separate reports.

Taylor Swope's Randomized Build Generator

The standout innovation was a program Taylor Swope built that randomly assigned QA testers their character builds. It specified:

• Which attributes to set (including low ones)
• Which skills to tag and invest in
• Whether to play with controller or keyboard
• Which platform to test on
• Specific story choices to make

This forced testers out of their comfort zones and into builds they'd never voluntarily play — exactly where the undiscovered bugs lived. If bugs existed primarily in melee combat or in a perk only melee players would pick, you'd never find them if everyone gravitated toward ranged builds.

Tim Cain's 16 Playthroughs

Tim played through The Outer Worlds start-to-finish 16 times (with additional incomplete runs during earlier development). His named characters reveal the breadth of testing:

1. Sneaky McKill — Ranged + Stealth sniper. Tim's classic RPG build and personal favorite.
2. What's Up Dude — Generalist. Spread skill points evenly across all skills, resulting in lots of options but slow specialization.
3. Snuggler — Pure dialogue character (Dialogue + Ranged as backup for monsters). Fun but Tim preferred the sniper.
4. Butch McSwipey — Melee + Low Intelligence. First test of the "dumb dialogue" options. First controller-only playthrough. Found a major difficulty spike on Devil's Peak (Monarch) — a vertical map where ranged enemies on platforms destroyed melee-only players. This got flagged and redesigned.
5. Doctor Leader — Low Dexterity, Tech + Leader. Terrible personal combat offset by strong companion support. Demonstrated how companion builds could carry weak fighters.
6. Rango McLeader — Ranged + Leader. First Supernova difficulty playthrough (hunger, thirst, sleep, restricted fast travel).
7. Eric the Milk — Named after producer Eric DeMilt. Kill-everything playthrough — often killed quest NPCs before talking to them. Since The Outer Worlds didn't tag NPCs as "essential," this uncovered interesting bugs.
8. Sneaker Speaker — Stealth + Dialogue. The one and only Story Mode test. Tim advocates for easy modes: "It shouldn't be called baby mode... it should just be story mode."
9. Stealth Boss — Stealth + Leader on Hard difficulty. Avoided combat when possible, let companions handle the rest.
10. Dr. Guns — Tech + Ranged, very low Strength, controller-only. First serious playthrough incorporating science weapons (which scale off Science, not weapon skill). "A little goofy but super fun."
11. Doc Block — Defense + Tech, low Charm and low Temperament. Solved problems through science skills, not dialogue. Harder than normal.
12. Teddy — Melee + Dialogue on Hard, controller-only, low Perception. Named after Teddy Roosevelt: "Talk softly but carry a big stick." More fun than Butch McSwipey since melee bugs had been fixed by this point.
13. Heavy Leader — Ranged (heavy weapons only) + Leader. Sold every non-heavy weapon. Companions handled distance threats.
14. Stealthy Leader — First playthrough using Taylor Swope's randomized program. It prescribed: Stealth + Leader, below-average Dexterity, medium difficulty, side with the Board, complete companion quests, choose Emerald Vale outcome, send power to deserters, put MSI in charge of Monarch, and take every drug immediately upon finding it. Constant withdrawal made medium difficulty feel like Supernova.
15. General Generalist — A character resurrected from Tim's original Fallout playthrough 20 years prior. Spread skill points randomly with Ranged + Stealth tags. "Fun and weird to play the same character 20 years later."
16. Dr. Stealth — Tech + Stealth on Hard. Avoided combat, used science weapons. Second-favorite character after Sneaky McKill.

Three Takeaways

Tim distills his playtesting philosophy into three rules:

1. You have to play the game you're making — not just watch others play it, not just review bug reports. Actually play it, repeatedly, in different ways.
2. A good, thoughtful QA group is invaluable — internal QA with open communication beats external QA that ignores your test plan.
3. Seek out the bugs — accept that you'll find them, don't try to avoid them. "I want this thing to break on me so it doesn't break on you."

Source

Playtesting The Outer Worlds — Tim Cain (YouTube)