When to Ignore Playtesters

Playtesting is a huge part of the design process. Playtest feedback is vital to design progress. So how do you know when your playtesters are wrong?

First, let's address the easiest one. If you only hear one complaint about something out of all your playtests and you disagree with that feedback, you should ignore the it. (Ignore in this case means write it down but don't make changes. Always record feedback, even 'bad' feedback.) Outlier feedback can send you down rabbit trails trying to make everyone happy. Designing to make everyone happy usually leads to a less focused game because you have likely strayed from your design vision. If you disagree with the feedback, wait for more people to bring up the issue. The way you know something is an issue in your design is if more than one person mentions it. 

However, there are times when a lot of people could give the same feed back and you should not implement it. Usually, this happens when you playtest outside of your target audience. You should playtest outside of your target audience, at least some. Different types of gamers see problems in different ways and can give fresh perspectives on your game. However, most playtesters want your game to be something they would want to play. So they give feedback in an attempt to shift the game to their preferred style. Go too far in that direction and you could end up designing a different game. 

This can manifest either mechanically or thematically. Ask your playtesters what types of games they typically play for fun (either before playing or before feedback), especially when playtesting with designers. Game designers give very 'authoritative' feedback about the number of cards your game should have, how heavy the game is, etc. Find out what weight of game they typically design, and let them know what your target weight is. Your game's weight is dictated by your target audience, so be careful about changes that add or remove complexity. 

After playtesting, ask your players if the theme appealed to them or not. Be wary of players who don't like the genre of theme but give extensive feedback about how to make the theme better. They aren't your target audience. Generally speaking, if they wouldn't play a published game with your theme, don't listen to their feedback on theme. Ask them for feedback on mechanics, instead. However, if you can't find anyone who fits the target market for your theme, that's a bigger problem. Themes do need to be playtested, so make a point of playtesting with people who are interested in your theme. 

Another thing to watch out for when playtesting with designers is the 'tried and true solution.' Playtesting groups typically develop a certain amount of groupthink which is particularly pronounced in a group of designers. Observe other playtests and listen to see if you hear the same feedback over and over. Sometimes, the feedback is necessary (yes, you do need a player aid in your prototype). Sometimes, the feedback is a design shortcut that leads to less creativity in a design. Implementing those design shortcuts could lead to a number of games produced by that group that all have a similar feel. (Good news if you're a design studio; bad news if you're a group of aspiring independent designers.) Instead, listen to what the problem is that the shortcuts are attempting to address and try to solve it in a different way.  Crowdsourcing your design decisions to other designers won't produce a very interesting game. Also, make sure to playtest with a number of different groups, especially if you are looking for fresh ideas. By playtesting with multiple groups you will, at least, have multiple design shortcuts to choose from. More likely, you will have a better understanding of what the issue is and be more equipped to address it by combining aspects of different feedback into a solution that is uniquely yours. 

Casual players, gamers, and designers can all be wrong. What do you do if your publisher is wrong? 

In all likelihood, the publisher isn't wrong. If they are asking for changes in exchange for a contract (aka money), then they want a game that they can sell. I also want a game that can sell. If, like me, you want someone else to publish your game, then you can't take hard lines with a publisher. Honestly, my best revisions to my games have come after publisher feedback. Interested publishers make the best playtesters because they help shape your game into a product. However, sometimes the publisher will present an idea that doesn't quite fit with your design vision. The best thing to do then is to offer an alternative or a revised version that keeps the direction the publisher was headed in but is more in line with your vision. You know your game better than anyone else. Where I have done this most is flavor text, but I'm a writer. In areas like game balance, present your case but let the publisher decide. Worst case scenario, the game flops and you get the rights back to try again. (Always make sure reversion clauses are detailed in a contract.)

At the end of the day, all playtester feedback has to be filtered through your design vision. Playtesting makes a game better, but not all feedback is created equal.