Opinion, Socio-theological babble, Theorycraft, WoW

What’s better than math?

Boy, she looks pretty angry! Maybe someone noticed she's using an MP5 weapon and shield for DPS?

It’s Thursday and I feel a hankerin’ to a ranterin’.  There’s a weird thing that happens in a lot of threads on the MMO-Champion and Wowhead forums (I don’t read the official forums much) where the argument goes like this:

OP: What’s best, A or B?

Poster 1: A is best.  Here is some evidence, some links to show how this evidence is widely attested and subjected to peer review, and some advice based on this evidence.

Poster 2: Nah, B is best.  You can’t just reduce everything to maths and theorycraft.

These sorts of conversations just boggle my mind.  The really ironic thing is that I’m one of those irritatingly post-modern types who believes in principles like the value of an explicitly subjective hermeneutic because of the inherent limitations of human epistemology.  Or to use other words, we can only know what we can experience and we can only generate that knowledge through interpretation of that experience – so an approach to praxis that acknowledges and embraces these limitations is to be desired above an approach which pretends it knows everything.

So I believe pure “objectivity” is by definition unobtainable by humans, or indeed by anything that we can conceive of within the boundaries of our understanding of the Universe, because in nature everything is related to everything else and experiences the world in the context of these interrelations.

Anyway, I’m getting sidetracked.

(Sorry to the guy who inspired this post, this is not a rebuttal or personal attack on you, or even on Poster B type responses in general, it’s just an interesting topic that I wanted to think out loud about and this most recent thread is what brought the topic back to the front of my mind.)

Best DPS

If somebody asks me for advice about how to play their class in WoW, I generally assume they want to know what’s going to give them the best results in a given role.  Defining that word “best” is a bit tricky, but for damage-dealers I generally assume “best” means “meeting all the requirements of the encounter vis-a-vis survival, resources, movement, utility use and placement et cetera, while outputting the highest amount of damage possible as fast as possible”.  That is, doing maxx DPS.

WoW’s DPS game works on numbers – it’s basically a mathematical system where every spell and ability is assigned a range of values that govern behaviour.  So it’s pretty easy to model parts of the game either in formulation (spreadsheets and stuff) or by simulation.  These models are based on using in-game testing to discover the mathematical rules which I just talked about, writing the same rules into the tool, and then establishing an environment where you can generate meaningful comparisons.  When “scientists” do experiments they have to carefully construct the experiment so that:

  • As much as possible, only the factor under investigation will affect results and
  • There is something to compare the results of the factor to

In pharmaceutical trials this usually means that you’ll give one group of people the actual drug you want to test and another a placebo.  You then compare the health of the people who took the experimental drug with the health of the people who took the placebo, and then you use fancy statistical maths to work out whether any differences are “significant”.

To translate this into WoW, let’s say we want to test a glyph in 25-man raids.  What we’d have to do is assemble a large number of raids with as similar a composition as possible and with similar player skills.  We then split up these raids into two groups with equal representation of raid balance, gear level and player skill and assign a very strict list of strategies and spell usage to each raid which they are not permitted to deviate from.  Then the shamans in one group of raids all get the glyph we’re testing and the shamans in the other group use a control glyph.  We then send each group of raid groups off to do the same boss fight over and over and over again – because we need enough samples to minimize the effect of WoW’s RNG.  At the end we evaluate the logs and determine which shamans did higher DPS.  We then adjust for gear level and raid balance if necessary and work out if there’s a statistically significant difference between the two glyph groups.  And, because glyphs generally contribute between 1-2% of our DPS, we have a very narrow margin to evaluate.

So we’d need a very large sample size with very strict controls and very accurate measuring, and very careful analysis of the results.  At the end we could reach some tentative experimental conclusions which would be subject to verification or challenge by future experiments.

Maybe it’s just me, but I think that seems just a little impractical.

So what are the alternatives?

One: Models

Models are great because they allow us to create artificially controlled conditions and, in the case of simulators, quickly generate vast numbers of samples (formulators don’t need vast sample sizes because every “test” using the same variables is identical).  The problem with models it that they aren’t “real”, they’re just models.  People can make mistakes while designing them or while using them, or there can just be inherent limitations in what they can do.

But, the people who make and use models know this and will deliberately try to minimize these flaws – their models will be subjected to “peer review” by other members of the community, results are reality checked against log data and even anecdotal evidence, and systematic limitations like the difficulty of modelling movement in a spreadsheet can be worked around by “best guess”-type approximations that fit within the model.

In other words, I think that our gameplay models are indeed explicitly and confessedly “subjective”, but they still take into consideration a vast matrix of factors and thus produce very useful results – probably at least as useful as the big impossible playtest trials I outlined above.

Two: Well I feel that…

The other alternative is just to go with what you feel is best.  I’m not comfortable using the word “feel” because I don’t want to imply an emotional judgement (or that emotions or intuition is bad!), but if I use a word like “think” it implies a level of, y’know, thought.  Let’s say that you do Deathbringer Saurfang one night using Glyph of Lava and you do 12k DPS.  You feel pretty good about it but are a bit miffed because that sodding arcane mage who won DFO last week beat you by 200 DPS.  Next week you change to Glyph of Flame Shock and are thrilled to discover you did 12.5k DPS while that laughably incompetent arcane mage only did 11k.  Perhaps you take a look at your World of Logs report and see that 10% of your damage that fight came from Flame Shock.  “10% is a lot”, you might think, “and I did 500 more DPS tonight so that glyph is clearly pretty great!”

The trouble is next week you might swap back to Glyph of Lava and do 13k.  What do you conclude then?  Alternatively what if you get beaten on DPS by a guy using Glyph of Chain Lightning and rocking 1k less gearscore than you?

Don’t you take my math away

I know this is a really convoluted and unnecessary way to get to this point, but really – if you take away the carefully constructed and rigorously tested models that we use to reach our conclusions about best practice in WoW, what is left to rely on?  If someone asks for advice, how do you answer if you just ignore the maths?  “Go with whatever you find fun” maybe, but most people find it fun to do the highest DPS they can.  How is it more or less fun to use, say, Glyph of Flame Shock over Glyph of Lava?

When I answer someone who’s looking for advice, I base my answer on the evidence available to me, and I factor in so-called subjective factors like player comfort, raiding situations (e.g. what you can rely on in pug raiding is often very different to hardcore progression raiding) and even that nebulous and ephemeral idea of “fun”.  But ultimately it’s based on interpretation of hard numbers because that seems the most reliable and accurate source of information.

If someone then replies that you can’t just base everything on maths and theorycraft, well, what can you base it on?  What is left?  Anecdotal evidence will give you as many different answers as there are anecdotes and is still a manifest example of using our brains to evaluate evidence.  What’s different between that and “maths and theorycraft”?

Wherein I admit that I’m a noob

I am pretty up to date with elemental shaman theory.  I use a spreadsheet to work out my best practice and compare the results with other community-generated tools, and evaluate my logs after difficult fights to see where I could do better.  I am a total minmax beast.

For every single other class and spec I play I just sort of wing it :D

I ask friends who I respect for advice, I think about what aspects of the playstyle I’m good or bad at, what situations I want to do best in, all that sort of stuff.  I read those excellent and accessible guides which litter the internets to get basic ideas of stat priorities, glyphing and talent choices, all that sort of stuff – but at the end of the day I take what’s available to me and do what I think will work best.  I enjoy doing that just as much as I enjoy the in-depth analysis and theorycraft that I use for my raiding main.

But if you ask me for advice about how to play those classes or specs, I’d have to give you a very qualified and incomplete answer because I genuinely don’t know.  I can talk about my experience, what I find fun, the strengths and weaknesses of my own choice of spec and gear and glyphs and whatnot, but I’d have to say to any serious inquirer that they should go find a really good in-depth guide and theorycraft tools and play around with those.

Certainly if someone gives advice based on such evidence I would not flatly contradict it based on my own noobish messing about.  I might say something like “I think Blood is overrated as a tank spec as compared to Unholy and Frost” and give a specific argument as to why, but without those reasons – even if they’re bad reasons – there’s no way for anyone to evaluate whether what I’m saying has any merit, and my advice is pretty much useless.

I don’t agree with the given conclusions from every piece of theorycraft I read, nor do I advocate what I produce here as some sort of shaman gospel, but I can’t think of any other way to go about discovering best practice than to use a carefully considered combination of tools, experience and analysis.  And if someone doesn’t want to use those tools, that’s fine by me!  It is, after all, a game that we play to have fun and enjoy ourselves (though, admittedly, with other people).  But if I’m not permitted to base my own conclusions and advice on all this lovingly-crafted theory stuff then what can I base them on?

About these ads

Discussion

3 thoughts on “What’s better than math?

  1. Your blog posts may not be shaman gospel, but ZAP! is about as close to elemental shaman gospel as we have right now. :P

    Posted by Ishdarei | September 10, 2010, 12:50 am
  2. This is a great subject. I’m glad to see it addressed.I don’t have the capacity to theorycraft anything for myself. I mean, if there’s no dollar signs attached to numbers I’d rather just not get to close to them if I don’t have to. I don’t wanna catch anything. That being said, I HAVE to have people like you out there doing the math for me. My entire view of WoW as a game changed once I started actually researching my classes. It has become immensely more fun now that I don’t suck. Sure,intuitive game play can make the difference in any given encounter. Knowing when, where and who to heal instead of dps for a given gcd aren’t going to come from math (they could, technically, but not in a practical sense)especially in a raid environment. But people who dismiss theorycraft as “just math” or act like it doesn’t translate to a raid environment are either too lazy to actually read up on it or quite simply don’t have the ability to understand any of it. Everything I read from theorycrafters seems to say “in a patchwork situation” or some other disclaimer that is perfectly apt. Anyone, like yourself, who can do the numbers and see the values and benefits of doing what when and in which order seems to understand that it is modeled in an ideal environment and that is not always the case in a raid. Obviously, this is true more often than not, but that doesn’t matter because sound principles of theorycraft are a base point, a launching pad for an intuitively good gamer to put into practice. An informed and educated player of decent “skill” will always outperform an uneducated and uninformed player of superior “skill” all things being equal and with minimal intervention from the RNG Gods. I just don’t see where there can be an argument otherwise. Sure the uneducated can do well on any given encounter, but that actually strengthens the argument for theorycraft in the long run.
    Sound math is never “wrong”, but it can be misapplied. Just because I don’t know what the hell a coefficient really is doesn’t mean that it doesn’t exist or matter. I don’t mind someone saying “I don’t really care about the numbers, I just like playing the game”. That is a perfectly fine stance to take, but taking that stance takes the person out of the conversation and is different entirely from saying “I disagree with theorycraft”.

    Why does this argument sound so much like the one I keep hearing about whether Boise State should be ranked in the top 3 or not?

    Posted by Tex | September 10, 2010, 3:00 am
  3. When you boil this game down, it is a numbers game. There’s gotta be some logic and ways of explaining how, say, 2 identically geared players of the same race/class/profession combination (literally clones of each other) can have even the slightest disparity in DPS (since DPSers are so concerned with numbers, I’ll use us as an example). All sorts of numbers come into play; amount of time spent moving, connection latency, the beloved RNG factor (say, getting Unleashed Magic then later being the target of frost tombs multiple times vs. Sindragosa)…the list goes on.

    While I won’t profess to be a number crunching theorycrafter myself, I keep a close eye on such things and do my best to think it all out as logically as possible. Better we at least try that rather than be “ME LUV BIG NUMBERZ” and not know nor care how they are achieved. Just my opinion, though.

    Posted by Kazgrel | September 14, 2010, 2:45 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 27 other followers

%d bloggers like this: