Pictured left to right: Mary Bucci, Partner, Finance, Brown Rudnick, and Katie Manty, Director, Online Marketing and Analytics, Fidelity. A special thanks to Brown Rudnick for graciously hosting our event!

The language of analytics is very appealing to our fact-based culture, but some metrics are simply not to be counted on.  Boston Women in Finance's March program, led by speaker  Katie Manty, Director, Online Marketing and Analytics, Fidelity, provided insights into being savvy with data.  She noted, “Data can be convenient and comforting, but it’s important that you be curious and questioning about what the numbers are telling you.”

Katie gave us four questions to ask when presented with analytics:

• What is the motive of the person/organization presenting the data?
• What is in the numerator?  How has the sample been selected that is being tested, surveyed, measured or monitored?
• What is the denominator?  How has the overall population been defined?
• How is the data being presented?  Is it a pie chart, bar graph or trend line?  What is the scale on the vertical axis?

For motive, consider that everyone has a purpose in presenting information.  They want to make their “business case” so that their cause will be rewarded.  For example, when the CDC reports an increase in the number of flu deaths in North Carolina, it’s important to remember that the CDC, like every other federal agency, is always interested in increasing their visibility and funding.  The context of the number is also important.  In this particular example, the total population of North Carolina had also increased during that time period so the increase in flu deaths may have simply been a result of larger numbers.

The second question to ask when presented with a percentage, “what is the sample set,” in other words, what makes up the numerator?  In her role analyzing chat center data, she has to be careful not to use only the data around weekend chats, or only weekday interactions.  Rather, a sampling of both is important to come to conclusions about all client interactions.

Just as important as the sample set is the overall population.  Here she pointed out that as we spend more time analyzing non-structured data, i.e. language and words, it is critical to understand not only what has been included, but also what has been omitted.  For example, if you are using a “word cloud” to analyze conversations, somewhere in the initial identification someone had to indicate which phrases would be included.  If no one checked the box to include the word “fees,” it is possible an analysis would lead to a conclusion that costs are unimportant.

Last, Katie advised that we make careful note of the way in which the data is presented.  A common misrepresentation is to stretch the vertical axis so a small change in absolute value can look like a steeply sloped  change.  Katie also cautioned those who produce charts not to over-complicate the presentation.  She advised, “Just because you took the time to learn the double axis trend-line overlay, doesn’t mean you should use it.”

Remember: the goal of analytics is to produce and interpret useful information.  Be mindful of tripwires such as built-in bias, non-random samples and misrepresentative conclusions so you can better interpret and judge the validity of analytics. 

Join Our Next Program!

Join us on Tuesday, April 8, 2014, as we present "Getting To Yes: The Art of Negotiation, Persuasion and Collaboration," where our panel of arbitrators, HR experts and state officials will discuss their best practices to motive their team toward positive outcomes. 

Registration is open now - click here to get your ticket!

- Posted by Beth Kurth, President, Boston Women in Finance