Friday, March 21, 2014

Your Initial CX Assessment...Part 2 - Internal Surveys

Continuing on in our initial attempt to get a high level understanding of the state of customer experience at Widget Inc., the initial focus is on the information contained in customer surveys.  The previous post, looked at industry-wide surveys typically conducted by market research firms, and identified some key items to look for from these sources, including peer rankings across various metrics, and always being cognizant of sample sizes for each question.  In this post, we’re going to look at internal (sometimes referred to as “proprietary”) surveys conducted by companies themselves.  Many large and mid-size organizations will typically contract with a market research firm to execute their own survey of their customers.  These surveys are generally of two types...

Transactional Survey - in this type of survey, the customer receives a questionnaire after a particular transaction such as, for example, an online or in-store purchase, or after a contact with the company’s call center.  Consequently, these surveys tend to be very “diagnostic” in that they ask questions about specific aspects of a transaction (e.g. were you greeted promptly, was the facility clean, how knowledgable was your attendant).  Transactional surveys tend to be the most common type of method companies use to gather customer satisfaction feedback.

Relationship Survey - these surveys can take a couple of different forms.  They could be presented in the form of a “brand health” assessment or audit, where respondents are asked to rate various components of the company’s operations including its products, service, community involvement, etc.  Given the general and high-level type of questions asked in a brand health survey, they are not likely to provide a very revealing or informative picture of the customer’s satisfaction with the company’s offerings.  Another type of relationship survey, asks more general questions about the customer’s experience over an extended period of time (i.e quarterly or annually).  The objective of a relationship survey is to account for some of the variability often seen in transactional surveys by asking customers, not about a particular encounter with the company (which may be positive one occasion, but negative on another), but about their perceptions across a variety of encounters with the company over time.

Let’s now move on to discuss some of the key benefits and drawbacks associated with most internal surveys.

Whether discussing a transactional or relationship method of gathering customer feedback, a key draw of an internal survey is the potential for associating a respondent with a particular transaction.  Typically, at either the beginning or end of the questionnaire, customers are asked for permission to reveal their identify, and in my experience, many respondents will disclose their name on the survey.  
The benefit of capturing the customer’s name is realized if the organization maintains a comprehensive transactional database.  If properly designed, these databases will contain a customer’s contact information, as well as their transaction history with the company.  Transactions could include purchases, repairs, and warranty claims, to name a few.  By matching a customer’s internal survey responses to a particular transaction, or series of transactions, insights can be derived into such areas as...
    • Do customers with similar attributes share the same type of feedback?  For example, if the company’s database segments customers along particular attributes, it would be worthwhile to determine whether other customers in that segment provided similar responses to survey questions.
    • The identification of a customer by name also facilitates the ability for the company to contact that person and offer a remedy as a response to an unsatisfactory experience.  This response to a problem takes on added importance in the case of loyal customers who frequently repurchase.  We’ll explore this “closed-loop feedback” method in much more detail in future posts.
    • Like the third-party industry survey we reviewed in the March 15 post, a well designed internal survey should also contain multiple years of results.  This allows for the trending of customer feedback over time in order to identify specific points where performance improved or deteriorated.  Your knowledge of trends will be critical as you proceed with your initial customer experience assessment because you’ll be in a position to ask what may have occurred organizationally at the time of the change in performance.
    • Finally, a big potential benefit associated with internal surveys is that they typically capture unfiltered and unedited customer comments.  If you have a large enough sample of replies, these “verbatim” comments can be invaluable sources of learning and assessment.  Spend some time with these comments and discern any trends and insights.  For example, are they gender specific?  Do they come from particular geographic or sales regions?  Are they associated with particular products or services?  Be careful not to draw conclusions from only a handful of replies.

Unfortunately, internal surveys can also have their share of shortcomings that you ignore at your peril. Let’s take a look at a few of those drawbacks that, if not recognized, can undermine your attempt to get a candid picture of your company’s customer experience...

  • Bias towards more favorable ratings - perhaps it’s because customers perceive that there responses may be seen by the company’s staff, they tend to indicate relatively higher levels of satisfaction than on surveys administered by a third-party.  This is the case even when customer submit their replies anonymously.  Conversely, customers tend to be more candid when they perceive that their responses will be viewed by an independent source not affiliated with your organization.  If you have access to the results of a third-party industry survey of your customers, it’s worthwhile to identify similar or identical questions that appear in both your internal survey, and the industry survey.  Now you’ll be able to compare how your customers responded to these similar questions...take the time to identify any trends in the ratings.  All other things being equal, I would have more confidence in the validity of the replies from the third-party survey...just make sure that there’s a statistically valid sample size.
  • Survey “gaming” - a second potentially significant drawback with internal surveys is, unfortunately, the possibility of customers being manipulated in some way by the staff they dealt with during their transaction.  Gaming can take several forms including a bribe - “if you give me a good score on the survey you’re going to receive, I’ll throw in a little extra in your purchase;” or a hard luck story - “my compensation is based on the score you give in the survey...”  These types of manipulations tend to be more common in some particular industries, so be on the lookout for this.  If you can identify widespread “gaming” be very careful about drawing any substantive conclusions of what your customers are saying about your company.

No comments:

Post a Comment