Sample Segmentation: Focusing Satisfaction Surveys for Sharper Insights

Customer satisfaction surveys are useful because they enable Consumer Affairs managers to obtain fresh customer reaction about their companies’ products and services. Timely surveys yield new information and insights for input to product and service improvements. In addition to measuring agent performance, they allow companies to understand the perceptions, challenges and needs of their customers.

New to satisfaction survey technique in Consumer Affairs is how some managers are segmenting contacts by selection criteria to better examine the reactions specific customer subgroups are having to their companies’ products and services. This focused practice is helping them gain a more accurate and complete understanding of customer reactions and sentiments and a more refined assessment of their customers’ particular needs and wants.

Survey administrators apply proven survey methodologies that ensure quantifiable results are obtained from respondent feedback. Managers use the results to establish baseline levels of satisfaction for ongoing annual, semi-annual, or quarterly surveys. To attain the desired survey results some departments are conducting in-depth interviews using skilled interviewers whose questions are carefully designed and scripted to elicit substantive information about specific products and services. These survey samples are usually of two hundred or more customers who fit a specific profile of criteria.

Using their consumer relationship system (CRS) tools, these managers are extracting target customer sets from their contacts databases. When the surveys are conducted by outside specialist survey vendors, the practice is to provide these clients with anonymous findings. However, when surveys are conducted in house using online tools integrated to the CRS, they can be set up to link the individual responses back to their individual consumers, if desired. This linkage establishes a direct connection between the survey responses and the actual outcome of the conversation.

Current survey technology captures the customer verbatim and links it to the evaluation process to drive quality and provide feedback. This linkage facilitates future reference, analysis, and reporting. It allows measurement not only of customer satisfaction, but of first call resolution from the customer’s perspective and can help provide understanding of what drives multiple calls. This integration facilitates understanding of the relationship between variables and the impact on willingness to recommend the brand and Net Promoter scores.

Cosmetics firm monitors pulse of consumer sentiment

The Consumer Affairs director of a renowned global cosmetics company surveys customer satisfaction with its products and service every six months. This practice helps the firm regularly verify their customers’ sentiment and satisfaction with its brands.

In practice, a professional market research survey specialist conducts telephone surveys of select sets of two hundred customers. Sample selection criteria include: 1) which perfume brand item has been purchased, 2) what each customer’s concerns and issues are, 3) that the customer contacted the center in the past six months, 4) that the customer is ‘acceptable’ for providing input, 5) that the customer’s demands for satisfaction have been reasonable, 6) that the customer reported either ‘happy’ or ‘unhappy’ sentiments in past contacts, and 7) the contact center representative who handled the customer’s contacts is known. Customers who are ‘repeaters’ and those with severe allergic reactions to the perfume products are excluded. And, finally, the sets must include equal portions of favorable and unfavorable contact reports.

The survey sample sets also are segmented by product and purchase location. Shoppers purchasing prestige packaged items in exclusive stores receive one survey version, while those purchasing differently branded items at mass merchandise stores receive another. Each sample set is retrieved and extracted from the departments’ CRS database with the complete contact information, including name, address, telephone, and email address.

This survey’s satisfaction measurement methodology, developed at Harvard Business School, produces feedback guidance on participants’ future purchasing intentions. The survey questions are designed to enable quantitative measurements of satisfaction that include such characteristics as the customer representative’s courtesy, the center’s fulfillment process, and how well promises were met, as well as comparative benchmarking against the firm’s competitors. The survey specialist vendor provides an executive summary of survey results with quantitative analysis and verbatim transcripts of participating customers’ responses.

To manage cases in which the customers have called back asking for a goodwill gift replacement, the firm added a field in its CRS contacts entry screen to record the shipper tracking number, delivery confirmation, and date received. This practice helps contact center representatives handle calls more effectively, and it enables them to provide callers with the shipping number, to confirm delivery and date delivered, and to minimize fraud.

CPG company delivers best-in-class practice

The Consumer Relations department of a leading international consumer packaged goods company monitors their service satisfaction level on a weekly, monthly, and quarterly basis with an annual roll-up report for top company management. The annual customer satisfaction level is reported as a best-in-class rating on a five-point scale where the goal is 85% or greater of customers surveyed responding at the top satisfaction level. Annual variance of a couple of percentage points is typical. Factors causing the fluctuation are identified, such as a new representative staff in the contact center, and explained.

Random samplings of consumers are extracted from the company’s CRS contacts database for its on-going program of customer satisfaction measurement. These samplings include all contact types (email, letter, and telephone), products, and issues. Excluded from the samplings are non-product inquiries and escalated issues. And only customers who have given permission to be re-contacted are included in the samplings. The samples comprise a representative cross section of contact type, method, product, reason/issue, and representative.

The company outsources its Customer Relations contact center and provides the outsourcer’s managers and trainers with its product and service policy information. The call center vendor trains its representatives who are dedicated to supporting the company.

The company’s consumer research manager reviews the periodic satisfaction survey questionnaires to be sure they are standardized and will provide the data needed for consistent year-to-year performance comparisons. The survey results are discussed, evaluated, and shared with the outsourcer’s managers for making adjustments and taking corrective reaction if needed. Several survey formats are used to carry out this program.

The company’s survey vendor also conducts weekly satisfaction surveys of 750 recent customer contacts selected randomly from the company’s CRS contacts management database. These short telephone surveys are designed to quickly gauge the contacts’ satisfaction with the contact center’s service in responding to their inquiries. Weekly results are rolled up into monthly customer service satisfaction measurement reports.

The vendor also conducts quarterly call-back surveys of customers who have called or emailed inquiries. A profiled selection of 600 telephone and 400 email contacts are surveyed in depth by their self-chosen contact method. Contacts who called are called for interview, and contacts who emailed inquiries are emailed the survey. The in-depth surveys ask pointed, probing questions designed to elicit quantifiable responses and capture current customer sentiments.

Clarifying surveys build understanding

To gain insights about customer satisfaction with specific products, the company uses their CRS survey module to conduct ‘clarifying’ surveys of groups of customers who have purchased those products. Either internal clients, such as marketing, quality, and product development, or the Consumer Relations department initiate these on-going surveys which focus on new or existing products. They are designed to elicit a better understanding of customer reactions and apply insights for improvements that will increase customer satisfaction and sales.

Contact center representatives’ performance is measured on a quarterly basis. This quarterly measure contributes to a measure of overall satisfaction level that is the key overall measure for the center. Survey results are not reported identifying individual contacts, except for contacts who report they have not received fulfillment of their issues. In those cases the company ensures the customers’ issues are resolved.

The company receives survey result reports from the vendor that do not reveal the contacts’ individual responses, and results are not stored in the company’s customer contacts database. However, the company does receive anonymous customer verbatim feedback reports from survey specialist’s telephone surveys that yield ‘nuggets’ of valuable customer sentiment insights. This information is culled through the surveyors’ skillful probing during the telephone interviewers and provides the company a better understanding of customer reaction to its service. These insights help the company know how to deal with its consumers better. They also enable the outsourced contact center managers to react and make adjustments to the perceived trends that can raise the satisfaction level.

Considerations for effective survey practice

Several approaches and methods of conducting post-contact customer satisfaction surveys are prevalent. Some are more appropriate and effective, depending on the nature of the products and services customers have used, as well as the goal of the survey program. Often, the best method is dependent on what you want to ask and how quickly you want to ask it. Cost can also be a consideration and can vary widely depending on the method employed. Consumer Affairs managers can best determine which approaches and methods provide the best results for their organizations.

Prevalent methods used to gauge post-contact customer satisfaction include

• In- and out-bound IVR surveys,
• Mailed letter and business reply card surveys,
• Anonymous public Web-form surveys,
• Post-contact emailed Web-form surveys,
• During-call interactive surveys, and
• Short and long in-depth telephone surveys.

How well these methods work in enabling capture of current consumer sentiment varies. [Survey method insights described are derived from an interview with Richard Shapiro, president of The Center for Client Retention, based on his Leveraging Customer Satisfaction Research to Reinforce Call Center Value, presented at Call Center Week, June, 2008. www.tcfcr.com.] The more immediate and direct the contact between customer and surveyor and the more interactive and spontaneous the dialogue between customer and surveyor, the better the opportunity to gain key information and insight. Mailed survey letters or business reply cards, as well as emailed surveys, are less immediate than live interactive surveys during the initial or follow-up calls between the consumer and representative. Often the most important aspect of feedback is timing. Gartner has determined that feedback collected immediately after an event is 40 percent more accurate than feedback collected 24 hours after the event. [Gartner Group, Implement Customer Satisfaction Management Processes to Improve Revenue, March, 2006.]

An IVR survey can be immediate, quick and short, with multiple-choice, opinion-scale, and open-ended questions. New speech recognition technology has reduced or eliminated many of the drawbacks of this method and it is very cost-effective. Most importantly, an IVR survey provides immediate customer feedback that can deliver real-time alerts to trigger "save the customer" actions. On the other hand, if immediate feedback is secondary, the flexible flow of an in-depth telephone interview enables the surveyor to probe the customer’s responses to specific questions and to explore nuances in the response and uncover insights of value to the company. Of course, live survey responses may be recorded and transcribed like IVR surveys if the budget allows.

What is to be gained from establishing a sample segmentation survey practice? Generally, managers use segmented sample surveys to discover reactions of particular sets of customers and record their sentiment regarding particular products and services. These surveys help them uncover ways to retain customers who may be at risk of defecting due to unrecognized or unclear reasons for dissatisfaction with the products or services. Managers also use the surveys to measure customers’ perception of contact center representatives’ skills in handling issues, to assess the value of their fulfillment processes, and to learn customer views of product and service changes or enhancements. Segmented sample surveys also can yield significant intelligence about competitors and collect data for assessing the ROI of contact center operation.

Some question types and formats are more effective for collecting particular types of information. When the aim is to quantify preferences from among several specific options, use a "pick list" that requires respondents make a definite choice or choices. Such questions may be designed to force respondents to communicate judgments on an absolute scale of 1 to 5, for example, to indicate an intensity of preference from neutral (e.g., -3, -2, -1, 0, 1, 2, 3), or to rank choices by an order of preference. Alternatively, if the aim is to collect descriptive personal impressions, then it is more effective to use a "fill-in-the-blank’ format with space for respondents to enter one or more words, or even paragraphs in an open-ended essay.

The criteria used to generate a well-profiled set of customers targeted for focused surveys are as important as the questionnaire design is for eliciting the desired result. Whose opinions does the company want to capture and from whom extract specific insights? Once-in-a-life-time contacts may have general preferences and knowledge of competitors that is desirable to understand better. Customers who repeatedly purchase the company’s products and services are loyal buyers who have extensive experience of them. They can provide valuable input for improvements. Other selection criteria to apply may include first-time versus current customers, age, sex, marital, and family status, as well as other social demographics and ethnic characteristics.

Today, when customers immediately and openly voice both their positive and negative points of view through social media, companies need to protect their valued customer relationships by forging a clear and immediate link to the voice of their customers.

Focusing customer satisfaction surveys to gain sharper consumer insights through segmentation may be a challenge, but the ability of segmentation to produce needed market intelligence and consumer insights that can continuously improve companies’ offerings and strengthen their customer relationships is worth the effort. Successful respected consumer packaged goods companies continue to fine-tune their customer satisfaction survey practices because they recognize that the surveying is a tool that can yield sustainable competitive advantage. And, segmentation gives them more precision in finding out what best satisfies their customers and why.