Knowledge Resources

The 511 national evaluation panel designed a core set of survey measures intended for use by all 511 evaluators conducting customer satisfaction surveys. The experience of fielding the core questions in Arizona, however, demonstrated that developing a core set of questions that can be used "off the shelf" by all 511 evaluators was more difficult than anticipated. Over the course of writing the survey for the Arizona evaluation, it became apparent that the core set of questions was too long, and it would be difficult for evaluators to use the full set. Prior to the formal pre-test of the survey instrument, there was substantial editing. Some of the core questions were not used or were revised because they were simply not applicable to the Arizona system and its users (e.g., questions regarding ferry service). The original set of core questions was pared down to include those questions that can be used "off the shelf," with little if any revising or tailoring. This revised set of core questions is recommended for use, as appropriate, in 511 customer satisfaction surveys. Observations on the types of questions that comprise the core are noted below (Appendix F of the lessons learned document contains the actual question wording).

• Include questions on general use and awareness of 511.
  • When did you first call 511?
  • How did you first hear about 511? (may need to adapt answer categories)
  • How many times called in the past month?
  • Most calls from cell phone or landline?
  • Ever call 511 in a different state?

These questions provide important data regarding customers' general use and awareness of the 511 service. In particular, the question on how the respondent heard about the service can gauge the effectiveness of different marketing tools being used to promote the service. These questions are also useful because they make it possible to segment customers according to how long they have used the service or how frequently they use it to determine if these use variables are related to satisfaction with the service.

• Include trip-specific questions.
  • Type of information requested from 511 when the call was intercepted (included in the intercept survey)?
  • Where were you when you phoned?
  • Called before trip or while traveling?
  • Type of transportation used for this trip (may need to adapt responses depending on transportation options available in your state)?
  • What was trip purpose?
  • How satisfied were you with information received for specific trip?
  • Did information affect traveler behavior?

The core questions in this section of the survey are designed to assess the caller's experience with the service based on the specific call they made when they were intercepted for the survey. For the specific trip the caller was making, these questions probe the nature of that trip, the conditions surrounding the call to 511, and the customer’s satisfaction with the information obtained. With a sufficient sample size, these questions could shed light on callers' patterns of use regarding the service. These data are critical, for as 511 program managers become better informed about how their customers are using their service and about the circumstances surrounding that use, they will be better able to understand the types of improvements that would be most valued by customers. In addition, a series of questions in this section address whether respondents changed their travel behavior based on the information they had received from 511 (did they take a different route, did they decide to leave for the trip at a different time, etc.). This question is included to assess the effects of the service on mobility.

• Include questions assessing overall customer satisfaction, mobility and efficiency.
  • Selected agree/disagree items for road information users.
  • Selected agree/disagree items for transit information users.
  • Selected agree/disagree items on ease of using 511, efficiency of system.
  • Benefits obtained using the service.
  • Improvements to the system (this list will have to be revised, based on the particular needs of each service.)
  • Quality of radio traffic reports versus 511.

The set of questions in this section of the survey is a useful complement to the trip-specific questions. These more general questions provide a broader view of the customers' experience with the service. They enable the evaluators to learn the different ways in which individual customers use 511, and provide data with regard to less-frequently used features, like transit and tourist and airport information. Moreover, for certain questions, such as ease of using system and whether the caller would recommend the service to a friend, it makes sense to base the question on overall experience with the service, rather than just a single experience. Several series of questions measuring customer satisfaction with the different types of information being provided by 511: road, transit, and airport and tourism were included. Respondents were screened appropriately, so that they were asked the satisfaction question only if they used the particular type of information. Another key question measured customers' perceptions of the benefits of using the service. Again, these data increase our understanding of why customers are using the service (e.g., is it to reduce travel time, to avoid congestion, to make better informed travel decisions) and thus provide insight on traveler behavior and on improvements that customers would find useful.

• Collect Demographic Information.
  • Gender
  • Age
  • Education
  • Income
  • Zip code

Demographic questions help 511 deployers understand the personal, social, and economic characteristics of their 511 customer base and are useful in guiding decisions on marketing and advertising programs. In addition, demographic questions are useful for the purpose of subgroup analyses, as they provide a means by which to measure the relationship between observable characteristics and traveler behavior (or purchase choices). Also, research can then be conducted on whether use and perception of the service differs by demographic groupings.

The core set of questions address key topic areas that will enable evaluators to better understand who their customers are, how they are using the service, and their satisfaction with different aspects of the service. Evaluators will want to supplement these questions with additional questions tailored to their service. (See Appendix C of the lessons learned document for the full set of questions used in the Arizona survey). In developing a core set of questions, the goal is to provide 511 evaluators with the tools they need to conduct their own user evaluations. To the extent that state evaluators use similar methodologies and employ the same questions (the core), it will be possible to compare survey findings among different states and draw more general conclusions about nationwide use and perceptions of 511.