William D Judge

2015 KROGER.COM REDESIGN & REARCHITECTURE Usability Study Facilitator’s Guide Prepared By: Bill Judge User Experience Architect THE KROGER CO. July 1, 2015 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Videotape Release Form I agree to participate in the study conducted and videotaped by the Kroger Co. I understand and consent to the use and release of the videotape by The Kroger Co. I understand that the information and videotape is for research purposes only and that my name and image will not be used for any other purpose. I relinquish any rights to the videotape and understand the videotape may be copied and used by The Kroger Co. without further permission. I understand that I can leave at any time. I agree to immediately raise any concerns or areas of discomfort with the study administrator. Your signature: ____________________________________________________ Date: ______________________________________________________________ Please print your name: ______________________________________________ Thank you! We appreciate your participation. -2- 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Non-Disclosure Agreement I, hereby agree to maintain the confidentiality of information disclosed during the usability study or interview whether it is conducted live, recorded, or performed in any other manner as follows: 1) Definition - For purposes hereof, “Confidential Information” shall mean information or material obtained or observed while attending a usability study conducted for the benefit of Kroger. By example and without limitation, Confidential Information includes any and all information concerning techniques, ideas, processes, trade secrets, innovations, discoveries, improvements, research or developments and test results, data, strategies, and forecasts. Information publicly known that is generally employed by the trade at or after the time the undersigned first learns of such information, or generic information or knowledge which the undersigned would have otherwise learned, shall not be deemed part of the Confidentiality Agreement. 2) I shall at all times hold in trust, keep confidential and not disclose to any third party or make any use of the Confidential Information. 3) All notes, reference materials, memoranda, documentation and records in any way incorporating or reflecting any of the Confidential Information shall belong exclusively to Kroger (the “Owner”) and the undersigned agrees to turn over all copies of such materials in the undersigned’s possession to Owner upon request. The undersigned agrees to the above terms of this agreement. Signed: _________________________________________ Date: ______________ Printed: _________________________________________ -3- 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Background Questionnaire 1) Gender: ___ Male ___ Female 2) Age: ___ 18 - 30 years old ___ 31 - 40 years old ___ 41 - 50 years old ___ Over 50 years old 3) Do you have an online account at Kroger.Com? ___ Yes ___ No 4) How often do you use your Kroger.com account? ___ Daily ___ Several times a week ___ A few times a month ___ Less than once a month ___ Never 5) How important is your computer and/or Smart Phone in your everyday life? ___ Extremely important ___ Very important ___ Somewhat important ___ Not very important ___ Not at all important 6) How interested are you in finding and using digital tools such as websites and apps to improve your experiences when planning or completing a shopping trip? ___ Extremely interested ___ Very interested ___ Somewhat interested ___ Not very interested ___ Not at all interested -4- 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide 7) How often would you say you do each of the following? Visit Kroger’s Web site: ___ Daily ___ Several times a week ___ A few times a month ___ Less than once a month ___ Never Use the Kroger App: ___ Daily ___ Several times a week ___ A few times a month ___ Less than once a month ___ Never Redeem a digital coupon at Kroger that you loaded to your shopper card from the Kroger coupon site: ___ Daily ___ Several times a week ___ A few times a month ___ Less than once a month ___ Never Visit Kroger’s Facebook or Twitter page: ___ Daily ___ Several times a week ___ A few times a month ___ Less than once a month ___ Never View the weekly ad online: ___ Daily ___ Several times a week ___ A few times a month -5- 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide ___ Less than once a month ___ Never Make an online shopping list: ___ Daily ___ Several times a week ___ A few times a month ___ Less than once a month ___ Never -6- 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Introduction and Setup (:05) We’re conducting a usability study of a redesigned version of Kroger.com, in order to gain an understanding of customer perceptions of the website, and how well it meets customer needs, expectations, and preferences. A usability study is a qualitative assessment of how easy an application is to use. Usability studies evaluate the following characteristics of a software application: 1. Understandability 2. Learnability 3. Efficiency 4. Memorability 5. Errors 6. Satisfaction For this study, we've chosen a specific type of qualitative usability assessment called a "First Impression" study. Different from other types of studies, the goals of this type are to document your perceptions, expectations, and feelings about the new Kroger.com design. We’ll use feedback gathered during this study to identify issues and inefficiencies and provide recommendations to improve the overall customer experience. Study Structure Our session will last approximately 45 minutes. We'll use an eye-tracking device during the study. The eye-tracking device will show us where on the screen you’re looking, in what order, and for how long. It works by projecting an infra-red stream to your retina, which in turn can be used to tell our systems where on the screen your eyes are focused. In the first part of the study, you'll look briefly at five screens from the new website, and I'll collect your first impressions of each. The second part of the study consists of a longer and more detailed walkthrough of the same five screens. As you explore each page, I'll ask a series of questions relating to your understanding and perceptions of the screen. In the final part of the study, we'll have a brief discussion of the new design, to what degree you believe it will meet your needs and expectations, and how you feel it compares to the existing Kroger.com website. Procedure We're interested in learning from your experience, so please think aloud as you explore each screen. Doing so will help us understand your perceptions and expectations. It’s important to remember that we’re evaluating the new design of the website, not your ability to use it. You may not understand some of what you see. That's natural. It's these "pain points" we're trying to uncover, so we can determine how to improve the website. Is there anything else you’d like to know about the study before we begin? -7- 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Eye-tracking Setup We’re now going to get you set up on our eye-tracking system. Because the structure of everyone’s eyes is different, we need to calibrate the device to work with your eyes specifically. When we start the eye-tracker, the first thing you’ll see is a target in the top-left corner of the screen. In order to properly calibrate the device, I’d like you to follow the target with your eyes WITHOUT moving your head. Once this process is complete, we’ll begin the first task. Do you have any questions before we begin? -8- 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Memory / First Impressions (w/Eyetracking) (:10) To start, we'll look at five redesigned Kroger.com screens. Here's how this will work: • • • I'll display a screen on your monitor for 10 seconds You'll try to remember as much about the screen as possible After 10 seconds, I'll take the screen down and ask you a few questions about it The website we're testing today is very limited in functionality - meaning, most links do not work, and you won't see much of the functionality you'd see when the new website is complete and available to the public. Screen 1: Home We'll begin by looking at the new Home screen. I'll display it for only 10 seconds. Try to remember as much as you can about it. Facilitator Questions: 1. What are your first impressions of the screen? 2. What types of things do you think you can do on this screen? 3. What stands out most to you? Screen 2: Savings Center Next, we'll look at the Savings Center screen. I'll display it for only 10 seconds. Try to remember as much as you can about it. Facilitator Questions: 1. What are your first impressions of the screen? 2. What types of things do you think you can do on this screen? 3. What stands out most to you? Screen 3: Product Listing Next, we'll look at the Product Listing screen. I'll display it for only 10 seconds. Try to remember as much as you can about it. Facilitator Questions: 1. What are your first impressions of the screen? 2. What types of things do you think you can do on this screen? 3. What stands out most to you? -9- 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Screen 4: Product Detail Next, we'll look at the Product Detail screen. I'll display it for only 10 seconds. Try to remember as much as you can about it. Facilitator Questions: 1. What are your first impressions of the screen? 2. What types of things do you think you can do on this screen? 3. What stands out most to you? Screen 5: Shopping Cart Next, we'll look at the Shopping Cart screen. I'll display it for only 10 seconds. Try to remember as much as you can about it. Facilitator Questions: 1. What are your first impressions of the screen? 2. What types of things do you think you can do on this screen? 3. What stands out most to you? - 10 - 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Screen Walkthroughs (:20) Next, we'll look at those same five screens in more detail. We're interested in learning from your perceptions, so please think aloud as you explore each screen. Screen 1: Home We'll begin by looking at the redesigned Kroger.com "Home" screen. Please use the mouse to explore the screen and let me know what you're thinking as you do. Please begin. Facilitator Questions: 1. What types of information and features do you believe are available on this screen? a. How would those features help you save time or money, or improve your life? 2. What words or phrases best describe this screen? 3. What do you like most about it? 4. If you had a magic wand, how would you improve it? 5. What's the most important feature or piece of information on this screen? Why? 6. In the main navigation, you'll see a series of links. What type of information would you expect to see if you clicked on each of them? 7. What do you think is the purpose of the items grouped under the heading "Your Next Shop?" a. How likely do you think it is that Kroger can accurately display the right items at the right time? 8. What types of items would you expect to see in the "Sale Items For You" section? 9. Does this screen contain the right mix of inspirational and lifestyle content (e.g., recipes, party planning tips, etc.), and product content that can be shopped? 10. Would the information. and functionality on this screen inspire you to keep coming back to it? - 11 - 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Screen 2: Savings Center Next, we'll look at the new "Savings Center" screen. Again, use the mouse to explore and let me know what you're thinking as you do. Please begin. Facilitator Questions: 1. What types of information and features do you believe are available on this screen? 2. What do you like most about it? 3. Is there anything about this screen that is unclear or confusing? If so, what specifically? 4. If you had a magic wand, how would you improve it? 5. What's the most important feature or piece of information on this screen? Why? 6. What are your thoughts on the three tabs toward the top of the page? What type of information would you expect to see on each? 7. The product items displayed in the center of the page -- what types of items do you think are being shown? a. Are those items helpful to you? If so, how? - 12 - 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Screen 3: Product Listing Next, we'll look at the new "Savings Center" screen. Again, use the mouse to explore and let me know what you're thinking as you do. Please begin. Facilitator Questions: 1. What types of information and features do you believe are available on this screen? 2. What do you like most about it? 3. Is there anything about this screen that is unclear or confusing? If so, what specifically? 4. If you had a magic wand, how would you improve it? 5. What's the most important feature or piece of information on this screen? Why? 6. What do you think is the purpose of the images and links in the center of the page? 7. What do you believe is the purpose of the links and checkboxes on the far left of the screen? - 13 - 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Screen 4: Product Detail Next, we'll look at the new "Product Detail" screen. Again, use the mouse to explore and let me know what you're thinking as you do. Please begin. Facilitator Questions: 1. What types of information and features do you believe are available on this screen? 2. What do you like most about it? 3. Is there anything about this screen that is unclear or confusing? If so, what specifically? 4. If you had a magic wand, how would you improve it? 5. What's the most important feature or piece of information on this screen? Why? 6. What are the options for us getting you this item if you ordered it online? What do they mean to you? 7. What are your thoughts on the product recommendations toward the bottom of the screen? a. What types of recommendations would be the most relevant and helpful to you? - 14 - 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Screen 5: Shopping Cart Next, we'll look at the new "Shopping Cart" screen. Again, use the mouse to explore and let me know what you're thinking as you do. Please begin. Facilitator Questions: 1. What types of information and features do you believe are available on this screen? 2. What do you like most about it? 3. Is there anything about this screen that is unclear or confusing? If so, what specifically? 4. If you had a magic wand, how would you improve it? 5. What's the most important feature or piece of information on this screen? Why? 6. The options toward the top of the screen (shopping in store, substitutes, fulfillment) -- is their purpose clear? What do you think they mean? 7. What would you expect to happen if you clicked the "Checkout Now" button? a. How and where do you believe you would receive the items you ordered? - 15 - 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Discussion (:05) 1. Now that you've seen the new website, do you feel you'll use it in the future? a. What would be the primary reason for doing so? 2. Can you think of any other digital tools we can provide to save you time or money, or to help you more effectively plan or conduct your shopping trip? 3. How do you feel this new website compares to Kroger's competitors (Target, Walmart, Whole Foods, Costco, etc.? 4. Would you recommend the new website to others? a. If so, how would you describe it? [Display hard copy screenshots to participant] 5. How do you believe the new design compare to the current version of Kroger.com? a. Which do you prefer? Why? - 16 - 2015 Kroger.com Redesign & Rearchitecture – Usability Study Facilitator’s Guide Post Study Questionnaire (:05) Please indicate your level of agreement with the following statements: Statement Strongly Disagree Disagree Neither Agree or Disagree Agree 1. The new website is visually appealing. 2. The new website is well organized. 3. The new website is clear and understandable. 4. The new website is interesting. 5. The website is vibrant and uplifting. 6. The new website is useful to me. 7. I prefer the new design to the existing Kroger.com design. Congratulations, you have completed the Kroger.com usability study. Thank you so much for your help! - 17 - Strongly Agree