Building an IA Framework for K-12 Educator Dashboards: A Case Study

For the past two years Discovery Education has been developing a robust set of interconnected dashboards to work within its Techbook product line up (think digital textbooks) for K-12 educators. These dashboards aim to provide teachers with information regarding student and/or class performance on various assessments found throughout each Techbook. The IA framework for these dashboards is built on three highly intertwined constructs: Data, Tables, and Connections. Here Data refers to the way information is represented visually, Tables refers to the way the Data is structured, and Connections refers to the way the said Tables containing said Data are connected to one another.

Continue reading

Mobile Research Toolkit

In the world of UXR, artifacts are key. Therefore, it is crucial that any researcher be equipped with a the proper tools to collect artifacts both in the lab and in the field.  The following post outlines what you’ll find in my mobile research toolkit (MRT) – and by mobile I am referring to research on-the-go as well as research involving mobile devices.

Parts

The first rule of putting together a MRT is don’t paint yourself into a corner. What I mean by that is, over time, you’re most likely going to be using the pieces in your MRT to test different devices in different settings – you’re going to need parts that are malleable with the ability to be easily disassembled for travel as well as rebuilt in a variety of ways to accommodate the circumstances at hand. Therefore any parts you use must be versatile in terms of configuration and software compatibility. Now, I’m not saying you have to use the parts I’m about to describe – I’m sure there are plenty of better ways to go about it – I’m just saying think through your configuration before purchasing parts in order to reduce the expenditures tied to trial and error.

Numbered2.fw

  1. Podium with three 5/8″ bores
  2. Microphone Gooseneck 5/8″ (female) to 5/8″ (male)
  3. 5/8″ (male) to 5/8″ (male) microphone mount adapter
  4. Mini Microphone stand – 5/8″
  5. iRig Microphone
  6. iPhone to mic stand mount
  7. Go Pro Pieces (enter in specs)
  8. Mic Clip – 5/8″
  9. Go Pro Pole Mount
  10. iPhone to universal camera mount adapter
  11. 5/8″ mic (female) to universal camera (male) adapter
  12. Go Pro to universal camera mount
  13. Agent V5 Webcam
  14. Go Pro Clip Mount (on iPhone case)
  15. iPod touch

** Note: The Content below is still a work in progress. **


 

Split

I store all my equipment in a hard plastic case I picked up from The Container Store.  You can also find the egg crate foam to keep your parts safe and from rattling around.

Mobile Device Research

MobilePhone

CamAdapter

iPadResearch

 

 

Remote Research

One thing with remote research I cannot stress enough is to use a separate laptop as your portal (connection / screen share) and payload delivery (prototype, wires, etc.).

tip7

Here, while you take notes and communicate with your team on your primary laptop you can utilize the portal laptop for:

  • Screen sharing (join.me)
  • Participant communications (Skype)
  • Study payload delivery
  • Session Recording (ScreenFlow)
  • Best to keep small (13in)

Now, for a complete, efficient remote research setup I suggest the following:

tumblr_m9uidtS4ra1r6hhxmo1_r1_1280

Moderator (laptop)

  1. Note taking (workflowy)
  2. Team (observer) communications – most important
  3. Test plan referencing
  4. General trouble shooting

Portal (laptop)

  1. Screen sharing (join.me)
  2. Participant communications (Skype)
  3. Study payload delivery
  4. Session Recording (ScreenFlow)
  5. Best to keep small (13in)

Active Observer (laptop)

  1. Note taking (workflowy) – most important
  2. Team (observer) communications
  3. Test plan referencing
  4. General trouble shooting

The Mass Signup

signup

My first UXR study campaign was sort of an experiment in itself.

I wanted to test the waters to see how end-users of our product would respond if given the chance to participate in research aimed at improving overall user experience.

If successful, I wanted to bring my coworkers into the rhythm of design-test-modify by providing weekly UXR sessions where we would test whatever was available at the time.

My sign-up calendar on mysignup.com had 8 open sessions every Wednesday for 6 months. Thus totaling 24 dedicated UXR days or 192 individual sessions. Overall this was really a cheap and easy shotgun approach that only cost me $3… and that was to get rid of the adds on the mysignup.com page. However, if it worked, I’d be set with participants for the weeks ahead and could focus on some other aspects of my position.

Now, I don’t remember the exact time the campaign was sent out but I sure as hell remember when the signup notification emails started pouring in – and I literally had to turn off my iPhone in order to fall asleep because it was vibrating about every 2 minutes. That sign up sheet that extended 6 months in time… yeah, it filled up in less than 48 hours. When the signup sheet filled up, then the emails regarding when I was going to open more sessions started flooding my inbox. It was something like 500 emails in total and it took me weeks to recover from the bandwidth consumption needed to reply to them all. Needless to say, I learned a thing or two from this one. Yet, all in all, as one participant said it best…

“These are the kinds of problems you want to have.”

My Initial UXR System

Setting up my initial UXR system revolved around four main objectives.

  1. Having something to test
  2. Finding someone to test it
  3. Providing a way to get that someone to test that something
  4. Establishing a way to observe that someone testing that something

Now 1 and 2, for me, were actually the easiest. Coming into an established company one can expect that there are a few legacy products or works-in-progress lying around waiting to be tested. Further, coming into an established company that caters to a pretty specific demographic, e.g. college and career counselors or like personnel, one can expect to be a step ahead in regards to connecting with your target demographic. So for objective 2, I was able to score a list of ~2,500 ‘power users’ of our legacy product. These power users were the most active members on our customer feedback forum.

Objectives 3 and 4 were a bit trickier and required some ingenuity and systems thinking. I had to provide a way for interested end-users to signup for individual sessions, receive confirmation, and understand what to do when it was time for them to participate. I should mention now that I planned on utilizing remote research for the purposes of testing each study’s payload. In brief, remote research involves… you guessed it… connecting with your participant(s) remotely. Here, participants connect with me over the Internet and are able to view and interact with the study payload in a fashion that allows me to monitor behavior while gathering verbal feedback.

Before I describe my initial setup I’d like to make the following disclaimers:

There were a lot of pain points in my initial setup that have been alleviated over time. Though these pain points were mostly on my end there were other aspects that affected participants and other departments within the company. However, I’ll get to those later.
Though I have made modifications to my system over time, much of it still remains the same. Further, because I am explaining this now-modified setup retrospectively with every intention of providing future posts about future modifications, I will go ahead and posit it in the present tense.
So without further adieu, here are the basics of my initial UXR system.

  1. End-users receive an email notification that Company X needs participants for one or more up coming UXR studies.
  2. Interested end-users follow a provided link to a mysignup.com page where they select from a pre-established set of session dates/ times. Sessions are an hour in length. However, I generally try to keep every remote UXR session to about 30-40 minutes.
  3. When an end-user signs up, I receive an email notification on my end. I then manually respond with session date/time confirmation, information about the study, and instructions on how to connect to me remotely. In addition, I inform them that I record UXR sessions for the purpose of re-cap and review and that I will remind them again the day of the study.
  4. On the day of the study, end-users (now participants) receive a call from me, via Skype, a minute or so before the start of their session. If they have not already, I remind them to connect to the web conferencing tool, join.me, in order to view my screen.
  5. Once the join.me connection is made and the participant is able to view my screen, I ask permission to record the session.
  6. When all is fine and dandy, I generally execute the following outline during each session. However, I’ll provide more information on methods and study plans later.
  • Introduction
  • Gather demographics (if necessary)
  • Explanation of tasks to be performed
  • Participant performance of tasks
  • Study debriefing
  • Q&A

Of note, I use ScreenFlow, a Mac application, to record any UXR session where the study payload can be visible on the screen. A really nice and convenient feature of ScreenFlow is its ‘configure recording’ window, which also houses the little red start recording button. Participants, viewing my screen, are able to watch me click the record button. The subsequent countdown to screen recording that follows is just icing on the cake.

Another great thing about ScreenFlow is that it has the ability to capture all internal and external audio. Seeing that a huge component to any UXR session is the verbal feedback you get from your participants, being able to capture their comments and your conversation clearly is a huge plus, especially if you or a stakeholder plan to revisit the session recording. Further, having the audio connection exist via VoIP provides for the best sound recording quality. Which is why I use Skype to connect with my participants.