Advice from Experienced Learning Professionals for Anyone Creating Training

What is Training?

• Training is a process of designing materials & experiences that help people to understand key information and change behaviors or actions.
• Keep training simple and designed so people learn from it.
• Training is more than an event; it’s a process and shouldn’t just occur in the classroom.
• Training can take many forms and should cause the audience to solve problems.
• Focus training on the audiences’ needs and use techniques to help them to want to learn.

Start Training Creation with the End in Mind

• Understand the goal of the training.
• What are you trying to accomplish?
• Understand stakeholder expectations and the processes people are currently using.
• After knowing expectations, learn about the audience.
  • What are their expectations?
  • What are their current experiences?
  • What will resonate with them?

Measuring Training is Important

• Measure your training and compare it to your desired outcome
• Use baselines to measure change, including change in available business metrics
• Focus training on what is NOT being done and what can be done better, and measure the impact
• Get feedback from participants to make sure you’ve impacted what you intended
• It’s okay to be wrong as long as you adapt and change as needed

Advice for Working with Subject Matter Experts (SMEs)

When working with Subject Matter Experts (SMEs):

• Talk with and focus the SMEs on what’s important for others to know.
• Set your expectations for them at the beginning.
• Use conversations, questions, and their stories while building a relationship to get key information.
• Don’t expect to get everything you need in one session.
• Check back with the SME to make sure the information you gathered is correct.
• Check and test what you create with your SME and with your target audience.

Using Effective Images and Video in Training

• Images and videos can help your audience learn and gain information quickly.
• Visuals should be relevant to your audience and have a clear purpose.
• Focus on helping viewers learn more than cool effects or movie-making.
• Anyone can make effective visuals for training.

Anyone Can Create Impactful Training

• Focus on outcomes
• Bring your value, experience, process knowledge, and information to the training you create
• Formal training can be helpful, but not required
• Ask questions and be open to learning opportunities
• Improve by reading, looking for inspiration, and taking responsibility for your own development

The Video Creator’s Guide to Working With Subject Matter Experts

Understanding Measurement for Videos

Understanding Measurement

One of the challenges with any training program is ensuring it achieves what it was designed for. The need for measurement is prevalent and critical if you want to know that your training solution is working.

You should put a lot of thought into ensuring you can measure your training. Planning to measure should start in the early phases of your development process. And before you write a single sentence, record a single video, or make a single image for your training, you should know:

• What are you going to measure?

• How does your measurement help you understand effectiveness?

• How will you assess if your audience is learning or can perform better or as needed?

• How will you gather your measurement data?

• What tools and resources do you need to gather the data?

• Are there key data points that your stakeholders will want or require?

• What will you do with the data once you’ve collected enough to determine the impact?

These are just a few questions you might consider as you prepare your course.

Many great resources are available to help you out with measurements, and we’ve listed a few below. The biggest takeaway is that you should plan on setting up metrics that help you understand the goal of the course. (e.g., help new instructional designers create a simple storyboard.)

This quick guide will focus on some of the measurements available for videos. Not all video players and hosts have the same metrics available, although many have similar ones or may call them by different names. Below are a few available common ones and what you can generally understand by looking at them.

4+ VIDEO METRICS YOU SHOULD KNOW

The analytics you will want to track and look at will depend on your goal and what you’re trying to understand. Remember, different systems may give you different information or call it different names. Below are the metrics you should know, what they measure, and what that information can tell you.

PLAYS/VIEWS

Usually, one of the most straightforward metrics to track with video is the number of times a video has been played. The play number may not be helpful as a standalone metric, but it can provide some key insights. First, the number of Plays tells you if your video is getting watched. If not, is there a technical reason people aren’t watching, like the video not working in the browser? Or maybe your audience isn’t finding the video? Or is there some other problem that is breaking or prohibiting the playback?

Caption: In this view, you can see daily views over the last 90 days.

Beyond seeing the number of plays, that are hopefully growing, what else should you look for? Are you expecting to hit a certain threshold but not seeing enough Plays?  (e.g., 30 people were registered to go through the course, but the video play number is only seven.) On the other hand, are you seeing a higher number of views than anticipated? It could mean people are watching multiple times, or the video draws in unanticipated viewers.

It’s hard to dig into specifics with video Plays. We’ll look at other data points that can paint a better picture of what is happening. However, remember it’s the easiest data to get and provides some information worth knowing.

UNIQUE PLAYS

The Unique Plays of a video is another data point that most services provide. First, a definition. A Play counts every time the video play button is pushed, including Plays by the same person. Unique Plays filters out repeat video playbacks. This means that if I play a video twice, the Play amount is two, and the Unique Play amount is one. You should know that a viewer who watches the video once from their laptop and once from their phone may count as two Unique Plays, depending on your video host’s capability to determine if a viewer is the same. You should check with your video host’s documentation to better understand what’s counted as a Unique Play.

Caption: You can see daily Unique Views over the last 90 days.

Knowing if your audience is watching a video multiple times can lead you to some questions and insights into the effectiveness of your video. If your viewers are watching your video more than once:

• Is it because it’s interesting, enjoyable, and/or useful?
• Was there confusion that required multiple viewings?
• Is the video being used as a job aid or reference?

To answer these questions, you may need to look at other metrics and data, or you may need to ask your audience. However, these are great questions to find answers to. If someone is confused, it might mean that the video isn’t effective, or the audience isn’t prepared for the presented information. Or, if the video is being used as a job aid and referred back to often, is video the best way to provide the information?

Unique Plays may not help you completely understand the impact your video is having. It can, however, help give you context about your viewer’s behaviors and lead to other interesting and important questions.

WATCH TIME

Another common statistic most video hosts provide is the video Watch Time. The Watch Time is a cumulative amount of how much your video has been seen by its audience. For instance, if 10 viewers each watch one minute of a video, the watch time would be 10 minutes. If one viewer of the 10 only watched 30 seconds, the watch time would be nine minutes and 30 seconds.

From this, you can see if the viewers are watching the majority of your video. When you use Watch Time with other stats, like Plays, you can start to paint a picture of what your viewers are doing when watching your videos. You also may see if viewers from specific traffic locations (i.e., Social media, website, advertisements, etc…) are garnering more attention and longer Watch Times. If one particular audience segment is watching more than others, you could make some decisions on future content. You can adjust your video to accommodate those not watching or lean in and support those who are watching by creating more content like what’s being viewed.

 Caption: Number of minutes the video has been played back.

Make sure that as you look at metrics like Watch Time, you consider them within the context of all the other metrics available. All available data will help you make better decisions about what viewers are doing and if any changes are needed to ensure viewers get what they need from your videos.

AUDIENCE RETENTION/ENGAGEMENT

One extremely useful stat is Audience Retention or Audience Engagement. Audience Retention is often depicted as a line graph that goes up as more people watch that part of the video and less as fewer people watch the video. So, if multiple people watch or rewatch the first 10 seconds of the video, the line will trend upward or be high up on the graph. As fewer people watch, the line will drop. The Retention graphic can trend up mid-video if multiple viewers rewatch a section of the video.

You can gain a couple of insights from the Audience Retention graph. One is a quick comparison of how many people started watching versus how many viewers were still watching at the end of the video.

Another takeaway is to see if retention spiked or went up at any place within the video. You’ll also want to look for changes that cause viewership to drop. In both cases, maybe something funny happened that caused people to watch that part multiple times. Or perhaps something visually interesting caught their attention? On the other hand, that part of the video could have moved too quickly or was confusing, causing viewers to watch it again to understand the information. Look for anything that might cause a change, and ask, “What happens at this point that would cause viewers to decide to take this action”?

The Retention Graph won’t tell you why viewers behaved the way they did; it can only tell you what they did. It will be up to you to interpret the data and determine why it happened. This may take some investigation, including watching the video to see if you notice anything that causes the behavior. You may need to survey or talk with your audience. Or you may need to take your best guess and determine if you need to change the video to help your audience better.

OTHER VIDEO METRICS

Other video metrics may be available, but your access will depend on your video host. These additional metrics will help paint a bigger picture of your audience and their behaviors and help you create better videos or adjust the ones you have.

DEMOGRAPHICS:

Demographics are data about the viewers, like age range, gender, socioeconomic status, or other information collected by the video host site. This information isn’t widely available on every platform and can sometimes be inaccurate, but when available, it can give you insight about who is viewing your video. Typically provided from sites like YouTube, demographic data may help you determine if you are generally hitting the right people or missing key segments. If you’re working with an internal audience in your organization, you may have access to this information through other means. 

TRAFFIC SOURCES:

This metric allows you to see where your video was watched. If your video only lives in one place, this isn’t helpful. However, if you’ve posted the video in multiple places, you should be able to see which are giving you the most views. Knowing the source will help you determine if the effort is worth posting in multiple locations. Later on, this data can also help identify locations where the video needs to be removed or replaced when the content is updated or determined to be outdated.

Caption: Traffic sources data may include data to help you know where viewers are watching.

VIEWER GEOGRAPHY:

See where your viewers have watched your video from, usually to a city, state, or country level. If your video is part of internal training, or for use inside an organization, this information may not be helpful. However, the benefit of looking at geography for public videos is that it can help you to understand your audience better.

Do you have a high viewership in a particular area of the world? Is your language frequently spoken in that area? If not, you may decide to make accommodations to support those viewers, like adding captions, translations, or other information. In addition to finding opportunities to better connect with your audience, geographic data may indicate that you aren’t reaching an intended audience as expected. As you consider Viewer Geography, keep in mind where you expected your highest levels of viewership. If the data doesn’t meet your expectations, determine if changes to your videos are needed. Or, perhaps, your expectations may have been off-target and need to be adjusted as you better understand more about your viewers.

PLAYBACK LOCATIONS:

Find out where viewers are watching your videos on the web. This includes websites, social media sites, advertisements, and more. The amount of Playback Location data is highly dependent on your video host and may be limited.

Playback Location data can also help determine how viewers are finding your videos. This is valuable when sharing a video in multiple locations to reach a wider audience and see which site gives you the best viewership.

Last Considerations

There are other data points and measurements to look at when evaluating your videos, but the core few discussed should give you a good starting point. Remember that any single point of data doesn’t tell the entire story. A high metric can sometimes be good or bad, depending on the cause. You should look across your data to fully understand your audience, their behaviors, and if you need to make changes to your videos. A Few Resources About Measurement As a trainer or instructional designer, not every course will be video-based. It’s important to learn about using data to help you understand the impact your training is having. Here are a few resources that you may find helpful as you learn more about measurement.

Research about Video Viewer Preferences (2024)

Bob Pike	Bob Pike, CPLP Fellow, CSP, CPAE Speakers Hall of Fame, is known globally as the “trainer’s trainer.” He has written more than 30 books and created a dozen video systems on designing and delivering training. His Master Trainer’s Handbook is the bestselling train-the-trainer book ever published, with more than 333,000 copies in print in four editions. He has designed more than 600 training programs of one day or longer since 1969.
Brent Schlenker	Brent Schlenker is Community Director for dominKnow Learning Systems. Over the last 25 years he’s built and re-built training organizations developing innovative training strategies and solutions for businesses of all sizes. He supports learning cultures through communities and hosting live-streaming shows like Instructional Designers in Offices Drinking Coffee.
Cara North	Cara North is an award-winning Senior Learning Experience Designer at The Ohio State University. As part of her role, she creates online courses and facilitates workshops for the College of Education and Human Ecology. She serves as President of Central Ohio ATD and is also a Ph.D. student in Learning Technologies at The Ohio State University. Cara was previously an instructional designer at Amazon where she created learning materials for Amazon Kindle.
Craig Sybert	Craig is a lifelong trainer and has been in the Learning and Development field for over 15 years. His degree is in fine arts with a focus on graphic design. He is an instructional designer and developer and handles all things in between including video, audio, voice-over work, programming, and storyboarding. Currently, he is an eLearning specialist with Envolvemedia where he specializes in training Adobe Connect and Adobe Captivate Prime, while still creating fun and interactive courses for clients.
Cyndi Laurin	Cyndi (Crother) Laurin, Ph.D. has 25+ years of experience in facilitating learning and organizational development, specializing in employee engagement and performance/operational initiatives. She is an author, speaker, general problem solver, and Certified LEGO (R) Serious Play Facilitator. She has a passion for people and performance improvement solutions to advance organizational effectiveness.
Debbie Richards	Debbie Richards, president of Creative Interactive Ideas, supports talent development clients with technology challenges. She updates her job knowledge by researching new technologies and software products; participating in educational opportunities (such as speaking at learning conferences); and participating in professional organizations. Debbie was named one of the most influential people in corporate e-learning for 2018.
Joe Ganci	Joe Ganci is President of eLearningJoe, LLC, a custom learning company located outside Washington. D.C. Since 1983 he has been involved in every aspect of multimedia and learning development. Joe holds a degree in Computer Science and writes books and articles about eLearning. He is widely considered a guru for his expertise in eLearning development and technology, and he consults with clients worldwide, creating eLearning modules and templates, often training personnel in their use and then making himself available to assist if necessary.
Kati Ryan	Expanding her side hustle into a thriving and profitable business in less than a year, Kati founded A Positive Adventure, a learning and development consulting firm. As a San Francisco-based founder, Kati more than tripled the company’s revenue YoY. Kati has built world-class, award-winning training programs for companies like Instacart, Marine Layer, Bill.com, Gannett, LivingSocial, and others.
Kevin Thorn	Kevin is an award-winning elearning designer & developer, consultant, and owner of NuggetHead Studioz, LLC., a boutique custom design and development studio specializing in online learning experiences. After retiring from the U.S. Army, Kevin pursued a career in corporate IT and Training & Development. With his combined military and industry experience, Kevin started the Studioz in 2012 working with clients in various industries solving problems in a wide range of creative projects.
Sam Rogers	Sam Rogers creatively bridges the gaps between technology and people, between information and actual learning. He’s passionately curious, and cares deeply about finding real-world solutions people will use. After twenty years as a Senior Instructional Designer, eLearning Developer, Video eLearning Producer, LMS Integrations Specialist, ILT Facilitator, and Learning Ecosystems Architect, Sam brings an integrated perspective to all things Learning & Development.
Shannon Tipton	As Chief Learning Rebel, Shannon Tipton is a skilled learning strategist and sought out international speaker focusing on learning disruption. Shannon has over 20 years of learning leadership experience developing successful learning infrastructures for training departments within organizations in North America, Europe, and China. Recognized as bringing real-world expertise into the learning field, Shannon develops frameworks and tools to support learning technologies, microlearning and learning reinforcement to ensure success for solving business pain points and learning initiatives.
Tim Slade	Tim Slade is a speaker, author, and award-winning freelance eLearning designer. Having spent the last decade working to help others elevate their eLearning and visual communications content, Tim has been recognized and awarded within the eLearning industry multiple times for his creative and innovative design aesthetics. Tim is a regular speaker at international eLearning conferences and author of The eLearning Designer’s Handbook.
Toddi Norum	Toddi is a passionate but “accidental” learning developer who is currently managing the corporate Learning Management System (Skilljar) for Geotab, a data and telematics company.In the past, she spent three years as the lead trainer for the U.S. Army SUGV robotics training program. Toddi says that she has lived a “charmed life” and has pursued many passions: writing, jazz music, photography, and pet rescue.
Trish Uhl	Trish Uhl, creator of the Learning Systems Engineering Framework™ and founder of the Talent & Learning Analytics Leadership Forum, works globally with learning leaders and business executives on transforming their organizations into Digital Workplaces and developing their people’s digital fluency and dexterity. Trish’s unique skill set combines data science, learning science, advanced analytics, artificial intelligence (AI), predictive modeling, ambient intelligence, voice UX, blockchain and emerging tech to promote positive people impact and deliver organizational performance.