Innovator Spotlight: Advancing targeted learning with technology

Technology continues to transform jobs across industries. Educators and institutions on campuses and online, in postsecondary institutions and workforce training, are challenged to keep up and stay relevant.

With advancements in learning science and digital tools, and more opportunities to provide learners with better, more personalized learning, access to improved and more contextual and applied online learning experiences should quickly be becoming ubiquitous. As has been made obvious with the COVID-19 pandemic, we clearly are far from this reality in providing high-quality, digital learning opportunities for all. The digital divide has been laid bare, as has the absence of online teacher training and use of digital learning across K-12 and higher education. Equipping our people with technology, broadband, and digital learning opportunities has never been more important. All industries must keep their IT staff, and thus their companies, competitive. Can we look to the upskilling of IT professionals for a case study for where learning may be headed?

Staying on top of skills development in technology is an ever-evolving art and science. To stay relevant and competitive, companies must engage their employees in continual upskilling and reskilling. Training programs must teach the skills that employers need—and forecast those needs into the future—while providing engaging, effective instruction. This approach to learning requires thoughtful, learner-centered instructional design coupled with AI.

Pluralsight is a technology company that delivers skills training to all types of technologists at the beginner, intermediate, and advanced levels to keep IT departments across industries skilled-up. Each learner begins with a skill IQ assessment to generate a baseline of their knowledge in a particular technology. Someone who has a set of skills in a certain tech stack or programming language in one area, for example, could discover a path to develop adjacent skills to solve particular problems or to help  evolve an organization’s tech strategy. Assessments and courses can be accessed by individuals or commissioned by companies, and used to prepare learners for on-the-job problem-solving,  industry certifications, or more customized programs.

Bringing together subject matter expertise and user-experience design, Pluralsight is building an AI-driven approach to training that gets learners on a shorter path to acquiring critical skills.

To better understand the role of instructional design and AI in supporting the constant and evolving tech skills in demand across industries, we interviewed Senior Director of Instructional Design at Pluralsight, Dr. Angela Payne.

Q: What is Pluralsight’s approach to instruction, and where is your company heading?

Dr. Angela Payne: Our market research and curriculum teams analyze what skills are most in demand and how the field is evolving, so we are constantly prioritizing the instruction we create in collaboration with subject matter experts. We integrate different types of activities around our content that are linked to specific outcomes and learning objectives, which are tied to assessments. That is where the baseline skill IQ’s get fueled.

A course may consist of a set of modules and clips. But what we’re considering is a more personalized learning approach based on an individual instructional object. This allows us to create and combine a variety of personalized instructional activities organized in different sequences to provide different types of learning experiences for each person.

Q. Take me through the learner journey on Pluralsight.  

Payne: I focus on designing impactful learning experiences, and, to me, that means having an empathetic focus on learner needs. Though our company is highly content-focused, and we have a huge suite of expert authors who create instruction, our main focus is on creating engaging and impactful learning to drive skill development.

For the individual, motivation is a key aspect of success for adult learners. You have to have a key outcome goal and clear visibility for how you’re going to get there. As you dedicate time to developing skills and learning, you’re getting what you need specifically to apply to your work — like in an apprenticeship.

In our curriculum planning, we use a roles and skills approach. We do an analysis of specific roles that exist in the marketplace and clearly understand the hiring manager’s demand for those skills, including how they craft the job description and conduct interviews. We also survey experts in that role to understand the job tasks that are completed every day and at what level of priority they’re completed or  how often a task is actually used. What parts are strategy? What parts are mindset? What parts are hands-on skills?

When someone comes to learn, they can do the assessment, see what their skill development  path options are, and know exactly what they need to do to show proficiency in that area to increase their confidence that the market needs that skill and they are prepared to contribute.

Q. Can you explain how you use assessment and measurement to improve the learning experience?

Payne: Our team measures learning effectiveness and impact. This creates a feedback loop that facilitates ongoing iteration and improvement on instruction and our AI engine. We can mix and match learning recommendations and drive it off of data using machine learning and other AI smarts. You have to have this set of data so that the machine can learn — the modular learning objective is the heart of it.

We want to know if customers are satisfied with the experience, and, on the back end, we assess how efficient and effective the instruction is. We have a full set of mixed methods metrics and we have this giant wishlist of all the metrics that we really want to create and use. Our customers’ main need is to measure the impact of skill application in a given role or the organization. We’re still working on the best ways to measure and report on these metrics as accurately as possible.

We also look at behavior, where we can collect quantitative metrics on the platform of what learners did. Where did they click? How long did they spend watching certain videos? What were their skill IQ scores? Then for learning effectiveness, we have an initial formula, so to speak, of how to combine these qualitative and quantitative metrics into effectiveness scores. What does it look like to have an impactful learning experience? And how can we tweak and improve the learning experience based on our understanding of the learners reactions and behaviors, both qualitatively and quantitatively?

Q: How are learners able to demonstrate what they know and can do once they’ve completed a course or path? 

Payne: We have recently released several evolutions to our experience, including Skill IQ retesting that moves learners beyond their baseline score to track skill development over time as their proficiency increases. We also use learning checks on the module level of each of our courses to allow learners to check their understanding and review material to resolve confusion prior to moving forward in their skill development activities. We are beginning to expand more of our skill challenges and project-based learning offerings that allow learners to solve specific professional problems designed to build expertise using feedback and a Q&A feature that supports a community of practice with peers and experts within their organization. We have also recently announced an integration of our new product called Flow with our existing product called Skills which allows teams to measure and track efficiency in delivering on technical problems. Flow provides a new level of reporting transparency aligned to how code commits on a project are delivering customer value and what the skill efficiencies and bottlenecks are to allow leaders to facilitate skill-up aligned to the organization mission, vision, and strategy much more efficiently. Also, developers each have what’s called a GitHub Profile. It works similar to a portfolio, where people can share sample code and projects as a demonstration of their skills.

We are exploring other avenues, such as the whole careering pathway aspect of our service, to connect skills to specific open jobs and help open talent doors to employers, but right now our primary strategic focus is on enterprises where we’re helping skill up the employees that are already there. A personalized approach to a skill development learning journey is part of our vision, and we’re just getting started exploring the potential to meet customer needs there.

Q: Why would someone come to a company like Pluralsight over a traditional degree program?

Payne: We are establishing the same foundations that everybody else does in order to create quality learning experiences. Where ours is different is we have engineers and some pretty incredible innovators that are trying to figure out how to automate and personalize the learning experiences. And we’re trying to reduce the friction for the learner by creating more personalized scaffolding designed for the technologist, which is always a balancing act.

One of the teams tackling this problem calls it reducing the distance to a skill. That’s how they describe efficiency. So, reducing the distance to a skill means I jump in and take some kind of a pre-assessment/skill IQ and then this system lets me move through concepts and it’s creating for me the most efficient path. So, instead of watching 20 hours of video to get to a skill, maybe I can condense this down to six hours of a little bit more interactive practice. Where we’re really trying to drive innovation is reducing this distance to skills so people can skill up faster and they can skill up more broadly with tightly designed hands-on activities.

It’s more of an apprenticeship model versus an information delivery model. To shorten the distance to the skill, you’ve got to apprentice the learner instead of having them inhaling a ton of information and spitting it back at you.

Q: How does Pluralsight work with businesses to identify and build the right employee skills? 

Payne: For business, one of the biggest problems we’re solving goes beyond upskilling the adult learner. We’re targeting a company’s digital transformation in the market. They have highly specialized and long-time employees that have critical insights into the business, but they must also continually innovate to keep up with disruptors. So we have to target specific training to meet their digital strategy and business goals.

Many of our large enterprise clients are companies that have been around for a long time, and a lot of them have defined the industry that they are in. They were the innovators in the past. They’re undergoing all of these changes, and so they partner with us to provide the skill training. But we also have professional services customer-success organizations that actually help them with their digital transformation strategy as well. They combine forces in a consultative way to determine what the outcome goal should be, then determine the skill-up recommendations for each organization.