MakingBetter

Strategy. Ecosystems. Meaning.

Experience API

Level Up Learning Data with xAPI in 2021

By

You’re likely going to be reading up seriously on how to work with learning data sometime in 2021. This will be true regardless of whether you work in online K-12, Higher Education or professional training — anywhere in the world. If you will “own” xAPI know-how on your team (or maybe for the whole organization), I want to help you.

Megan and I have certainly been in your shoes.

So, I made a resolution for 2021 – I’m going to try and blog daily again like I did maybe 15 years ago about Flash For eLearning. It’s time for us all to level our data game up.

First, let’s start with this webinar (above), where I recently demonstrated a new tool coming in Q2 from ADL, the xAPI Profile Server we produced for ADL with our friends from Veracity. In the video, I explain what this is, what problems it’s going to help with and how you’ll likely put it to work. Spoiler: it’s a tool to help you author and serve xAPI Profiles, doing a lot of the heavy lifting of complex rules thingies for you.

Next, I’ll blog about how to frame a problem space to apply learning analytics with dimensions.

The One-Year Countdown to an xAPI Profile Server

By

tl;dr: Wednesday, 20 November at 1pm EST, the Advanced Distributed Learning (ADL) Initiative will host a webinar on an xAPI Profile Publishing Tool & Server being developed for 2020. 

Since 2017, Megan & I have been pretty heads-down, laser-focused on doing what we can to harden our best practices to work with xAPI to further support ADL’s mission. We researched and documented conformance requirements for learning record stores (LRSs). We gathered best practices and worked with deep subject matter experts to develop a companion spec to xAPI (xAPI Profiles). We gathered the requirements for an xAPI Profile Server capable of publishing, validating and serving (via an API) xAPI Profiles. Now, thanks to ADL, we’re designing and developing a reference xAPI Profile Server with our friends (and former colleagues) at Veracity Technology Consultants!

How Will an xAPI Profile Help L&D Capabilities Scale?

Depending on how removed you are from touching anything that looks like code, if you’re reading this, you’re likely to have some idea of how much non-value-added work goes into how  L&D content and services are delivered today. Software integrations are difficult and almost always custom. Testing eLearning in LMSs (specifically SCORM or AICC) is manual and laborious. Reporting anything other than completions and scores from your LMS can seem impossible. Even if you’re managing content, you’re probably not managing what given learning content tracks, let alone reports to the LMS. This state of eLearning has stagnated over 10 years, which resulted in critical processes that must still be rigid (and often fragile) to author, publish, test, deliver and revise eLearning content everywhere.

xAPI Profiles make it possible to exercise some control for the quality and consistency of learning data generated by related learning activities. xAPI Profiles as specified today directly address how verbs, activity types, attachment types and extensions can be governed by xAPI Profiles. The xAPI Profile Server will provide a wizard-like interface to make authoring and publishing. The xAPI Profile Server will support publishing of the JSON-LD xAPI Profile so applicable technical rules that should govern any subscribing learning experience can be managed together with one tool. Users will be able to validate their xAPI Profiles. The application will serve xAPI Profiles via an API enabling workflows integrations with the means to revise and push changes in xAPI Profiles to subscribing systems and services.

On Wednesday, 20 November, Jason Haag of Veracity Technical Consultants and I will co-present on the work our teams are working on together. We’ll talk about some ins and outs of what you can do with xAPI Profiles, and some more details on what to expect from the xAPI Profile Server. Sign up for the webinar: https://attendee.gotowebinar.com/register/7745952068399329549

xAPI Camp Preview: dominKnow ONE

By

On March 26, 2019, I’m gathering best examples of the tools that enable, and the professionals who define xAPI’s best practices at xAPI Camp (number 14!) hosted by the eLearning Guild at the Learning Solutions conference in Orlando, FL. In the run-up to the event, I’m highlighting these industry colleagues, and why I think they’re enabling and defining best practices. Today’s post focuses on what dominKnow is doing for savvy instructional designers with their ONE platform, with different authoring modes for different types of content authoring.

If you have some questions about the learner, the learning experience or the efficacy of the learning program, there’s some impressive flexibility in dominKnow ONE I want to point out.

About that Competency Dashboard in RISC…

In my last post, I shared a competency reporting dashboard in RISC’s VTA. RISC’s dashboard (like many useful dashboards) relies on specific, well-formed data. Two companies offer such flexibility to customize what and how you track learners engaging with their authoring tools: Trivantis and dominKnow.

For, seriously, a little extra effort, generating custom statements that conform with profiles or just follow best practices offers so many more valuable insights. In what’s been demonstrated previously, much of the workflow to track to competencies or learning objectives is even automated, meaning as a content author, you wouldn’t have to “code” so much as keep content organized… which you’d want to do anyway as just good instructional design. dominKnow ONE by default, and with no programming, provides a hierarchy and tracking of Course > Module > Learning Object, in which the “Learning Object” contains the content and interactions that, together, would support meeting a particular learning objective or competency. When a learner satisfies the requirements to complete the learning object, the dashboard will show that learner has met the competency requirement(s).

Taking it to the next level and optimizing an individual’s learning

You probably already know that xAPI allows us to track all sorts of information, including the aforementioned objectives/competencies. The data can be leveraged by content, though — a capability that many rapid authoring tools just don’t leverage. At next week’s xAPI Camp at Learning Solutions, dominKnow will demonstrate how you can track author-defined competencies in your content and then use the xAPI data you tracked to dynamically personalize the learner’s content. If a learner has already demonstrated/completed a given competency, why should they be forced to re-demonstrate it or an author need to create unique learning content for each user use case? Not only is it cool — it’s interoperable and dominKnow will demonstrate that while doing this requires instructional design skills, no coding skills are required.

Why is this good?

Out of the box, dominKnow is making it easy to organize content without locking you into this default way of organizing your content, leveraging that to personalize your learning, and enabling you to more easily tie it to your company’s objectives. That’s exactly the flexibility and ease I want to track and report out with xAPI. For most content I need to work with, the 1:1 Learning Object:Competency Objective relationship works, but if I have something more complex, I can take dominKnow ONE down the rabbit hole. As much of xAPI’s best practices require tinkering, the more flexible tools I have available to me, the more use I have for them.

xAPI Camp Preview: RISC VTA

By

On March 26, 2019, I’m gathering some of the best tools that enable, and the professionals who define, xAPI’s best practices. This will happen at the upcoming xAPI Camp (number 14!) hosted by the eLearning Guild at the Learning Solutions conference in Orlando, FL. In the run-up to the event, I aim to highlight who’s going to be there, and why I think they’re enabling and defining best practices. Today’s post focuses on the outstanding capabilities RISC has built into its Virtual Training Assistant (VTA) learning management system that enable flexible training approaches, like spaced learning.

When Knowing Matters

A question I get asked regularly is “when is xAPI really worth doing?” and my answer is always something like “xAPI is worth doing when the stakes are high and the cost of failure is too heavy.” When it comes to safety and health — when lives are at stake — that’s when a deep analysis with xAPI is worth the effort. RISC’s VTA is geared for high-stakes compliance needs. If your workforce is on a gas or oil pipeline and you care that they can do the job and not hurt themselves or others in the process, then knowing the capabilities of a given worker and the team(s) they’re on matters.

My funk soul brother Duncan Welder goes into detail on how RISC approaches tracking competencies in VTA here. It’s worth the read and to see the dashboard in action, but I can tell a lot about what they’re doing in terms of how they’re using xAPI by what I see here.

Why is this good?

Checklists, Competency Assessments, Questions, Pass/Fail activities — these are different types of xAPI activities VTA is reporting on. Using xAPI to also detail incident reporting makes it possible to juxtapose the learning activities with real-world outcomes to demonstrate the correlations between learning performance and job performance. There is so much depth we can get into with this established, as RISC can continue to get more granular with the progress toward specific competencies and the incident trends over time.

That’s exactly what I want to track and report out with xAPI. From what I’ve seen in the market, there are some really good LRSs providing solid reporting on data when there’s no assumption about what’s actually in the data. RISC, however, has a history of designing reports that tells specific stories about specific learning activities, like their PDF Annotator and Message Board features. What RISC can report may not be for everyone, but if you want specific insights, there’s no other vendor on the market that’s able to give you these insights with this fidelity.

Andrew Downes, in his stunning blog series documenting just what exactly customers are tracking with Watershed, breaks down a high level categorization of learning analytics distinguishing the learner, the learning experience and the learning program. Downes cites that Wateshed customers learning analytics are about the learning program, and ery little attention is tracked by Watershed customers about the learning experience or the learners themselves. In that light, it’s even more impressive to me that RISC is tracking competencies in this way because performance-to-competency is a dimension that sheds light not just on the learning program but the learner and the learning experience, too.

How does this work with “spaced learning?”

When Duncan and Art Werkenthin present at xAPI Camp on March 26, they’re going to focus on spaced learning — the delivery of small, repetitive pieces of learning content with breaks between the learner’s sessions with the content. It’s a researched approach to learning content delivery that is particularly effective for long-term retention. There are a lot of technical things RISC is doing with xAPI (like relying on cmi5 for reliably handling the launch of xAPI-enabled learning content). VTA’s support for cmi5 reduces the variability of the learning experience and technically enables a spaced learning approach (among enabling lots of other learning experiences). The reality is that RISC’s implementation of cmi5 makes a lot of things possible where we need to know who the learner is, and we want to track learning activity to an LMS.

And, with RISC able to track a spaced learning experience, they can look at the data on the content delivered over time, against the performance outcomes (like incidents reported) and optimize the spacing relative to the type of content, the subject of the content, the complexity of the job performance, etc. VTA clearly gives you all the tools you need to do this. All you need is content that uses xAPI in specific ways to populate these reports with data that is formatted to deliver insights. After all, I’m highlighting what RISC can report, but that requires data that is useful and usable to populate these reports with appropriately formatted data.

For that, I can’t wait to share my next blog post on what DominKnow is doing that enables what RISC (and other tools) can report on.

Interested in xAPI visualizations but can’t make xAPI Camp?  Duncan and Art have a session Wednesday, March 27th at Learning Solutions on getting measurable results with xAPI titled Design with the End In Mind.

xAPI Camp DevLearn Presentations

By

Here are links to all the presentations from xAPI Camp DevLearn. Enjoy!

iFest and xAPI Profile Servers

By

We’ve been hard at work the last few months identifying what an xAPI Profile Server should do beyond the basic requirements in the xAPI Profile Specification as part of a BAA with ADL. Last week we took the show on the road to iFest in Washington DC. We brought along a poster which explained why a profile server is necessary and how it will improve interoperability. That poster won a people’s choice award for best narrative!

(evidence^)

That’s why I’m writing this post, because we have never won a poster award. In fact, there are a lot of awards we haven’t won. And we’re okay with that. Since this one was a people’s choice award, we thought more people might like to see it. So, here it is…

 

Another post is coming with a summary of the profile server research. Big thanks to ADL for a great event and thanks to our designer, Jason, for a prize poster!

Filling in xAPI’s Most Important Gaps

By

2017 is a year that is really testing our capacities and limits. Politics, natural disasters, cyber security threats, budgets… even ADL has hit some road bumps. We formed the Data Interoperability Standards Consortium (DISC) a few years ago specifically to hedge against disruptions to ADL’s capacity for spec stewardship so that the people who rely most on xAPI wouldn’t be held back the way the learning industry was with SCORM 2004 (after ADL stopped innovating on it… in 2004). Anyone who’s worked in or with a government for more than a few years knows to expect that there are these hiccups which, for an indeterminate amount of time, stall progress.

Industry and momentum, let alone organizational strategies, shouldn’t be subject to the whim of such hiccups — especially when the industry could address these on its own.

Working hand-in-hand with ADL over the last year, DISC and the xAPI Community hit some pretty major milestones that really stabilized xAPI, in terms of implementation, to enable more adoption, faster. Also with ADL, we identified a series of priorities to work on together over the next several years. ADL’s funding situation means that those plans would have to be put on hold for us to work contractually with ADL. However, that (realistically) could be almost a full year in waiting to get started. As an advocate for its progress, I’d rather we work on these things in a different way, not needing to rely on ADL, so that when ADL’s budget capacity is restored, they can advance things much further afield. As an industry that depends on xAPI, it’s in the xAPI Community’s best interests to keep things going.

Megan and I are sharing the priorities for open source development and documentation we worked out with ADL with the hope that the xAPI Community will seize this moment for the opportunity it presents: to chart its own path to growth and ubiquity.

  1. Stand up a xAPI Profile server. With the xAPI Profiles specification released in June, the xAPI Community has finally addressed a huge gap in achieving semantic interoperability. However, without even a reference implementation of an xAPI Profile Server, the spec will never see its potential to scale xAPI across industries, and we will continue to have challenges of semantic interoperability across implementations. There’s a minimal development effort as a lot of the xAPI Profile Server leverages existing JSON-LD tools.
  2. Stand up a service to test for valid, well-structured xAPI Profiles. Right now the only way to create xAPI Profiles in JSON-LD format is to do so by hand, which is a laborious process that is prone to manual errors. It’d be helpful to have something available in lieu of tools that validated an xAPI Profile, highlighted where the errors are and the nature of them, so that the people producing xAPI Profiles could do so easily, and so those xAPI Profiles could be used by others with confidence. This likely requires a bit more development than the xAPI Profile Server requires.
  3. Stand up a tool to help people publish xAPI Profiles in proper JSON-LD format. What would be even better than producing xAPI Profiles by hand would be a web-based application with an interface that made it much easier for people to create valid xAPI Profiles, so that subject matter experts (or, more likely, data people working with subject matter experts) could generate profiles without needing to encode JSON-LD themselves. Not only does this require some development savvy — it would benefit from having a product manager and interaction designer so that this could be both usable and useful to many, thus encouraging more organizations to generate and share xAPI Profiles.
  4. Develop conformance requirements and a conformance test for xAPI Profiles and the xAPI Profile Server. Much like DISC facilitated with the xAPI Community in 2016, it’s likely that many LRS vendors (and probably authoring tool vendors as well) will incorporate an xAPI Profile Server and want to validate the data collected in their LRSs against valid xAPI Profiles. It would be wise to get ahead of inevitable interoperability challenges and develop (first) conformance requirements and (later) conformance tests that could ensure consistency in what we deem to be “conformant.” Related — if it’s a common goal by LRS vendors and stakeholders that LRSs will validate statements against profiles, that should be made explicit in the xAPI specification and additional LRS Conformance requirements must be developed, as well as additional unit tests in the LRS Conformance Test Suite.

That’s a solid list, and the community could rally itself to take this on and commit to seeing it done in the next twelve months. The organizations willing to commit resources to develop these efforts to a defined set of outcomes should be embraced by the community.

The things listed above are needed now. The market can probably tolerate a year without it impeding xAPI adoption, so long as these things are available to the market and implemented as organizations budget for FY19. There’s more for to do in the coming twelve months, but in the general category of making things, these are things with definitive outcomes that would help everyone.

So… who’s going to take on what in filling in these pretty important gaps?

Tim?

With Gratitude…

By

Megan and I find ourselves grateful, as the year comes to a close. 2015 gave MakingBetter an amazing journey that was full of surprises. Most were wonderful and some were very scary. Through it all, we found ways to make our clients happy and successful doing work we believed in. When we had struggles and found roadblocks, we worked together to get over whatever the hump was and we got to a better place.

With Gratitude

We created things with great people. Our projects ranged from designing and developing custom reporting for software products, training providers and large enterprises. We launched an online journal, the xAPI Quarterly, kickstarting the publishing arm of our business, Connections Forum, and we ran our first events, xAPI Camps that each were co-created with our participants. In January 2015, we planned for one and as the year closed we had four with five more scheduled in 2016. Our next one at the Autodesk Galleria in downtown San Francisco, February 11. We celebrated another amazing year with the amazing community, Up to All of Us, which will convene again in Sonoma County, February 12-15. We started a non-profit. More on that in a minute.

Grateful to Make MakingBetter Happen

MakingBetter at the Grand Tetons

Megan came on full-time with MakingBetter in June, this year. We took our first serious vacation ever in July. We spent a lot of time with our families and friends. We lived and worked, together, on our own terms for the first time in our lives. We dealt with emergencies and surprise medical concerns. We innovated when we needed to and we stuck to tried and true processes when we needed to, too. We lived well in 2015. I say all this because it’s important to celebrate success and to make sure that credit goes where credit is due. I write tonight grateful for a true partner like Megan, grateful for each and every client we had this year, grateful for each and every person who’s influenced how we do what we do, grateful for our sponsors and our partners and especially…

Grateful for the xAPI Community

It’s the xAPI community I want to talk specifically to now. There are a myriad of reasons why 2015 was good for Megan and I, but the one reason that stands out is the incredible gains in xAPI’s adoption that happened this year. We know there’s been incredible growth in xAPI adoption. Our business boomed and so did that of many software vendors who create solutions that are tailored to meet some of the many things people use xAPI for. We know projects are already being planned for the beginning of 2016 at a scale that equals the whole of xAPI adoption in 2015. These are measureable outcomes of an open source community that has been lovingly and painstakingly attended by the US Department of Defense and its particular initiative, Advanced Distributed Learning. xAPI is in every way a stunning success. It is proving that open government practices, a pro-entrepreneurial approach and an authentic embrace of open source can stimulate innovation, enable implemented approaches to complex and serious challenges, and catalyze economic opportunities. It is far from the applied research and development activity it was four years ago. It is a mature spec that is growing its own industry.

Grateful for xAPI’s Growth

xAPI Camp - DevLearn

xAPI is so successful that it’s actually becoming a challenge for ADL to support it to the scale it now demands. The Design Cohort program that began in 2013 became so well attended and populated that it couldn’t be supported by ADL anymore — they just don’t have the resources to do it on their own. The maintenance of the spec is labor intensive enough for the resources ADL has, that certification isn’t something they can handle on their own without stopping something else important. When SCORM was being created, it was an epoch ago for information and instructional technology, and ADL had over 40 engineers they could apply to SCORM alone. ADL now has dozens of high priority projects and there are maybe six full-time engineers they can resource for xAPI. Fortunately, those of us who brought to ADL the concepts that enabled xAPI’s creation knew that the day would come when specs and standards would need to move beyond ADL to truly mature. This is why open source was so crucial a path for xAPI. It’s because xAPI is licensed Apache 2.0 that anyone can take xAPI and mature it, and that’s just what we’re about to do, given ADL’s blessing and commitment to participate in the effort.

Grateful to Serve Our Community

The non-profit we started at the close of 2015 is the Data Interoperability Standards Consortium, or DISC (because, acronyms). There are many challenges to working with data: interoperability, security, privacy, professional competencies, validation, provenance, ethics, legalities, languages, formats, etc. We intend for DISC to offer the table where all communities of practice, individuals, organizations, governments and industries can work together to meet the complexities of working with data. It’s about more than xAPI, but make no mistake, xAPI is our priority in 2016. The transition from an ADL-organized xAPI Community to a DISC-organized xAPI Community will begin in the first quarter of 2016. By the middle of the year, we’ll have established working groups and special interest groups to explore ways in which xAPI may be extended as well as certification requirements. By the beginning of 2017, we’ll have a certification program in place and an array of tools that will make working with xAPI’s vocabulary much much easier.

That seems like a lot to get done in one year, and you’re right. It is a lot. But it will be done because it has to be done. xAPI is growing so much that if we don’t have certification in place by the end of 2016, we risk xAPI’s long term future. We predict a massive catalyst for international adoption to emerge by the end of 2016 in the way of procurement requirements for governments around xAPI, because having data that everyone can understand and can make use of is in the interest of public good institutions. When governments are a year away from requiring xAPI support and certified products are all that will be purchased, it makes right now the very moment where xAPI goes big. It is exciting, frightening and uncertain – and it’s fun, and this is what it’s like for us to be so fully invested when the stakes are this high. The fact that the stakes for xAPI are this high should be the reassurance everyone needs that xAPI really is a big deal and it’s worth our sweat to invest in its growth right now.

Grateful for a Future We Can Forge Together

Seattle Gathering

Because xAPI is open source, and because xAPI will have an organization that is focused expressly on its maturity, it’s going to get the chance to grow in a way that no learning technology has ever had the chance to do. Megan and I are proud to have an incredible team on DISC’s Board of Directors from around the world who represent years of extraordinary work in leading professional organizations, the science of learning analytics, the development of industry organizations, professional practice and xAPI itself. Very soon, we’ll announce our founding Board of Directors and post our by-laws, our 2016 goals and objectives and we’ll open membership. xAPI will forever be Apache 2.0 and we intend to ensure that it remain open source and cared for by an open community as long as it remains relevant. The organization we’re creating will finally structure how decisions about it are made, balancing the needs of those most invested in xAPI requirements with the needs of those most impacted by xAPI applications. Without the burdens and caveats that come with moving this activity into large spec and standards groups, as a community and an industry with many verticals, we can design our own future with xAPI.

Grateful for Your Help In What Comes Next

There’s been only a few sketchy roadmaps for what Megan and I have been doing together as MakingBetter. There are even fewer notes on what we’re about to do with DISC in forming an industry organization to support a major open source project with the cooperation of its stewards in the US DoD. But, this isn’t the first time Megan and I have had to work with a community to create something that didn’t previously exist. We’ve done it with Up to All of Us. We did it with growing xAPI into a fully realized community of designers, developers, content and data wranglers. We did it with figuring out how to fit open source for US government. And now we’re going to figure out, with the full interest by and for the community, how we grow the industry and professional practice around xAPI. It will require paying members and continued open community participation. It will require a level of dedication, enthusiasm and grit that hasn’t been demanded yet. Given all that, I’ve never been more confident in our abilities, all of us together, to figure this out. We’ve been able to plan and go off-plan and get this far. It stands to reason we’re going to go a lot farther together.

Megan and I are staking our business on xAPI. We’re staking our families on xAPI. We’re committing our lives over the next couple of years to the community and industry around xAPI and we are grateful to do so.

We wish you all the best for this holiday season and for the new year to come. We’ve loved hanging with you. We’ve loved working with you. We’ve learned so much in doing so and we can’t wait for the next level shit about to happen!

More soon!