MakingBetter

Strategy. Ecosystems. Meaning.

Experience API

xAPI Camp Preview: dominKnow ONE

By

On March 26, 2019, I’m gathering best examples of the tools that enable, and the professionals who define xAPI’s best practices at xAPI Camp (number 14!) hosted by the eLearning Guild at the Learning Solutions conference in Orlando, FL. In the run-up to the event, I’m highlighting these industry colleagues, and why I think they’re enabling and defining best practices. Today’s post focuses on what dominKnow is doing for savvy instructional designers with their ONE platform, with different authoring modes for different types of content authoring.

If you have some questions about the learner, the learning experience or the efficacy of the learning program, there’s some impressive flexibility in dominKnow ONE I want to point out.

About that Competency Dashboard in RISC…

In my last post, I shared a competency reporting dashboard in RISC’s VTA. RISC’s dashboard (like many useful dashboards) relies on specific, well-formed data. Two companies offer such flexibility to customize what and how you track learners engaging with their authoring tools: Trivantis and dominKnow.

For, seriously, a little extra effort, generating custom statements that conform with profiles or just follow best practices offers so many more valuable insights. In what’s been demonstrated previously, much of the workflow to track to competencies or learning objectives is even automated, meaning as a content author, you wouldn’t have to “code” so much as keep content organized… which you’d want to do anyway as just good instructional design. dominKnow ONE by default, and with no programming, provides a hierarchy and tracking of Course > Module > Learning Object, in which the “Learning Object” contains the content and interactions that, together, would support meeting a particular learning objective or competency. When a learner satisfies the requirements to complete the learning object, the dashboard will show that learner has met the competency requirement(s).

Taking it to the next level and optimizing an individual’s learning

You probably already know that xAPI allows us to track all sorts of information, including the aforementioned objectives/competencies. The data can be leveraged by content, though — a capability that many rapid authoring tools just don’t leverage. At next week’s xAPI Camp at Learning Solutions, dominKnow will demonstrate how you can track author-defined competencies in your content and then use the xAPI data you tracked to dynamically personalize the learner’s content. If a learner has already demonstrated/completed a given competency, why should they be forced to re-demonstrate it or an author need to create unique learning content for each user use case? Not only is it cool — it’s interoperable and dominKnow will demonstrate that while doing this requires instructional design skills, no coding skills are required.

Why is this good?

Out of the box, dominKnow is making it easy to organize content without locking you into this default way of organizing your content, leveraging that to personalize your learning, and enabling you to more easily tie it to your company’s objectives. That’s exactly the flexibility and ease I want to track and report out with xAPI. For most content I need to work with, the 1:1 Learning Object:Competency Objective relationship works, but if I have something more complex, I can take dominKnow ONE down the rabbit hole. As much of xAPI’s best practices require tinkering, the more flexible tools I have available to me, the more use I have for them.

xAPI Camp Preview: RISC VTA

By

On March 26, 2019, I’m gathering some of the best tools that enable, and the professionals who define, xAPI’s best practices. This will happen at the upcoming xAPI Camp (number 14!) hosted by the eLearning Guild at the Learning Solutions conference in Orlando, FL. In the run-up to the event, I aim to highlight who’s going to be there, and why I think they’re enabling and defining best practices. Today’s post focuses on the outstanding capabilities RISC has built into its Virtual Training Assistant (VTA) learning management system that enable flexible training approaches, like spaced learning.

When Knowing Matters

A question I get asked regularly is “when is xAPI really worth doing?” and my answer is always something like “xAPI is worth doing when the stakes are high and the cost of failure is too heavy.” When it comes to safety and health — when lives are at stake — that’s when a deep analysis with xAPI is worth the effort. RISC’s VTA is geared for high-stakes compliance needs. If your workforce is on a gas or oil pipeline and you care that they can do the job and not hurt themselves or others in the process, then knowing the capabilities of a given worker and the team(s) they’re on matters.

My funk soul brother Duncan Welder goes into detail on how RISC approaches tracking competencies in VTA here. It’s worth the read and to see the dashboard in action, but I can tell a lot about what they’re doing in terms of how they’re using xAPI by what I see here.

Why is this good?

Checklists, Competency Assessments, Questions, Pass/Fail activities — these are different types of xAPI activities VTA is reporting on. Using xAPI to also detail incident reporting makes it possible to juxtapose the learning activities with real-world outcomes to demonstrate the correlations between learning performance and job performance. There is so much depth we can get into with this established, as RISC can continue to get more granular with the progress toward specific competencies and the incident trends over time.

That’s exactly what I want to track and report out with xAPI. From what I’ve seen in the market, there are some really good LRSs providing solid reporting on data when there’s no assumption about what’s actually in the data. RISC, however, has a history of designing reports that tells specific stories about specific learning activities, like their PDF Annotator and Message Board features. What RISC can report may not be for everyone, but if you want specific insights, there’s no other vendor on the market that’s able to give you these insights with this fidelity.

Andrew Downes, in his stunning blog series documenting just what exactly customers are tracking with Watershed, breaks down a high level categorization of learning analytics distinguishing the learner, the learning experience and the learning program. Downes cites that Wateshed customers learning analytics are about the learning program, and ery little attention is tracked by Watershed customers about the learning experience or the learners themselves. In that light, it’s even more impressive to me that RISC is tracking competencies in this way because performance-to-competency is a dimension that sheds light not just on the learning program but the learner and the learning experience, too.

How does this work with “spaced learning?”

When Duncan and Art Werkenthin present at xAPI Camp on March 26, they’re going to focus on spaced learning — the delivery of small, repetitive pieces of learning content with breaks between the learner’s sessions with the content. It’s a researched approach to learning content delivery that is particularly effective for long-term retention. There are a lot of technical things RISC is doing with xAPI (like relying on cmi5 for reliably handling the launch of xAPI-enabled learning content). VTA’s support for cmi5 reduces the variability of the learning experience and technically enables a spaced learning approach (among enabling lots of other learning experiences). The reality is that RISC’s implementation of cmi5 makes a lot of things possible where we need to know who the learner is, and we want to track learning activity to an LMS.

And, with RISC able to track a spaced learning experience, they can look at the data on the content delivered over time, against the performance outcomes (like incidents reported) and optimize the spacing relative to the type of content, the subject of the content, the complexity of the job performance, etc. VTA clearly gives you all the tools you need to do this. All you need is content that uses xAPI in specific ways to populate these reports with data that is formatted to deliver insights. After all, I’m highlighting what RISC can report, but that requires data that is useful and usable to populate these reports with appropriately formatted data.

For that, I can’t wait to share my next blog post on what DominKnow is doing that enables what RISC (and other tools) can report on.

Interested in xAPI visualizations but can’t make xAPI Camp?  Duncan and Art have a session Wednesday, March 27th at Learning Solutions on getting measurable results with xAPI titled Design with the End In Mind.

xAPI Camp DevLearn Presentations

By

Here are links to all the presentations from xAPI Camp DevLearn. Enjoy!

iFest and xAPI Profile Servers

By

We’ve been hard at work the last few months identifying what an xAPI Profile Server should do beyond the basic requirements in the xAPI Profile Specification as part of a BAA with ADL. Last week we took the show on the road to iFest in Washington DC. We brought along a poster which explained why a profile server is necessary and how it will improve interoperability. That poster won a people’s choice award for best narrative!

(evidence^)

That’s why I’m writing this post, because we have never won a poster award. In fact, there are a lot of awards we haven’t won. And we’re okay with that. Since this one was a people’s choice award, we thought more people might like to see it. So, here it is…

 

Another post is coming with a summary of the profile server research. Big thanks to ADL for a great event and thanks to our designer, Jason, for a prize poster!

Filling in xAPI’s Most Important Gaps

By

2017 is a year that is really testing our capacities and limits. Politics, natural disasters, cyber security threats, budgets… even ADL has hit some road bumps. We formed the Data Interoperability Standards Consortium (DISC) a few years ago specifically to hedge against disruptions to ADL’s capacity for spec stewardship so that the people who rely most on xAPI wouldn’t be held back the way the learning industry was with SCORM 2004 (after ADL stopped innovating on it… in 2004). Anyone who’s worked in or with a government for more than a few years knows to expect that there are these hiccups which, for an indeterminate amount of time, stall progress.

Industry and momentum, let alone organizational strategies, shouldn’t be subject to the whim of such hiccups — especially when the industry could address these on its own.

Working hand-in-hand with ADL over the last year, DISC and the xAPI Community hit some pretty major milestones that really stabilized xAPI, in terms of implementation, to enable more adoption, faster. Also with ADL, we identified a series of priorities to work on together over the next several years. ADL’s funding situation means that those plans would have to be put on hold for us to work contractually with ADL. However, that (realistically) could be almost a full year in waiting to get started. As an advocate for its progress, I’d rather we work on these things in a different way, not needing to rely on ADL, so that when ADL’s budget capacity is restored, they can advance things much further afield. As an industry that depends on xAPI, it’s in the xAPI Community’s best interests to keep things going.

Megan and I are sharing the priorities for open source development and documentation we worked out with ADL with the hope that the xAPI Community will seize this moment for the opportunity it presents: to chart its own path to growth and ubiquity.

  1. Stand up a xAPI Profile server. With the xAPI Profiles specification released in June, the xAPI Community has finally addressed a huge gap in achieving semantic interoperability. However, without even a reference implementation of an xAPI Profile Server, the spec will never see its potential to scale xAPI across industries, and we will continue to have challenges of semantic interoperability across implementations. There’s a minimal development effort as a lot of the xAPI Profile Server leverages existing JSON-LD tools.
  2. Stand up a service to test for valid, well-structured xAPI Profiles. Right now the only way to create xAPI Profiles in JSON-LD format is to do so by hand, which is a laborious process that is prone to manual errors. It’d be helpful to have something available in lieu of tools that validated an xAPI Profile, highlighted where the errors are and the nature of them, so that the people producing xAPI Profiles could do so easily, and so those xAPI Profiles could be used by others with confidence. This likely requires a bit more development than the xAPI Profile Server requires.
  3. Stand up a tool to help people publish xAPI Profiles in proper JSON-LD format. What would be even better than producing xAPI Profiles by hand would be a web-based application with an interface that made it much easier for people to create valid xAPI Profiles, so that subject matter experts (or, more likely, data people working with subject matter experts) could generate profiles without needing to encode JSON-LD themselves. Not only does this require some development savvy — it would benefit from having a product manager and interaction designer so that this could be both usable and useful to many, thus encouraging more organizations to generate and share xAPI Profiles.
  4. Develop conformance requirements and a conformance test for xAPI Profiles and the xAPI Profile Server. Much like DISC facilitated with the xAPI Community in 2016, it’s likely that many LRS vendors (and probably authoring tool vendors as well) will incorporate an xAPI Profile Server and want to validate the data collected in their LRSs against valid xAPI Profiles. It would be wise to get ahead of inevitable interoperability challenges and develop (first) conformance requirements and (later) conformance tests that could ensure consistency in what we deem to be “conformant.” Related — if it’s a common goal by LRS vendors and stakeholders that LRSs will validate statements against profiles, that should be made explicit in the xAPI specification and additional LRS Conformance requirements must be developed, as well as additional unit tests in the LRS Conformance Test Suite.

That’s a solid list, and the community could rally itself to take this on and commit to seeing it done in the next twelve months. The organizations willing to commit resources to develop these efforts to a defined set of outcomes should be embraced by the community.

The things listed above are needed now. The market can probably tolerate a year without it impeding xAPI adoption, so long as these things are available to the market and implemented as organizations budget for FY19. There’s more for to do in the coming twelve months, but in the general category of making things, these are things with definitive outcomes that would help everyone.

So… who’s going to take on what in filling in these pretty important gaps?

Tim?

With Gratitude…

By

Megan and I find ourselves grateful, as the year comes to a close. 2015 gave MakingBetter an amazing journey that was full of surprises. Most were wonderful and some were very scary. Through it all, we found ways to make our clients happy and successful doing work we believed in. When we had struggles and found roadblocks, we worked together to get over whatever the hump was and we got to a better place.

With Gratitude

We created things with great people. Our projects ranged from designing and developing custom reporting for software products, training providers and large enterprises. We launched an online journal, the xAPI Quarterly, kickstarting the publishing arm of our business, Connections Forum, and we ran our first events, xAPI Camps that each were co-created with our participants. In January 2015, we planned for one and as the year closed we had four with five more scheduled in 2016. Our next one at the Autodesk Galleria in downtown San Francisco, February 11. We celebrated another amazing year with the amazing community, Up to All of Us, which will convene again in Sonoma County, February 12-15. We started a non-profit. More on that in a minute.

Grateful to Make MakingBetter Happen

MakingBetter at the Grand Tetons

Megan came on full-time with MakingBetter in June, this year. We took our first serious vacation ever in July. We spent a lot of time with our families and friends. We lived and worked, together, on our own terms for the first time in our lives. We dealt with emergencies and surprise medical concerns. We innovated when we needed to and we stuck to tried and true processes when we needed to, too. We lived well in 2015. I say all this because it’s important to celebrate success and to make sure that credit goes where credit is due. I write tonight grateful for a true partner like Megan, grateful for each and every client we had this year, grateful for each and every person who’s influenced how we do what we do, grateful for our sponsors and our partners and especially…

Grateful for the xAPI Community

It’s the xAPI community I want to talk specifically to now. There are a myriad of reasons why 2015 was good for Megan and I, but the one reason that stands out is the incredible gains in xAPI’s adoption that happened this year. We know there’s been incredible growth in xAPI adoption. Our business boomed and so did that of many software vendors who create solutions that are tailored to meet some of the many things people use xAPI for. We know projects are already being planned for the beginning of 2016 at a scale that equals the whole of xAPI adoption in 2015. These are measureable outcomes of an open source community that has been lovingly and painstakingly attended by the US Department of Defense and its particular initiative, Advanced Distributed Learning. xAPI is in every way a stunning success. It is proving that open government practices, a pro-entrepreneurial approach and an authentic embrace of open source can stimulate innovation, enable implemented approaches to complex and serious challenges, and catalyze economic opportunities. It is far from the applied research and development activity it was four years ago. It is a mature spec that is growing its own industry.

Grateful for xAPI’s Growth

xAPI Camp - DevLearn

xAPI is so successful that it’s actually becoming a challenge for ADL to support it to the scale it now demands. The Design Cohort program that began in 2013 became so well attended and populated that it couldn’t be supported by ADL anymore — they just don’t have the resources to do it on their own. The maintenance of the spec is labor intensive enough for the resources ADL has, that certification isn’t something they can handle on their own without stopping something else important. When SCORM was being created, it was an epoch ago for information and instructional technology, and ADL had over 40 engineers they could apply to SCORM alone. ADL now has dozens of high priority projects and there are maybe six full-time engineers they can resource for xAPI. Fortunately, those of us who brought to ADL the concepts that enabled xAPI’s creation knew that the day would come when specs and standards would need to move beyond ADL to truly mature. This is why open source was so crucial a path for xAPI. It’s because xAPI is licensed Apache 2.0 that anyone can take xAPI and mature it, and that’s just what we’re about to do, given ADL’s blessing and commitment to participate in the effort.

Grateful to Serve Our Community

The non-profit we started at the close of 2015 is the Data Interoperability Standards Consortium, or DISC (because, acronyms). There are many challenges to working with data: interoperability, security, privacy, professional competencies, validation, provenance, ethics, legalities, languages, formats, etc. We intend for DISC to offer the table where all communities of practice, individuals, organizations, governments and industries can work together to meet the complexities of working with data. It’s about more than xAPI, but make no mistake, xAPI is our priority in 2016. The transition from an ADL-organized xAPI Community to a DISC-organized xAPI Community will begin in the first quarter of 2016. By the middle of the year, we’ll have established working groups and special interest groups to explore ways in which xAPI may be extended as well as certification requirements. By the beginning of 2017, we’ll have a certification program in place and an array of tools that will make working with xAPI’s vocabulary much much easier.

That seems like a lot to get done in one year, and you’re right. It is a lot. But it will be done because it has to be done. xAPI is growing so much that if we don’t have certification in place by the end of 2016, we risk xAPI’s long term future. We predict a massive catalyst for international adoption to emerge by the end of 2016 in the way of procurement requirements for governments around xAPI, because having data that everyone can understand and can make use of is in the interest of public good institutions. When governments are a year away from requiring xAPI support and certified products are all that will be purchased, it makes right now the very moment where xAPI goes big. It is exciting, frightening and uncertain – and it’s fun, and this is what it’s like for us to be so fully invested when the stakes are this high. The fact that the stakes for xAPI are this high should be the reassurance everyone needs that xAPI really is a big deal and it’s worth our sweat to invest in its growth right now.

Grateful for a Future We Can Forge Together

Seattle Gathering

Because xAPI is open source, and because xAPI will have an organization that is focused expressly on its maturity, it’s going to get the chance to grow in a way that no learning technology has ever had the chance to do. Megan and I are proud to have an incredible team on DISC’s Board of Directors from around the world who represent years of extraordinary work in leading professional organizations, the science of learning analytics, the development of industry organizations, professional practice and xAPI itself. Very soon, we’ll announce our founding Board of Directors and post our by-laws, our 2016 goals and objectives and we’ll open membership. xAPI will forever be Apache 2.0 and we intend to ensure that it remain open source and cared for by an open community as long as it remains relevant. The organization we’re creating will finally structure how decisions about it are made, balancing the needs of those most invested in xAPI requirements with the needs of those most impacted by xAPI applications. Without the burdens and caveats that come with moving this activity into large spec and standards groups, as a community and an industry with many verticals, we can design our own future with xAPI.

Grateful for Your Help In What Comes Next

There’s been only a few sketchy roadmaps for what Megan and I have been doing together as MakingBetter. There are even fewer notes on what we’re about to do with DISC in forming an industry organization to support a major open source project with the cooperation of its stewards in the US DoD. But, this isn’t the first time Megan and I have had to work with a community to create something that didn’t previously exist. We’ve done it with Up to All of Us. We did it with growing xAPI into a fully realized community of designers, developers, content and data wranglers. We did it with figuring out how to fit open source for US government. And now we’re going to figure out, with the full interest by and for the community, how we grow the industry and professional practice around xAPI. It will require paying members and continued open community participation. It will require a level of dedication, enthusiasm and grit that hasn’t been demanded yet. Given all that, I’ve never been more confident in our abilities, all of us together, to figure this out. We’ve been able to plan and go off-plan and get this far. It stands to reason we’re going to go a lot farther together.

Megan and I are staking our business on xAPI. We’re staking our families on xAPI. We’re committing our lives over the next couple of years to the community and industry around xAPI and we are grateful to do so.

We wish you all the best for this holiday season and for the new year to come. We’ve loved hanging with you. We’ve loved working with you. We’ve learned so much in doing so and we can’t wait for the next level shit about to happen!

More soon!

September Recap: xAPI Quarterly, DevLearn, Consortium

By

September was one hell of a month. We…

1. Introduced the xAPI Quarterly

This is a journal we will be publishing four times a year. We will publish the newest information around xAPI. This will be the space for you to find out what’s happening and what’s changing. We will ask many of our xAPI Camp presenters to extend their messages into articles, helping xAPI Camp discussions live outside of the rooms where the event happens. Writers will come from everywhere, please reach out if you have an article in your head that needs to get out 🙂

This inaugural issue includes articles on xAPI security, instructional design forays into xAPI, best practices for major authoring tools (Storyline, Captivate, and Lectora), information on an xAPI Consortium in the works (see #3!) and much more.

Take a look at the table of contents here

2. Ran xAPI Camp – DevLearn

Happy xAPI Campers
Happy xAPI Campers

Whoa. Keanu whoa. This event blew us away. Another sold out house. Another amazing group of speakers. Another engaged and motivated audience. Another group of generous sponsors. We really can’t ask for more than this. The xAPI community never ceases to amaze and this was proof beyond doubt that you are THE BEST PEOPLE anyone could ask to work with.

All of the slides collected from the event are in the xAPI Camp Archives

3. Started a non-profit!

Wah? Yeah, we’re right there with you. This is big and we’re just getting started. There is now a non-profit corporation in the state of Pennsylvania called the Data Interoperability Standards Consortium, this is the way forward for the work we need to do as a community and industry.

This is explained in more detail right here.

And to October we say, ‘bring it on.’

xAPI Shifting to 3rd Gear

By

Mark Oehlert kicking off xAPI Camp - AmazonYou might be surprised to find out we’re in 2nd gear. We shifted out of 1st gear at DevLearn last year. Between November 2014’s xAPI Hyperdrive and the xAPI track at DevLearn 2014, we hit our stride. We had people well outside of the ADL bubble talking about real world things things they were using xAPI to accomplish.

What happened at xAPI Camp Amazon?

This wasn’t merely a great event. This was a game changer. The participants, including the speakers and our partners — this event really couldn’t have come out more epic than it did.

Let me talk a bit about our speakers. They are not messing around. All killer, no filler high-octane Awesome. We got Sean Putman to come in from Detroit to talk about what he all did with Altair’s software and xAPI. Ben Erlandsson, who’s working on the boldest and most complex application for xAPI I’ve ever encountered — and how xAPI stands to help make the most impact of any social initiative to date. Myra Travin came from across town in Seattle to articulate the field of learning experience design in a way that really made sense. Kirsty Kitto and Aneesha Bakharia are rockstars in research that’s driving the learning analytics field — and they’re pointing out ways in which we need to be using xAPI better. Bill McDonald spent 25 years helping run the Aviation Industry’s Computer-Based Training Consortium, eventually overseeing their own standard evolve to be completely based on xAPI. Russell Duhon talking about what it takes to run xAPI in Enterprise… at scale. Duncan Welder talking about the ways in which their LMS is helping Big Energy improve their compliance — not just get a check in the compliance training box, using xAPI for both content analytics and social. Their presentations are all archived here.

A breakout group at xAPI Camp - AmazonThese speakers were incredible people, many of whom are outside the xAPI Developer/Contributor/Adopter circles. Real folks making real decisions about real world problems and using xAPI to help. They’re haulin’ ass.We had small emergent breakout groups and large breakout groups moderated by our partners Shelly Blake-Plock, Mike Hruska and Nick Washburn. Participants got into the weeds of learning architecture, project management and scaling the technology to respond to growing organizational demands in near real-time. Everything discussed was approachable in real-world, real-human language but nothing was watered down. People took notes and tasked themselves for next-actions. Attendees were solving real problems while they were there.

The format for the event worked, it really worked. Myra Travin saw so much in it that she wrote a post on just how well it worked. You can read it on LearnxAPI.

Why was this camp so significant?

No matter what Amazon does with xAPI, their embrace of xAPI, even to bring the community to their house so we could learn together — what xAPI enables and (frankly) what value xAPI holds for them — is huge. Just huge.

You need a company to reference when asked, “Who’s doing xAPI?”

Your answer now is “Amazon.” Done. Mic drop.

Because when we talk about the future of learning, usually our professionals from ISDs to CLOs immediately talk about recommendations… “like Amazon.”  Every conference that talks about elearning at all — K-12, Higher Ed, Community Colleges, Corporate, Gov, .mil — everyone talks about recommendations “like how Amazon does it.”

…And even Amazon is going xAPI.

You hold onto that for a moment. Let that thought just linger in the air a bit. Enjoy it. Savor it. This is the kind of moment — those of us who’ve helped make xAPI happen — this is the moment we’ve been waiting for. Given their engineering culture, given their fanatical attention to customer service, given their massive supply chain and the people power it takes to make all of those parts work in harmony, xAPI could not have a better stakeholder.

It marks the beginning of the end of early adoption for xAPI. You go back to the bell curve model of Rogers’ Diffusions of Innovations theory. We hit the level taking us to Early Majority.

This is the shift from 2nd to 3rd gear. There’s more to be done. It’s not just events, it’s not just fancy new adopters. There’s a post coming later this week on what has to happen next. (Hint: we need to formalize our work together to make sure what the early majority is doing keeps working and makes the space to iron out the wrinkles existing today.)