• Skip to main content
  • Skip to footer

MakingBetter

Skills and Strategies to Engage, Change and Grow Others

  • Data Interoperability
    • xAPI Camp Archives
    • xAPI Quarterly
    • Investigating Performance
  • Who We Are
  • News
  • Let’s Talk

MakingBetter

Question-storming

April 29, 2021 By MakingBetter

white clouds

“Question-storming” is an exercise I tapped into back in the very early xAPI days. I hit a wall; one I suspect most of y’all come up against when you’re ready to start doing something meaningful with xAPI. I struggled to articulate questions that felt like it made xAPI worth doing, and I would get discouraged about it.

A headshot of Aaron sporting a thick "fu-manchu" mustache.
Speaking of question-able choices, what about my mustache circa 2016?

When I look at what analytics solutions do (not data visualization tools – I’m talking web analytics), the analytics are geared towards defined funnels and targeted actions that map to KPIs. That’s a little too simplistic for what I plan to infer from the datasets y’all and I will build, together, with xAPI.

xAPI can provide feedback that is more attuned to how people think and relate information. It requires a different perspective on what questions are important to answer with data than how analytics work most everywhere else in technology. It’s no wonder to feel frustrated at the start to design, engineer or architect beyond replicating patterns established in eLearning or video.

Waxing Philosophical

All the learning environments we create online, all the social media we use, the apps we build, the laptop or tablet or phone you’re reading this on… it’s all built on a system that, at its very metaphorical base, is one of files and folders. One might think of hard drives as file cabinets, etc.… nice, neat and tidy. There are rights to knowing things in a folder vs only knowing about a specific file. There are ways to reference information precisely. This is important and useful stuff for referencing things technically.

The thing is that everything I read and learned and ultimately synthesized through my master’s program at University of Wisconsin informs me that humans organize information related to people, to places, to shared experiences. These relationships are not limited to strict hierarchies, like file and folder structures, as much of our technologies are. A hypothesis I have is that after two decades of constant online living, we as people get hyperfocused on the box we’re in, when we need to be looking through all the things that are outside our box we can play with.

Constants like time, for example, exist beyond the limits of your, and my, creativity (or lack of it). It’s a handy place to start question-storming because time is a concept that everyone involved with making, or taking meaning from the data can understand similarly.

Prioritize Research

At Elsevier, my product’s manager, Lauren, fleshed out a quarterly OKR that puts applied research and development as a focus for our team’s work. Each quarter, we decided, we’ll come up with a question to answer with our data set that we don’t quite know how to answer. A chance for us to improve our capacity and capability for learning analytics, as a cross-functioning product team that includes software and quality engineers, product managers, UX designers, commercial folks, content folks, users in multiple roles.

It’s the investment in drinking our homebrew that excites me most. We are all new at doing this. We get smarter about learning analytics when we establish trust among everyone who depends on the same set of data, regardless of why it may be important for different reasons to different people.

Yes, I’m talking ‘bout Praxis.

Question-storming “How Long Does It Take…?”

So, with time as our ally, Team Apollo at Elsevier began question-storming “How long does it take to get through an activity?”

Seems like a really simple question, right? Give it a few seconds to get your own doubts going. Turn those into questions.

  • Did he mean “learning activity” or, like, a survey?
  • What does it mean to “get through” an activity, specifically?
    • All the way to the last frame of linear content?
    • Do you need to have seen all the content?
    • Must you have passed in order to be considered complete?

Now… reframe the main question to reflect all the permutations of these bulleted conditions on how to interpret it. That is the question-storming.

Next post, I’ll highlight what we produced to answer our research question(s), what statements we used… I’ll go full nerd. I might get into how the results set a research agenda for the year. Meanwhile, in the comments, maybe share your takes:

For now, what are other ways to interpret “How long does it take to get through an activity?”

Filed Under: Learning Analytics Tagged With: design, question-storming, strategy, xAPI

Filling in xAPI’s Most Important Gaps

September 11, 2017 By MakingBetter

Cat on a Box

2017 is a year that is really testing our capacities and limits. Politics, natural disasters, cyber security threats, budgets… even ADL has hit some road bumps. We formed the Data Interoperability Standards Consortium (DISC) a few years ago specifically to hedge against disruptions to ADL’s capacity for spec stewardship so that the people who rely most on xAPI wouldn’t be held back the way the learning industry was with SCORM 2004 (after ADL stopped innovating on it… in 2004). Anyone who’s worked in or with a government for more than a few years knows to expect that there are these hiccups which, for an indeterminate amount of time, stall progress.

Industry and momentum, let alone organizational strategies, shouldn’t be subject to the whim of such hiccups — especially when the industry could address these on its own.

Working hand-in-hand with ADL over the last year, DISC and the xAPI Community hit some pretty major milestones that really stabilized xAPI, in terms of implementation, to enable more adoption, faster. Also with ADL, we identified a series of priorities to work on together over the next several years. ADL’s funding situation means that those plans would have to be put on hold for us to work contractually with ADL. However, that (realistically) could be almost a full year in waiting to get started. As an advocate for its progress, I’d rather we work on these things in a different way, not needing to rely on ADL, so that when ADL’s budget capacity is restored, they can advance things much further afield. As an industry that depends on xAPI, it’s in the xAPI Community’s best interests to keep things going.

Megan and I are sharing the priorities for open source development and documentation we worked out with ADL with the hope that the xAPI Community will seize this moment for the opportunity it presents: to chart its own path to growth and ubiquity.

  1. Stand up a xAPI Profile server. With the xAPI Profiles specification released in June, the xAPI Community has finally addressed a huge gap in achieving semantic interoperability. However, without even a reference implementation of an xAPI Profile Server, the spec will never see its potential to scale xAPI across industries, and we will continue to have challenges of semantic interoperability across implementations. There’s a minimal development effort as a lot of the xAPI Profile Server leverages existing JSON-LD tools.
  2. Stand up a service to test for valid, well-structured xAPI Profiles. Right now the only way to create xAPI Profiles in JSON-LD format is to do so by hand, which is a laborious process that is prone to manual errors. It’d be helpful to have something available in lieu of tools that validated an xAPI Profile, highlighted where the errors are and the nature of them, so that the people producing xAPI Profiles could do so easily, and so those xAPI Profiles could be used by others with confidence. This likely requires a bit more development than the xAPI Profile Server requires.
  3. Stand up a tool to help people publish xAPI Profiles in proper JSON-LD format. What would be even better than producing xAPI Profiles by hand would be a web-based application with an interface that made it much easier for people to create valid xAPI Profiles, so that subject matter experts (or, more likely, data people working with subject matter experts) could generate profiles without needing to encode JSON-LD themselves. Not only does this require some development savvy — it would benefit from having a product manager and interaction designer so that this could be both usable and useful to many, thus encouraging more organizations to generate and share xAPI Profiles.
  4. Develop conformance requirements and a conformance test for xAPI Profiles and the xAPI Profile Server. Much like DISC facilitated with the xAPI Community in 2016, it’s likely that many LRS vendors (and probably authoring tool vendors as well) will incorporate an xAPI Profile Server and want to validate the data collected in their LRSs against valid xAPI Profiles. It would be wise to get ahead of inevitable interoperability challenges and develop (first) conformance requirements and (later) conformance tests that could ensure consistency in what we deem to be “conformant.” Related — if it’s a common goal by LRS vendors and stakeholders that LRSs will validate statements against profiles, that should be made explicit in the xAPI specification and additional LRS Conformance requirements must be developed, as well as additional unit tests in the LRS Conformance Test Suite.

That’s a solid list, and the community could rally itself to take this on and commit to seeing it done in the next twelve months. The organizations willing to commit resources to develop these efforts to a defined set of outcomes should be embraced by the community.

The things listed above are needed now. The market can probably tolerate a year without it impeding xAPI adoption, so long as these things are available to the market and implemented as organizations budget for FY19. There’s more for to do in the coming twelve months, but in the general category of making things, these are things with definitive outcomes that would help everyone.

So… who’s going to take on what in filling in these pretty important gaps?

Tim?

Filed Under: Experience API, Open Source

For Individuals

April 17, 2015 By MakingBetter

Do you feel like you’re the lone voice of change within your team, school, organization? Are you trying to change your system from the bottom up and the inside out? We’re with you, fellow Change Agent. We share your struggles, heartache, heartburn and doubts. The struggle is real. But it’s not for naught. We can help you move your causes forward in visible, measurable ways. If you’re fighting the good fight, we want to upgrade your skills and confidence.

Over the last four years, we’ve hosted space for master practitioners to help each other grow. Learn more and join us at Up to All of Us.

Filed Under: Home 5

For Organizations

April 17, 2015 By MakingBetter

We’re about creating the most change with the least change. In other words, sustainable, continuous growth that looks to beyond short-term solutions and disrupting what’s already working for you. Bring on your toughest, mission-critical programs. We’re ready to hunker down, get to know your team and your systems, and make lasting improvements with measurable outcomes. Whether your product is learning, education or training or they’re just a big part of what you do to get the job done.

Filed Under: Home 3

For Communities

April 17, 2015 By MakingBetter

We light fires. For motivation, and for light. We bring people together to collaborate. We run events, publish articles, a journal, even a book (also flammable) to help communities understand new topics and grow side by side. We produce xAPI Camps. An event we love running but would love it even more if you took the reins. We publish the xAPI Quarterly, along with the help of many brilliant people. We published a book called Investigating Performance: Design and Outcomes with xAPI. We even archived the material from all of our xAPI Camps, let’s take a walk down memory lane.

Filed Under: Home 4

We are MakingBetter.

April 17, 2015 By MakingBetter

We work in the void between instincts and outcomes. Between individuals and organizations. Situations and standards.

We are a solid foundation of research framed by design thinking. Revealing ways to honor and grow the individuals who help you grow as a collective.

We create cycles that grow, too. Getting smarter and faster over time. Leaving organizations full of change agents in its wake.

We love a good challenge. And a good beer.

And we can’t wait to meet you.

Filed Under: Home 2

Footer

Sign up for email updates

Learn about xAPI

  • About xAPI
  • Adaptivity
  • Authoring Tools
  • Case Studies
  • cmi5
  • Content Strategy
  • Data Strategy
  • Learning Analytics
  • xAPI Statements

Learn More

Upcoming xAPI Camps!

Search

  • Home
  • Connections Forum
  • Investigating Performance
  • xAPI Quarterly
  • xAPI Camp
  • How We Can Help
  • Case Studies
  • Learning Tech Consulting
  • Talk to Us!
  • News

Copyright ©2014-2022, MakingBetter d/b/a Bowe & Silvers Partners LLC.

 

Loading Comments...