Home › Novella › Scene 16: Jack and Paloma review activity choices
Posted on by — No Comments ↓

Scene 16: Jack and Paloma review activity choices

Facebook
Twitter
LinkedIn
Share

In which Jack and Paloma delve much deeper into what’s going on in the Social Studies activity.

Instructional design processes will observe and predict cognitive effects with increasing accuracy and experiential results.

Jack had persuaded Paloma to be a sounding board for his initial analysis. He once again put up the five choices,

[STATE]'s state government spends about [EXP] dollars a year on health care, which is [PCT]% of the budget. Do you think [EXP] is too much? Not enough? Just right?"

  1. too much, because, our state government spends too much
  2. too much, because, so many medical services are wasteful
  3. just right, because, our best experts have decided the right amount
  4. too little, because, so many people don't get enough treatment
  5. too little, because, healthier people are happier and more productive

 
and began, "So, it turns out the kids who pick number 3 are from Socialist-leaning families…"

"Inferred as Socialist," interjected Paloma.

"Yes, of course, inferred."

"Right. Really!"

"Yeah! And, (big surprise,) since they're from wonky families, their persistence is high."

"Right. Wow."

Jack let this sink in. It called into question several things about the design of the activity. The required specifications for this particular activity segment—for whatever reasons—had dictated that greater branch density should be allocated to "advanced" kids. Not that kids in Socialist party families were automatically the most "advanced," but… well, it was well-proven that they knew more about government and tended to have longer attention spans.

"So, what to do?" Paloma finally asked. Should we develop new material and new branches for people who pick number 3? Or do we change the choices so that there's more interesting ones for smart kids?"

The political significance notwithstanding, their focus was purely professional. If the activity did not draw particular students down particular paths that led to deep emotional resonance and to the defined learning outcomes, then the activity was broken and needed to be fixed. Jack and Paloma were skilled at combining the current science of attention, analysis of the empirical data, and their own skill at learner empathy to make the right design fixes.

As they discussed it, they realized that they needed to know why kids from Socialist families (who typically had learned Socialist biases and slogans at home) sometimes picked the "just right" choice instead of the "too little" choices.

They had at their disposal a video database that had recorded nearly all of the hundreds of thousands of interactions kids had had with the activity. All, of course, was carefully managed in accordance with the privacy and data-handling laws.

The most useful videos were those where two or three kids had done the activity together, since they were verbalizing their reasoning aloud among each other. The system did its best to annotate what it thought was going one in each of those interactions, based on particular kid profiles and histories, and on particular features of the activity. These annotations, of course, couldn't always be relied upon for correctness.

Jack and Paloma

"Idea…" started Paloma.

"Shoot."

"We should listen to a Global-plus-Socialist pair who debated between 2  and 4. But: dumber kids, not smart."

"Why not smart?"

"Eh," Paloma grimaced, "hearing academicky arguments won't help us, we…I want to know, which slogans tend to stick in their heads… you know, stuff they heard at the dinner table, and how they understood it."

Jack liked this strategy, and so constructed a longish query to get the best possible interaction samples. They scanned the list and picked one out to play. The video started, underneath displaying the kids' profile summaries and other annotations. It played at an accelerated but comprehensible rate. Pippin was a Socialist kid and Dev was the Global kid.

Pippin:    …Geez. Well, not the second one either, the wasteful one.

Dev:    Why not?

Pippin:    Eh, I dunno. How could medical services be wasteful?

Dev:    It says – no, it doesn't say they're all wasteful, it says some of them.

Pippin:    Oh…

Dev:    But it's… this is stupid, how are we supposed to know? We're not experts.

Pippin:    Well, duh, you're supposed to guess!

Dev:    Right.

Pippin:    Just guess something.

Dev:    You guess something.

Pippin:    Ok, the fourth one. The government is supposed to pay for everything.

Dev:    But—everything? You think the government can just pay for anything we ever want?

Pippin:    Well, I don't know, you guess then!

Paloma couldn't resist an appreciative smile. She loved seeing when kids were engaged with the questions.

ANALYSIS
This scene depicts some of the blocking and tackling involved in instructional design. Not every bit of learning content will merit the same level of analysis and evaluation, but when very large numbers of students will be accessing, fine-tuning is worthwhile. Today, the greatest amount of this type of effort goes into interactive math learning materials; however, as our expertise and tools advance, more and more subjects can and will be so improved.

Much of the battle involves figuring out what is going on inside learners’ heads when they encounter each bit of content. Watching pairs of learners talk about their own reasoning processes is a particularly valuable source. Machine learning programs will also help: combining evidence from many thousands of interactions with individual learner profiles and histories, and algorithmically finding patterns and reporting them to the instructional designers.

Protection of privacy will be critical for all of this progress to happen. Data relating to students must only be used for purposes that everyone agrees are value-added. And potential misuses must be blocked—with laws, of course, but also with information system safeguards analogous to what we have at banks and credit agencies.

 

cast of characters