unning a website populated by bright math students creates a number of unique challenges. When we make mistakes, they get called out pretty quickly. When we do something cool, everyone is thrilledâŚ and then also offers up advice on how we could have done it better. And when we build something interesting, someone is always going to want to know how it works.

One of the tools that sparks the most questions is Alcumus, a free tool from AoPS that helps students learn by adaptively delivering problems for them to study. At its core, Alcumus is a giant database of well-written problems and solutions. Sitting on top of that database is the Alcumus engine which âratesâ students and decides how well they understand topics, determining when they should move on and when they should keep studying.

(If youâre thinking, âWait, Alcumus is a math thing? I thought it was a Transformer,â then you might want to check out this AoPS News article to get better acquainted with Alcumus before you read on.)

## How do you rate students?

Alcumus problems are sorted into topics. Students get to choose which topic they want to study, and Alcumus tracks their performance in each topic with topic ratings. A hypothetical student, Melissa, would like to learn about geometric probability, so she might set Alcumus to the Using Geometry in Probability topic. Thatâs a hard one. Melissa may have her work cut out for her.

Alcumus gives Melissa a rating between 0 and 100 for every topic. This number describes how well the system thinks she understands the topic. When Melissa gets a problem right, her rating goes up, since Alcumus thinks she understands it better. Getting a question wrong will make that rating go down. But what does all that mean?

First, Melissaâs topic rating is the probability that she will correctly answer the average problem in the topic. So if Melissaâs Using Geometry in Probability rating is 75, then Alcumus believes Melissa will get three out of four average problems in the topic correct.

Under the hood, every student and every problem has a hidden score. We use these scores to guess at the probability of the student getting the problem right: if the studentâs score is way higher than the problemâs, then the probability is close to 100%. If itâs way lower, then the probability is close to 0%. If their scores are equal, thatâs 50%. The problem in Using Geometry in Probability with the highest rating is:

*A boss plans a business meeting at Starbucks with the two engineers below him. However, he fails to set a time, and all three arrive at Starbucks at a random time between 2:00 and 4:00 p.m. When the boss shows up, if both engineers are not already there, he storms out and cancels the meeting. Each engineer is willing to stay at Starbucks alone for an hour, but if the other engineer has not arrived by that time, he will leave. What is the probability that the meeting takes place?*

And the problem with the lowest rating is

*The fair spinner shown is spun once. What is the probability of spinning an odd number? Express your answer as a common fraction.*

As it turns out, at Melissaâs current score, Alcumus gives her a 97% chance of answering the second problem right, but only a 4% chance on the first one. The function we use to take scores and return probabilities is called the *logistic curve.* It looks like this:

This is actually the same curve that underlies how chess ratings work. The specific function Alcumus uses to predict the probability that a student will answer a question correctly is:

Probability=11+đproblem scoreâstudent score.

Melissaâs rating in a topic is the probability we get when we use this function to compare Melissaâs score to the average of the problem scores in the topic.

Once we have that set up as our model, we let the students play and let the Alcumus engine do its work adjusting the student (and problem) scores as it watches what happens.

## How do the ratings change?

So, how exactly do we measure how well a student understands a topic? What happens when Melissa starts to get better at geometric probability? First, we start with some guesses about how good Melissa is and we watch her solve problems. Next, as we start to see what she can and canât do, we refine our opinion of her. If youâre a robot and youâre doing all this with math, itâs called Bayesian Updating.

Bayesian Updating is based on a fancy toy called Bayesâ Theorem. Statistics students first learn Bayesâ Theorem by solving a sequence of contrived problems about cancer or people dying in hospitals. (If you Google âBayesâ Theorem examples,â you get about 1.4 million hits. If you Google âBayesâ Theorem hospital,â you get around 700 thousand.) Statisticians are dark people.

When you write it out, Bayesâ Theorem is a scary bunch of symbols

đ(đ´|đľ)=đ(đľ|đ´)đ(đ´)đ(đľ).

I spent a lot of time staring at that equation once upon a time, so I know exactly how impenetrable it can be. Itâs also not even ârightâ for us. All four of those copies of đ in there mean something different and theyâre all hiding their own little secrets that confuse us mathematicians over and over and over again.

So letâs start over.

Weâre given a brand new student. We donât know much about her. We think thereâs about a 50% chance sheâs âaverage,â a 25% chance sheâs âabove averageâ and a 25% chance sheâs âbelow average.â Thatâs called a *prior*, as in, âThis is the information that I have prior to watching the student.â Mathematicians like to use the phrase âprioring onâ to sound smart and say, âThis is why I think some silly thing is going to happen.â As in, âPrioring on his outfit, I think heâs the most likely student to fall out of his chair,â or âDid you know he was going to fall out of his chair? No, but I had strong priors.â

Next we have the observation. If our student solves the first 10 problems easily, we might stop feeling good about our prior, and we update it. Now maybe weâd say, sheâs 5% likely to be âbelow average,â 20% likely to be âaverage,â and 75% likely to be âabove average.â This new belief is called our *posterior,* as in, um, âThis is the information that I have posterior to watching the student.â Sadly, the meaning of the word posterior has evolved a bit in the past couple thousand years, as in âThe student fell out of the chair and onto his posterior.â

How is this Bayesâ Theorem? Well, there are two ingredients to the posterior. First, thereâs our prior belief. If youâve been teaching a student for a full year, you have a very well-defined prior belief about her. She rarely falls out of her chair, so maybe we think of her as âprobably above average.â Seeing how she answers one problem isnât going to have much of an effect on that belief. Second, thereâs how well the story fits the outcome. Getting all 10 problems right on a really hard test is more likely for an above average student than for a below-average student. So that tells us sheâs more likely to be an âabove averageâ student than âbelow average.â What Bayesâ Theorem tells us is that we just smash these two effects together to get the posterior.

Posterior belief that sheâs above average = (how well being above average predicts the result) * (prior belief that sheâs above average).

Easy.

Homework for those inclined: the exact correspondence is

- prior: P(student is average)
- posterior: P(student is average | getting all 10 right)
- observation: P( getting all 10 right | student is average) / P(getting all 10 right).

What does this mean? Hint: Write đ(đľ)=âđ´đ(đľ|đ´).

## How does Alcumus pick problems?

This is one of studentsâ most common questions. First off, itâs random: Alcumus flips a coin and picks a problem for the student. However, not every problem is equally likely.

Alcumus begins by deciding whether to give a student a problem in the current topic or giving the student a review problem (from a topic he or she has already passed). After the topic is chosen, Alcumus gives each problem in the topic a probability of being picked. Problems youâve seen before are less likely to be delivered. If your rating is on the low end of the topic, Alcumus will prefer easier problems. If your rating is on the high end of the topic, Alcumus will prefer harder problems. If you change the difficulty, that shifts these probabilities, too.

Picking problems is highly constrained: if a topic has only hard problems, itâs going to give you a hard problem. Advanced Quadratics only has hard problems, so setting Alcumus to Easy and trying Advanced Quadratics will still give you hard problems. Is Alcumus adaptive? Does Alcumus teach?

We close with these lovely questions, which speak to a lot of the current issues in modern education, circling around what teaching actually is. Alcumus adapts to students in some ways. It tracks where you are and what you understand and adjusts what you see based on that information. It points you to resourcesâbook references, videos, community conversationsâthat you can use if you get stuck. It has solutions that you can read at exactly the point you need them.

The one tiny missing piece is that students still need to use these resources or some other available resource to get past that block. Alcumus as a learning tool works best when wrapped in some amount of teaching. Many students are excellent at teaching themselves, whether by reading a book or by working carefully through problems theyâve already seen. Others may choose to ask a parent, sibling, tutor, or someone else when they are stuck. We at AoPS embed Alcumus as a motivation and training tool into some of our classes to complement the teaching that we do there.

If you havenât already, go give Alcumus a try, and see if you can sense how the engine is working in the background as your ratings change. And weâd love to hear your opinions about how teaching fits into learning on our Facebooks or Twitters.

â