Cognitive Task Analysis
ABOUT
For this assignment, students will do a cognitive task analysis to uncover details of student thinking in a given task domain: geometry problem solving. The main objective is to figure out what is the most effective strategy and skills required for students to solve these problems. Afterwards, we will create knowledge models so that we can pinpoint what is working and what isn’t working during the process of solving. In the end, with the implications and information found out from the protocol analysis, there should be an educational game that is designed to help students exercise the geometry skills needed.
TASK DESCRIPTION
Before coming to a final decision for my final task, I asked an expert to solve the whole packet. I tested all the problems first with an expert to see how long it would take and if all the problems were worth testing. I was able to distinguish my expert participant because she is currently a sophomore in Math and said that she has recently taken a high level geometry course. She was given the whole packet, including the theorems, without any time restraints. After analyzing and watching her solve the whole packet, I realized that I wanted to cut down on the problems. I noticed that during the first 5 to 10 minutes, she worked effectively and seemed to get the problems solved quickly without having any buffers. However, after it hit the 15 minute mark, she seemed to get distracted and was not paying much attention to the problems. She was able to finish the whole packet in about 20 minutes. We went over her answers together and noticed that she got answers wrong in the first few and last few questions. They were not wrong because she didn’t know how to solve them, but because she did her simple math wrong. She had the correct equations and did everything the way it should be, but she just added and subtracted them wrong. I was thankful enough to discuss with her her process and we came to the conclusion that the first few answers that she got wrong were because she got “cocky” since she thought it was so easy, causing her to rush and do simple math wrong. Similarly, the last few questions she got wrong were also because she was not paying attention and just did her simple adding and subtracting wrong because she just wanted to finish the problems as soon as possible.
After working with the expert, I decided to give each participant 5 questions. There was no time limit, so novices were able to try as much as they wanted to. I used the first 5 questions from the packet. I also gave everyone the theorems to see how they utilized it differently. The first 3 are ranked as easy for level of difficulty while the last two are slightly more challenging. The main objective was to see how participants solve easier questions and more difficult questions and if there is a difference in how they solve questions from different levels of difficulties. Are they more susceptible to getting them wrong more than challenging problems, because they think it is so easy and just do not pay closer attention to the problems or are they more susceptible to getting the more challenging problems wrong because they just do not understand the materials well enough?
THEORETICAL CTA DIAGRAM
I sketched out and looked at different ways that people may solve the task before creating my final theoretical CTA Diagram. It was interesting that there are so many different ways a person can think, so creating a general concept map of how an expert might solve the task was much harder than I thought. I think it was much harder because I myself am not an expert because I haven’t taken a math class in 4 years and haven’t done any math in college at all.
THEORETICAL CTA SYNTHESIS
While answering this prompt, I tried to understand from an empathetic standpoint of how a novice participant would think. I tried recalling when I tried solving a problem that I did not know well. I recall trying to solve the example problems given with the formulas so that I could get a better grasp on how the formula works. In addition, I went to the formulas sheet multiple times back and forth because I did not know how to the solve the problems. After looking back when I was a novice at problem solving, it helped me get a better understanding of the process a participant may go through.
I gave the packet with the formulas first and then the problem sets. I infer that the novice participant is going to read the formulas first. Then start the first question. Afterwards, he or she will go back to the formulas to find the right one. Then, they will go back to the problem and try applying the theorem. If the theorem is incorrect, the participant will go back to the formulas sheet and do everything over until they get the correct formula and get the finalized answer. I believe that this process is what distinguishes a novice to an expert. Novices will be unable to manage their time, so they will waste time on things that could be prevented i.e. getting the formulas quickly. An expert will go straight to the questions to see what formulas are needed so that they do not have to keep going back and forth. The finalized chart that I created would be more of what an expert would do. I believe that novices will have many more steps. For example, I can see some participants actually solve the example problems so that they can better understand the formulas.
EMPIRICAL CTA PARTICIPANT REQUIREMENT
In order for me to recruit my participants, I asked my fellow peers what their confidence level was in solving a geometry question at that very moment and when the last time they took geometry was. Each participant that I asked, excluding the expert, had not taken geometry since middle school or high school. Also, their first reactions about solving geometry questions were very negative as all of them did not have any confidence and said things on the lines of “I’m probably going to get them all wrong” and/or “I don’t remember anything from that class.” I did not mention that I would be giving them the formulas to see if they look at the problems before they start or while they are answering. Because of them not knowing that information, every participant was concerned that they do not remember the formulas.
The first participant was the expert and she is a sophomore in Math. The last time she took a course related to geometry was last year for fun. The first novice to participate was a current sophomore in Design and had not taken geometry since middle school. The next participant was a sophomore in computational biology. She said she took math courses while attending CMU, but they had nothing to do with Geometry. The last time she took that class was also in middle school.
EMPIRICAL CTA PARTICIPANT STRATEGY: EXPERT 1
It was interesting to see how novices and experts work differently. This participant was an expert as she is a sophomore majoring in Math. She was concerned in the beginning because she has not taken geometry since last year and she did not remember any of the formulas. When she received the packet, she first throughly looked over the formulas and then moved on to answering the questions. While she answered the questions, she only looked at the formulas two times out of the five sets of problems. Because she had reviewed the formulas in the beginning, she did not have to go back and forth. This could also have to do with the fact that she took a geometry course last year so she has some recollection of this information. She got all the problems correct and used the formulas correctly.
EMPIRICAL CTA PARTICIPANT STRATEGY: EXPERT 1
EMPIRICAL CTA INDIVIDUAL DIAGRAM: EXPERT 1
EMPIRICAL CTA PARTICIPANT STRATEGY: NOVICE 1
This participant was an novice as she is a sophomore majoring in Statistics and Machine learning. Like other participants, she was extremely concerned because she has not taken geometry since middle school and even then, she said she struggled with that subject. When she received the packet, she went straight to the questions. She tried solving the problem without the formulas for the first question, but soon after went to the formulas sheet because she didn’t remember anything. When she looked at the sheet, she struggled to focus and understand the theorems, because she kept going back to the sheet. After she found the theorem, she went to answer the question. After she got the solution, she forgot what formula she used, so she had to go back again to get the name. She continued to work like that with the other questions. She This could also have to do with the fact that she took a geometry course last year so she has some recollection of this information. She got all the problems correct and used the formulas correctly. She flipped back to the formulas sheet 11 times.
EMPIRICAL CTA INDIVIDUAL DIAGRAM: NOVICE 1
EMPIRICAL CTA PARTICIPANT STRATEGY: NOVICE 1
EMPIRICAL CTA PARTICIPANT STRATEGY 1 : NOVICE 2
This participant was a novice as she is a sophomore majoring in Design. She was extremely concerned because she has not taken geometry since high school. In addition, she has not done any math in college, so she was freaking out and trying to back out without even looking at the problems. When she received the packet, she read everything. The first few pages of the packet were the theorems, so she took about 5 minutes to read through them. Then she started her first problem. She read it and immediately went back to the formula sheet.
EMPIRICAL CTA INDIVIDUAL DIAGRAM: NOVICE 2
EMPIRICAL CTA PARTICIPANT STRATEGY: NOVICE 2
EMPIRICAL CTA UNIFIED DIAGRAM
EMPIRICAL CTA SYNTHESIS
The unified empirical CTA diagram describes the overall problem solving strategy that novices use. One main problem that they had a difficult time digesting was knowing what formulas to use. After solving the questions, I asked them about their experience. Both emphasized how they wish they had more practice with the formulas, because they would keep forgetting and having to go back to the theorem sheet to look for the correct formula.
Adding on, some of the participants tried using mental math, which made them get the incorrect answer without them knowing. There were many instances where the participant wanted to get the question over with, so they rushed and did not throughly execute the problem. That issue could have been resolved if participants slowed down and wrote and solved everything on paper. Mental math made it more difficult for novices.
Overall, novice participants functioned in similar manner to each other. Both participants looked back at the formula sheet multiple times because this material was so old they did not remember much of it. I gave every participant the formula sheet and then the problem sets. The novices read through everything before starting which was different from how the expert solved the problems. Afterwards, they read the first question and then looked back at the theorem sheet to find the necessary information needed to move on with the questions. For each problem, both novices had to look back at the formula sheet. The expert rarely looked at the sheet. She only looked at it throughly after she read the first two problems. After those times, she did not have trouble with remembering the theorems. It was much easier for the expert to retain the formulas and concepts because she had taken a class similar to this not too long ago. On the other hand, the novices haven’t taken Geometry in a couple years, so even though they had the formulas sheet, they did not remember the concept for how to use the theorems which made them take more time for solving the problems.
REFLECTION AND DISCUSSION
In order to create an educational game for players to better practice their geometry skills, I created three ideas that I thought would be interesting but also effective in teaching the learning goals. There were many things that I noticed that participants failed to keep in mind, so each game deals with a different issue. One issue that I saw was that players just had not touched upon this material so they needed some practice. Once they solved multiple sets, they were getting better at solving. In addition, some participants rushed through the problems and did not pay close attention, so I wanted to create a game where players throughly and effectively solve the problems without making simple mistakes. Lastly, there were some participants that took way too long, so being able to create a time limit so that players are thinking effectively rather than zoning out.
The first idea that I had consists of having a game that is visually similar to Twister. The rules do not have any correlations with Twister. I wanted this game to be quick so that players can really exercise their angle finding skills. In addition, this is a in-person physical game which can be beneficial when players do not understand a concept and their peers can help on the spot. On the game board, it will be a big plastic sheet that has a huge circle. This will be the “circle” that people will base off of when looking for the answer. Around that circle, there are smaller circles that have angles on them. Every round, there will be a problem given for players to try to figure out what the angle is. Once they have their answer, they can stand on which ever spot has the angle they solved. This is a fun and quick game for students to play in class. The purpose of the game is for players to quickly find the answer with the concepts that they learn and then quickly go to the spot that they think is correctly. One issue that I thought while creating this was that some players may not know and will just follow other players to a spot rather than solving the problems themselves.
The next concept that I had creating an educational game which gives players a set of problems similar to what we were given, but the difference is that players are not allowed to pick up their writing utensil until they get the answer. The purpose of assigning this rule is so that players really consider what process they take so that they are not rushing and really thinking through how to get the answer. While discussing this idea with my peers, they mentioned that it seemed unnecessary to have and would rather go with the first idea.
Afterwards, I thought of my final educational game concept by using bingo. Students would be able to create their own bingo board by picking out angles. Once they have completed their board, students will be given problems and they would have to solve them. The last player to get the answer will not be able to place a mark on their board. This game pushes players to try to find answers quickly while also reviewing geometry. Each player will be given a briefcase type board game. One side will be a white board so that players can create their board games while also some space for players to solve and do the work. On the other side of the brief case will be sections for the cards. One section of the cards will be of the numerical angles that they choose. The next section of the cards will be of a visual of the angle. Players will be able to use any of the cards to add onto the bingo board.
Overall, I thought it was interesting to see how every thinks differently. I understand that my flow chart does not follow everyone’s process because everyone perceives things differently, but I saw some process’ that I did not even think about. I was correct with the idea that experts do not go back to the formula sheets as much as novices because they are able to use their time wisely and just look at it once or twice. On the other hand, every novice went back to the formula question when starting a new set of problems. This aspect would be interesting when thinking about the MDA of the game, so that I can implement a time constraint so that participants are purposeful about their actions when trying to accurately and effectively solve the sets. On thing that I noticed was that the expert was really focused and barely talked while solving, while novices continued to talk while solving the problems, causing to them to get distracted and not effectively work. I infer that the expert was able to solve the problems correctly was because she was working effectively since she has a passion for it and confidence in solving. In comparison, the novice participants all lacked much confidence so they did not concentrate as much as they usually do for a subject that they are passionate for. This observation is interesting as someone could try creating a game for strengthening effective concentration. At first, I was skeptical about this activity because I had not done math since high school and had not touched it at all during college. However, I had much more fun with this than I expected. It was nice for me to go back to this subject. It was helpful for me to test an expert since I was not confident in my geometry skills. I was able to gain insight on how to make successful decisions when solving geometry problems, which I would not have ever known if I did not ask an expert. I understand with time, we are unable to finalize the game concept that we were supposed to make, but if I could, I would want to finish and see where it goes, because I think the educational game that comes out of this will be interesting and extremely effective with helping reach the learning goals.
