This is a sequel to my MOOC learning journey post, in which I compared the different online learning platforms and recommended good AI/Math/PM courses. In this post, I want to discuss how using meta-cognition (to be aware of what/how one’s doing and thinking) can improve learning effectiveness, in terms of absorption and knowledge application. Such self-awareness helps you adjust your strategy/approach as needed. It is like having an alter-ego that zooms above you, instrumenting each process and guiding your next step with rational data.
What makes a good learner?
As shown by the conductor Scott Yoo in his PBS special on music prodigies, there are three building blocks for a good learner:
- talent to play music
- talent to learn
- the ability to self reflect
- the ability to take immediate feedback to improve. These young musicians employ a very short feedback loop, compared even with software engineering agile cycle.
- know how to self-correct without guidance
- motivation to learn
- know why you like something and how you want to proceed
- know how to structure your learning
We might not have the innate talent to do something, but we can all learn to learn, i.e., imitate the genius’ learning ability in the 2nd and 3rd points above.
Be mindful of your own learning style
From my online learning experience, I realize that if you don’t understand what the lecturer says, just keep looking for other offerings. It might have nothing to do with your natural disposition or learning ability, but just a mismatch of your learning style with the teacher. For example, I am BOTH a visual and hands-on learner. I cannot listen to hours of proving a formula. I have to first see how it is applied to daily operations, then examine it through visual aids such as animations or charts. The latter is greatly facilitated in this era with more visual media available. For example, when I learnt the transformer architecture, Andrew Ng in Transformer Network intuition explained the q, k, v matrics and multihead attention in the encoder in the best manner (for me to understand), because he lays out flatly not the what, but why we even need them in calculation. The 3D Bert poster below takes care of my visual need in understand the how in each time step. For learners like me, grasping the why and a broader, 3D view of an abstraction really helps to develop intuition.
Learning by video is often challenging for me as I prefer to iterate between reading and doing. Viewing something in a big chunk and practicing in a long assignment isn’t what is most efficient for me. But modern media all focus on video and I have to admit that animation helps tremendously in understanding multi-dimension concepts such as linear algebra and deep learning.
Adopt retrieval practices to improve memory recall
The latest research in learning also gives us more tools to solidify newly minted knowledge. Specifically, we’re attempting to overcome the forgetting curve, which illustrates how memory retention declines over time. It was first introduced by German psychologist Hermann Ebbinghaus in the late 19th century. He found several distinct qualities of memory:
- Rapid Decline: Memory retention drops significantly shortly after learning. Ebbinghaus found that most forgetting happens within the first hour.
- Exponential Decay: The rate of forgetting slows down over time. After a few days, the decline in memory retention becomes less steep.
- Impact of Review: Regular review and practice can help counteract the forgetting curve. Techniques like spaced repetition are effective in retaining information longer.
Spaced repetition
Spaced repetition is a learning technique that involves increasing intervals of time between subsequent review of previously learned material to exploit the psychological spacing effect. This method is particularly effective for improving long-term retention of information. Here’s how it works:
- Initial Learning: You start by learning the material thoroughly.
- First Review: Review the material after a short period, such as a day.
- Subsequent Reviews: Gradually increase the intervals between reviews (e.g., after a week, then two weeks, then a month).
- Enhanced Retention: By spacing out reviews, you reinforce the memory just as it starts to fade, which strengthens the neural connections.
For my study, when I start a daily session, I’ll try to write down one thing I remember from the same course yesterday, last week, and last month. Since I am using Obsidian to manage my learning notes, the Spaced Repetition plugin helps to “slip” spacing into my routine easily. The key is to try recalling across different lessons. Per Carpenter and Agarwal’s 2020 research, cramming (intense brain dump the night before exam) only helps short-term memory, but has no effect in long-term retention. It is better to break a long cramming session into several smaller sessions spreading across time to increase the recall frequency.
Spaced practice helps recall significantly as shown in the following image.
Repeated spaced practice improves recall. Data from from Karpicke & Bauernschmidt (2011). Image Credit: Maverick Learning and Educational Applied Research Nexus
Interleaved practice
Interleaved practice attempts to mix different subjects in a session. It is shown to increase retrieval by 30% per a study by Rohrer, Dedrick & Agarwal in 2017.
Blocking vs. interleaving strategy from Weinstein, Madan & Sumeracki (2018). Image Credit: Maverick Learning and Educational Applied Research Nexus
In my experience, just following the quiz and assignment does not mean that the knowledge has sunk in. To do so, I have to connect the newly gained knowledge with my existing knowledge system. The goal is to index, process, mesh, link (just like in Obsidian tool where you can backlink and create a graph diagram of your information collection). Doing the assignment just means “doing and completing”. At the end of the day, it is just 1 task completed.
I also combine PM and AI courses in the same learning session, alternating between different courses to improve connection between concepts, and reduce burnout.
Use output to consolidate learning
In terms of input/output, online courses actually force you to output by having scheduled formative assessments such as in-video quiz (quickly checking if you really understand what is actively being discussed), weekly quiz (to recap the whole module and checking more in-depth reasoning) as well as programming assignment (hands on practice of what you learnt).
We can enhance our retention by mixing different output channels, such as blogging, collaboration with other learners on an open-sourced project, or teaching the same material to another learner (the Feynman method).
From my own programming experience, I use a project-based approach, by trying to solve an existing problem that is beyond my knowledge. I google, read blogs, find some pointers, read official doc to understand how to use some library modules, implement small POC to test on toy data, then gradually read more blogs to understand how to integrate different features. I seldom program the whole algorithm in one shot (like in Coursera programming assignments). Rather, it is more like piece by piece learning. I also notice that my memory recall for some NLP concepts (e.g., n-gram, perplexity, predicting next word probability, etc.) is way stronger from this class. It is because I implemented the code all by myself without following any prompt, guardrail or lengthy description. This shows that to really sink in knowledge, you have to venture outside the classroom and freely build something on your own.
Improve long term absorption
As my Coursera journey spanned from April to September in 2024, I find it ripe for a checkup now (late Jan 2025). What I did was going through all the courses I took and tried to extract knowledge from them. Here’s what I found:
- anything that requires no deliberate thinking, design and tangible output (that lasts outside the class) is useless afterwards. For example, quizs are useless as they serve as just instant memory recall right after the course, with no solidification of information into knowledge. Unless you quiz yourself by spacing (into the future), everything is down the drain after the quiz.
- as long as something casts no effect on your daily life, or make you change in some way, it will fade away. This is in fact how the brain works, by actively pruning unused neural connections.
- Those with capstone projects in presentations and hands on design work are best in memory retention and practical usage. Specifically, Digital Product Management and AI Product Management both require the learner to iterate through a complete product planning, design (prototype and code), experimenting and testing cycle. Everything is done by hand and key phases are added in each module. The final delivery is a video presentation which utilizes mixed-modal formats of coding, writing, speaking and teaching, further reinforcing the knowledge. These are the things that can be used immediately at work and yield the best ROI.
- coding is the most effective, as you can practice and test something new immediately, and whether it works or not is instantly verifiable. With deliberation, you can apply a new tool or pattern into your current project right after class.
Form your own knowledge system
To improve memory recall, I think it is important to distinguish knowledge domains that require strict memorization, such as learning foreign languages or playing a musical instrument, and those with abstract concepts such as macroeconomics or chemistry that might not easily be applicable to daily life. The former is like scaffolding, things that can be constructed piece by piece by using flashcards and spacing. The latter is harder to integrate into our current knowledge system even after recall, so they stay only in the knowing level that is meaningless. Thus they require totally different strategies.
connect to real life experience
For example, if you learn a term, can you do further research and blog about it? Or use it in a presentation of your own project? Or teach someone? It is useless to quiz yourself “what is an indifference curve?” and recite an explanation from memory. It is more productive to find a real world example (ideally somewhere around you) and explain to someone how an indifference curve is driving the phenomenon you see. Your audience is likely to ask more questions which will hopefully expose gaps in your understanding. By scrutinizing yourself in this way, your newly gained knowledge isn’t just a superficial definition but something that is deeper ingrained in your life, something that you see as an underlying building block of your daily experience.
For blogging, if it’s just like jotting or “book review” of major ideas, it’s like doing a brain dump that will easily be forgotten. It’s just a bit better than quiz as you have to both recall and write, but no better than asking LLMs to summarize a book! In fact, this was my experience with Mind Maps to organize what I gleam from books before. I thought that organizing my notes into Mind Maps wasn’t just neat, it also helped me to restructure what I learn and grasp important concepts. However, what actually happened was that those are still isolated information bits that belongs to the author, and has nothing to do with me. Only after I switched to writing diligently about why is it important, how to apply it for each book, am I able to truly integrate the newly acquired knowledge to become my own.
iterate on a project
I also find that maintaining a consistent output cycle by incorporating your newly gained knowledge into your own project and presenting it to another person, adapting and relaunching is very effective. With this, we are no longer bound to our classroom but become active practitioners, honing our skills with actual outcomes.
Conclusion
I am a bit surprised by my checkup above, but it just confirms what learning experts and psychologists know all along. Finding relevancy and deliberate practicing are key to learning effectively. As I continue my learning journey, I expect to integrate it with public projects that can provide me with feedback and inspiration.