When we think of what it means to learn, we often reference our experiences in school. We equate learning to being spoon fed information from an authority figure, diligently copying down the salient nuggets for future reference. Those nuggets are harvested weeks later, in preparation for an “exam” that is meant to test our learning, but typically only tests our ability to regurgitate what we were “taught”. Shortly after the exam, the information is forgotten, necessitating urgent cramming sessions before the final exam.
Unfortunately, this system of “learning” affects us long after we go to school. All too many of us end up in tutorial hell: cramming in a dozen tutorials on a single topic to “learn” the material. Don’t get me wrong, tutorials have their place in the learning process. After all, I spend a significant amount of time producing them, so I must think they have value. They should not, however, be the focus of your learning activities. They should be a supplement.
Okay, so I’ve argued that our education system is fundamentally broken. Is there a better alternative, then?
Indeed, there exist far better ways of learning, and in this blog post I’m going to share them with you. You won’t find anything on improving your memory, or mnemonics for committing information to memory, rather I’m going to share a framework I’ve developed for learning difficult topics that doesn’t require hours and hours of cramming when the information is needed.
First, a digression. You’re probably familiar with the fact that Toyota is synonymous with quality automobiles. This wasn’t always the case, and Toyota’s leadership in quality engineering came about as a result of implementing something called the Lean methodology. No, it’s not a diet pill to make you shed pounds fast. It’s a system that aims to reduce waste in the manufacturing process, streamlining the flow of information and resources as well as empowering employees to make efficiency changes to their little corner of the process.
It is the reduction of waste that interests us. Toyota realized that stockpiling materials and parts for the assembly and manufacturing of cars is a massive source of waste. Money is spent on materials that won’t be used for quite some time, and space is used to store those materials. The money could be better spent on more immediate needs, and the space can be used to increase production capacity. The solution is called “just in time inventory”. Only order what you need, and with just enough lead time that it arrives precisely when it is needed.
We can apply this just in time principle to learning. Rather than consuming massive quantities of information at one time, and then trying to recall that information later, we only consume enough information to solve the problem immediately in front of us. You may think I’ve performed a bait and switch here, going from learning to problem solving. However, the reality is that learning and problem solving are intimately related. Information that is not immediately used is often forgotten, hence why you need to cram for exams. Moreover, what is the point of learning if not to solve some problem? I suppose that if you’re trying to learn history you’re not really problem solving, but then what is the point of remembering all those dates and events? I promise you, you will not be the cool guy at the party if you can rattle off dates of ancient history.
In addition to just in time information for problem solving, one should think in terms of cause and effect relationships. Back at Intel, we called this model based problem solving, but in graduate school it was just… how we solved problems. The basic idea is that you are solving some larger problem and encounter the inevitable snag, say a bug in your code. The first step to solving this problem is to write down all the possible root causes of that problem. Then rank them in order of likelihood and start attacking the root causes. As you encounter issues that require new information or skills, then look up just enough information to keep making progress on solving the larger problem. This is basically the scientific method applied to problem solving.
You would be surprised at the breadth of problems that can be solved in this manner, and extension the topics you can learn. You may, however, have a question: “How can I get started learning something completely new”? In that case, you pick a toy problem to solve, and apply the above two principles to that toy problem. You will inevitably have to read some basic tutorials or documentation, but this should be just enough to get you started on the problem.
One final tip about learning and problem solving. Sometimes you will come across what seems like an impenetrable barrier. A problem that cannot be solved, a bug that cannot be squashed. Fear not, I have the solution for this as well.
When nothing you try works, when everything has failed, then you have almost certainly assumed something to be true that is not. Alternatively, you may have missed an important nugget of information about the problem and replaced it with some assumption. For (a trivial) example of the former, suppose you’re coding up a deep Q agent and find that it doesn’t learn. You try tinkering with model architectures, learning rates, you try bug hunting the learning function, and nothing works. Everything seems right, but yet the agent won’t learn. What could it be? What have you assumed? You probably assumed that your implementation of the agent memory was correct, yet when you check the code you find an embarrassing bug: you don’t increment the memory counter and so the agent is just overwriting the same memory. It’s like Groundhog day for your agent, and it simply can’t learn from a single memory.
While this was a trivial example, in the video below I relive a painful memory from grad school to share with you a more real world example of assumptions gone wrong. Make sure to check it out before you go.
But before you do, let’s recap the system I’ve laid out for you:
- Only learn what you need to solve the problem right in front of you
- Think of problems in terms of cause and effect. List the cause and effect relationships and then eliminate the causes one one, using the above principle.
- When all else fails, recheck your assumptions. You’ve probably assumed something to be true that isn’t, and it’s manifesting as an unsolvable problem.
I hope this was helpful. I know that this framework has allowed me to learn AI and reinforcement learning with no prior exposure in school. I’m confident it can help you learn also. Make sure to try it and report back on your experiences.
