• In first part we explored the statistical model underlying the machine learning problem, and used it to formalize the problem in terms of obtaining the minimum generalization error. By noting that we cannot directly evaluate the generalization error of an ML model, we continued in the second part by establishing a theory that relates this elusive generalization error to another error metric that we can actually evaluate, which is the empirical error. Our final result was that:

    That is: the generalization error (or the risk) $R(h)$ is bounded by the empirical...

  • Last time we concluded by noticing that minimizing the empirical risk (or the training error) is not in itself a solution to the learning problem, it could only be considered a solution if we can guarantee that the difference between the training error and the generalization error (which is also called the generalization gap) is small enough. We formalized such requirement using the probability:

    That is if this probability is small, we can guarantee that the difference between the errors is not much, and hence the learning problem can...

  • Motivation

    Most people when they were kids were fascinated by magicians and magic tricks, they were captivated by what appeared to be reality-defying and riddled with curiosity about how it was being done to the point they wished they become professional magicians as adults. Some of them took that fascination and curiosity to the next level and started to self-learn how to do magic tricks. But no matter how many tricks they knew, they didn’t really master the art of misdirection and the psychological aspects of manipulating the audience’s attention. Some took the extra mile and went to train...

  • Introduction

    In Part 1 we talked about the challenges we could face in doing iterative asynchronous work, and we devised two patterns that we could use to overcome these challenges. Now we turn our attention to the challenges that we would encounter in trying to achieve recursion of asynchronous operations and how we could overcome them. As in the last time, we start our discussion with an example problem to work on, then we investigate the possible solutions.

    Problem 2

    You and your team are creating a module system for your little organization (something that’s very...

  • Introduction

    If any thing made node.js as popular as it is now, it’d would its non-blocking asynchronous I/O calls. You baiscally call an I/O routine, give it a function to call back when your operation is done and you go on continuing whatever business you still have without waiting (blocking) for that operation to finish. How simple is that ?!

    Well, it turns out that it’s not that simple when iterations come in picture. In a lot of situations we may find ourselves in need to iterate over many I/O calls before we can do some specific task (or...

  • Tic-Tac-Toe AI

    When we say the word game in the context of AI we usually don’t mean it in the sense of entertainment games, instead we refer to a more general definition of a game:

    A game is a multi-agent environment in which agents compete and/or cooperate on some specific task(s) while meeting some specific criteria. An agent is referred to as a player.

    With this general definition we could see that a situation, like driving your car to a specific place, is a game. In such a situation, you (as a...