<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="/thoughtsofaservant/feed.xml" rel="self" type="application/atom+xml" /><link href="/thoughtsofaservant/" rel="alternate" type="text/html" /><updated>2026-06-30T13:37:48+00:00</updated><id>/thoughtsofaservant/feed.xml</id><title type="html">Thoughts of a Servant</title><subtitle>A personal blog documenting toy projects in Statistics, Computer Security, Computer Science, and AI — plus life snippets along the way.</subtitle><author><name>Choonyong Chan</name></author><entry><title type="html">From Apple Trees to Search Trees: Optimising Classical AI to Overcome TicTacToe</title><link href="/thoughtsofaservant/algorithms/from-apple-trees-to-search-trees/" rel="alternate" type="text/html" title="From Apple Trees to Search Trees: Optimising Classical AI to Overcome TicTacToe" /><published>2026-06-30T00:00:00+00:00</published><updated>2026-06-30T00:00:00+00:00</updated><id>/thoughtsofaservant/algorithms/from-apple-trees-to-search-trees</id><content type="html" xml:base="/thoughtsofaservant/algorithms/from-apple-trees-to-search-trees/"><![CDATA[<p><em>This post was proofread with the assistance of AI.</em></p>

<hr />

<p><a href="https://choonyongchan.github.io/thoughtsofaservant/algorithms/i-ruined-tictactoe-for-my-children-and-for-math/">Last time</a>, I handed a 4×4 TicTacToe board to children at a volunteering centre, changed the win condition to 3-in-a-row, and watched a familiar game become genuinely hard. I also showed why that hardness is not superficial: the number of reachable game states grows exponentially, early wins make closed-form analysis intractable, and even counting unique board positions requires a full recursive search.</p>

<p>That is where AI begins. Classical AI’s answer: <strong>treat the game as a search problem</strong>. Define it formally, build a tree of future possibilities, search it intelligently. This post traces how that idea evolved, from the naïve depth-first search a first-year student might attempt, through seven algorithms, to what actually powers modern chess engines.</p>

<hr />

<h2 id="formalising-the-game-start">Formalising the game: START</h2>

<p>Before we can search, we need a language for the problem. Every search problem fits a five-part structure. I remember it as <strong>START</strong>: <strong>S</strong>tate, <strong>T</strong>ransition, <strong>A</strong>ction, <strong>R</strong>eward, <strong>T</strong>erminal.</p>

<p><strong>State</strong> $s$ is a mathematical snapshot of the game at a moment in time. A 3×3 board becomes a 3×3 matrix:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> X | O | .         [["X",  "O",  null],
---+---+---   →     [null, "X",  null],
 . | X | .          [null, null, "O" ]]
---+---+---
 . | . | O
</code></pre></div></div>

<p>State design is an art. Include too little and the agent cannot distinguish situations that call for different moves. Include too much (weather, room temperature, a player’s hydration level…) and the search space balloons needlessly. In AI, a game where both players see the entire board is <em>fully observable</em> (TicTacToe, Chess); one where players see only a portion is <em>partially observable</em> (poker, autonomous driving). TicTacToe is fully observable, which keeps the state simple. The set of all possible states is the <em>State Space</em>. For TicTacToe it is finite, and for larger boards it is exponentially large.</p>

<p><strong>Transition</strong> $T$ describes how state $s$ becomes $s’$: place an X on the top-left cell, and $\text{matrix}[0][0]$ goes from <code class="language-plaintext highlighter-rouge">null</code> to <code class="language-plaintext highlighter-rouge">X</code>. A useful sanity check: the number of null cells must drop by exactly 1 at each transition. Simple invariants like this catch implementation bugs before they compound.</p>

<p><strong>Action</strong> $A(s)$ is the set of legal moves from state $s$, that is the occupation of any one empty cell in TicTacToe. The full set across all states is the <em>Action Space</em>, but not every action is legal in every state.</p>

<p><strong>Reward</strong> $R(s)$ (alternatively, $R(s, a)$/$R(s, a, s’)$) assigns a numerical value to states or sequences so the agent can prefer some paths over others. For TicTacToe, the simplest choice is a terminal score: $+1$ for a win, $-1$ for a loss, $0$ for a draw. I will revisit reward design below (spoiler: it is easier to get wrong than it looks).</p>

<p><strong>Terminal</strong> defines when search stops: $k$ consecutive marks in a line, or a full board with no winner.</p>

<hr />

<h2 id="the-search-tree">The search tree</h2>

<p>With the game formalised, the search becomes a tree. The root is the empty board. Each edge is a legal action. Each child is the resulting board.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>              [empty board]
            /       |        \
     [X:top-L]  [X:centre]  [X:top-R]  ...  (9 branches)
     /      \
[O:top-M]  [O:centre]  ...                   (8 branches each)
</code></pre></div></div>

<p>Let $b$ be the branching factor and $d$ the maximum depth. Total leaf nodes: $O(b^d)$.</p>

<p>For 3×3 TicTacToe: $9! = 362{,}880$ nodes. Fine.<br />
For 5×5 TicTacToe: $25! \approx 1.6 \times 10^{25}$ nodes. Not fine.</p>

<p>Every algorithm below is an attempt to search <em>less</em> of this tree while arriving at the <em>same</em> answer.</p>

<hr />

<h2 id="the-algorithm-chain">The algorithm chain</h2>

<p>These seven algorithms are not a menu of alternatives. They are a chain: each one identifies the precise weakness of its predecessor and fixes it.</p>

<h3 id="1-minimax">1. MiniMax</h3>

<p><img src="https://upload.wikimedia.org/wikipedia/commons/6/6f/Minimax.svg" alt="MiniMax game tree — squares maximise, circles minimise, values propagate from leaves to root" />
<em>Square nodes maximise; circle nodes minimise. Terminal values ($+1$, $0$, $-1$) propagate upward.</em></p>

<p>MiniMax adapts depth-first search for two-player zero-sum games. One player (the <em>maximiser</em>, X) drives the terminal score up; the other (the <em>minimiser</em>, O) drives it down. Terminal nodes receive $+1$, $-1$, or $0$. Internal nodes take the max or min of their children:</p>

\[v(s) = \begin{cases}
R(s) &amp; \text{if } s \text{ is terminal} \\
\max_{a \in A(s)}\, v(T(s, a)) &amp; \text{if maximiser's turn} \\
\min_{a \in A(s)}\, v(T(s, a)) &amp; \text{if minimiser's turn}
\end{cases}\]

<p>The result is optimal play against a perfectly rational opponent. The cost: every node, $O(b^d)$ of them (with patience — lots of patience).</p>

<h3 id="2-alpha-beta-pruning">2. Alpha-Beta Pruning</h3>

<p><img src="https://upload.wikimedia.org/wikipedia/commons/9/91/AB_pruning.svg" alt="Alpha-Beta pruning — crossed-out branches are safely skipped" />
<em>Crossed-out subtrees cannot change the root value regardless of their contents.</em></p>

<p>Alpha-Beta pruning keeps MiniMax’s exact guarantees while ignoring subtrees that cannot affect the result. It tracks two bounds:</p>
<ul>
  <li>$\alpha$ — the best score the maximiser has secured so far (lower bound)</li>
  <li>$\beta$ — the best score the minimiser has secured so far (upper bound)</li>
</ul>

<p>When $\beta \leq \alpha$, the branch is cut. The minimiser would never allow a result this good for the maximiser, so searching further is pointless.</p>

<p>With perfect move ordering — best moves explored first — complexity drops from $O(b^d)$ to $O(b^{d/2})$. A search that would take 100 hours now takes 10.</p>

<h3 id="3-negamax">3. NegaMax</h3>

<p>A sharp observation: for zero-sum two-player games, the maximiser and minimiser are doing the same thing in opposite directions. If we negate the returned score whenever the active player switches, both players become maximisers of their own perspective:</p>

\[v(s) = \max_{a \in A(s)}\bigl(-v(T(s, a))\bigr)\]

<p>Same search tree explored. Fewer variables. One recursive case instead of two.</p>

<p>(As software engineers, we know how it feels when the codebase becomes excessively complicated. The KISS principle is just like a first kiss, bringing warmth and comfort to our hearts.)</p>

<h3 id="4-scaled-rewards-a-warning">4. Scaled Rewards: a warning</h3>

<p>An appealing idea: reward faster wins more than slower ones. Score a 5-move win as $+1.0$ and a 7-move win as $+0.8$, incentivising the search toward quicker victories.</p>

<p>The problem is subtle. In plain MiniMax with reward space ${-1, 0, +1}$, finding a $+1$ terminal means the absolute best outcome is secured. Search at that node stops. With scaled rewards, finding $+0.8$ does not justify stopping: there might be a $+1.0$ elsewhere via a faster path. The algorithm must now confirm it has the <em>fastest</em> win, not merely <em>a</em> win.</p>

<p>The heuristic changed the question from “find any win” to “find the fastest win.” The latter is strictly harder. A heuristic designed to shrink the search made it larger. Measure before assuming.</p>

<h3 id="5-negascout">5. NegaScout</h3>

<p>Alpha-Beta prunes when $\beta \leq \alpha$. NegaScout increases the chance of this condition holding.</p>

<p>If the first move explored at a node really is the best, then every sibling only needs to be confirmed <em>worse</em>, the exact alpha-beta values are unnecessary. NegaScout searches siblings with a <em>null window</em> $[\alpha,\, \alpha+1]$ rather than the full window $[\alpha, \beta]$:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Standard window:   [α ─────────────────── β]
Null window:       [α ─ α+1]
</code></pre></div></div>

<p>A null-window search triggers far more cutoffs. If the sibling falls inside the window, it is confirmed inferior (= move on). If it exceeds the window, the first-move assumption was wrong and a full re-search is needed.</p>

<p>NegaScout is fast when move ordering is good and degrades gracefully when it is not. Its efficiency is entirely dependent on exploring the best move first.</p>

<h3 id="6-mtdf">6. MTD(f)</h3>

<p>MTD(f) takes the null-window idea to its conclusion: use <em>only</em> null-window searches, always, via binary search over the true minimax value.</p>

<p>Starting from a guess $f$, it calls NegaMax with window $[f, f+1]$. Each call returns a lower or upper bound on the true value. Successive calls narrow the interval until the bounds meet:</p>

\[\text{repeat until } \ell = u: \quad \text{NegaMax}([f,\, f+1]) \to \text{new bound; update } \ell,\, u,\, f\]

<p>Because every call is a null-window search, every call benefits from maximum pruning. The tradeoff: multiple passes revisit parts of the tree. A <em>transposition table</em>, a cache of already-seen positions, is not optional; without it, re-searches undo the savings entirely.</p>

<h3 id="7-best-node-search">7. Best Node Search</h3>

<p>Rather than finding the <em>value</em> of the best move, why not just identify <em>which move is best</em>? That is a strictly weaker question, and sometimes easier to answer.</p>

<p>Best Node Search iteratively guesses a threshold and counts how many moves score above and below it:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Guess 0.23 →  11 worse,  9 better
Guess 0.43 →  20 worse,  0 better
Guess 0.33 →  18 worse,  2 better
Guess 0.38 →  19 worse,  1 better  ← one candidate remains above threshold
</code></pre></div></div>

<p>Once only one move exceeds the threshold, that move is optimal by elimination. BNS achieves strong performance in large search spaces precisely because it stops asking “how good is this?” once the contest between top candidates is settled.</p>

<hr />

<h2 id="iterative-deepening-the-crosscutting-technique">Iterative Deepening: the crosscutting technique</h2>

<p>NegaScout requires good move ordering. MTD(f) and BNS need a good initial value. Iterative Deepening supplies both.</p>

<p>Run a complete search to depth 1. Use the best move found. Search to depth 2 using that result. Continue.</p>

<p>Revisiting shallower depths looks wasteful, but the overhead factor is only $\frac{b}{b-1}$, a constant — because the final depth dominates the total node count exponentially. The payoff: the best move from depth $d$ becomes the first move explored at depth $d+1$, giving NegaScout exactly the move ordering it needs; the minimax value from depth $d$ becomes MTD(f)’s and BNS’s starting guess at depth $d+1$.</p>

<p>Iterative Deepening doesn’t make any single algorithm asymptotically faster. It makes the whole family work better together.</p>

<hr />

<h2 id="putting-numbers-to-the-theory">Putting numbers to the theory</h2>

<p>All algorithms were run as both players from the opening move, with a 1-hour wall-clock timeout. States visited counts total traversals including re-visits through the transposition table; timeout entries show progress at cutoff, not the full game-tree size.</p>

<table>
  <thead>
    <tr>
      <th>Algorithm</th>
      <th>3×3, k=3</th>
      <th>4×4, k=3</th>
      <th>4×4, k=4</th>
      <th>5×5, k=5</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td> </td>
      <td><em>states / s</em></td>
      <td><em>states / s</em></td>
      <td><em>states / s</em></td>
      <td><em>states / s</em></td>
    </tr>
    <tr>
      <td>MiniMax</td>
      <td>618,184 / 8.57</td>
      <td>TIMEOUT</td>
      <td>TIMEOUT</td>
      <td>TIMEOUT</td>
    </tr>
    <tr>
      <td>MiniMax with Alpha-Beta Pruning</td>
      <td>21,652 / 0.30</td>
      <td>42,340 / 19.13</td>
      <td>TIMEOUT</td>
      <td>TIMEOUT</td>
    </tr>
    <tr>
      <td>NegaMax</td>
      <td>24,698 / 0.34</td>
      <td>282,469 / 132.45</td>
      <td>TIMEOUT</td>
      <td>TIMEOUT</td>
    </tr>
    <tr>
      <td>NegaScout</td>
      <td>45,801 / 0.68</td>
      <td>390,989 / 350.56</td>
      <td>TIMEOUT</td>
      <td>TIMEOUT</td>
    </tr>
    <tr>
      <td>Best Node Search (BNS)</td>
      <td>58,226 / 0.81</td>
      <td>2,553,446 / 2809.92</td>
      <td>TIMEOUT</td>
      <td>TIMEOUT</td>
    </tr>
    <tr>
      <td>Best Node Search + Iterative Deepening</td>
      <td>7,171 / 0.17</td>
      <td>125,484 / 42.51</td>
      <td>958,872 / 376.01</td>
      <td>TIMEOUT (11.4M)</td>
    </tr>
    <tr>
      <td>MTD(f)</td>
      <td><strong>1,817 / 0.04</strong></td>
      <td>113,351 / 6.29</td>
      <td><strong>196,677 / 19.43</strong></td>
      <td>TIMEOUT (9.7M)</td>
    </tr>
    <tr>
      <td>MTD(f) + Iterative Deepening</td>
      <td>5,297 / 0.15</td>
      <td><strong>40,377 / 13.39</strong></td>
      <td>265,719 / 147.50</td>
      <td>TIMEOUT (13.7M)</td>
    </tr>
  </tbody>
</table>

<p><strong>n is the binding constraint, not k.</strong> The jump from 3×3 to 4×4 with k=n kills every algorithm except MTD(f). Alpha-Beta Pruning goes from 21K states to timeout; NegaScout and BNS never finish. Dropping k from 4 to 3 on the same 4×4 board rescues Alpha-Beta Pruning: 42K states in 19 seconds instead of timeout. At 5×5, k barely matters — the 25-cell branching factor overwhelms any depth reduction from an earlier win condition, and every exact solver times out regardless of k.</p>

<p><strong>MTD(f) wins by a large margin — even accounting for re-visits.</strong> MTD(f) makes multiple passes over the tree, so its states-visited count includes states looked up more than once via the transposition table. Even so, it visits 1,817 states on 3×3 against Alpha-Beta Pruning’s 21,652: a 12× reduction. On 4×4 k=4, it is the only algorithm to finish at all, in 19 seconds against BNS + Iterative Deepening’s 376. The null-window bisection prunes so aggressively on each pass that the multi-pass overhead is more than recovered.</p>

<p><strong>Theory and benchmarks diverge for NegaScout and BNS.</strong> Both are theoretically stronger than Alpha-Beta Pruning in the general case. On 4×4 k=3, Alpha-Beta Pruning visits 42K states in 19 seconds. NegaScout visits 391K in 350 seconds. BNS visits 2.5M in 2,800 seconds. The bottleneck is move ordering: both algorithms incur large overhead when moves aren’t explored best-first, because the null-window re-searches trigger frequently. A simpler algorithm with decent ordering outruns a sophisticated one without it. Theory describes best-case behaviour; these benchmarks measure actual behaviour.</p>

<p>Note that timeout occurred even at a very shallow depth, limiting our evaluation of iterative deepening. I believe that with more resources and a deeper search, algorithms using iterative deepening (i.e. MTD(f) and Best Node Search for our experiment) would demonstrate their asymptotic behaviour and the relative effectiveness of their strategies more strongly.</p>

<hr />

<h2 id="what-i-learnt">What I learnt</h2>

<p><strong>1. Progress is cumulative.</strong> MiniMax feels almost trivially simple: explore every state, propagate values upward. But Alpha-Beta has no foundation without it. NegaScout has no foundation without Alpha-Beta. MTD(f) builds on NegaScout. Each algorithm is a targeted fix to a precisely identified weakness, not a replacement of what came before. The pattern generalises: the great discoveries in any field are rarely isolated flashes of genius but iterations on a preceding idea that someone took seriously enough to examine closely. I have learnt not to trivialise incremental wins.</p>

<p><strong>2. Heuristics are not the enemy; bad heuristics are.</strong> The Scaled Rewards example is easy to misread as a reason to distrust heuristics. The actual failure was simpler: the heuristic changed what was being optimised without anyone noticing. That is a calibration problem, not an indictment of the approach.</p>

<p>When accuracy is the bottleneck: medical diagnostics, logistics routing, you search more thoroughly. When time is the bottleneck: autonomous driving, real-time systems, a good-enough answer in 50ms beats a perfect answer in 5 seconds. Leaders face this daily: dozens of decisions, limited time, imperfect information. A good rule of thumb makes choices fast, simple, and consistent; the risk is when the rule drifts from the actual objective. In pathfinding algorithms, admissible heuristics carry a formal guarantee that the estimate never overstates the true cost, a concrete way to ask “is this heuristic any good?” The question is not whether to use heuristics; time and real-world complexity do not give you a choice, and neither does the autonomous car. What matters is whether your heuristics reflect what you actually care about.</p>

<p><strong>3. Domain knowledge is the highest-leverage input.</strong> Domain-agnostic improvements (Alpha-Beta, null-window search) work on any game tree. Domain-aware heuristics, ones that know the specific game, reach gains no general technique can. The world is too complex to reduce entirely to heuristics, but being domain-aware consistently outperforms being broadly clever. I am encouraged to go deep, to read widely, to talk to people who know things I do not. Even though I will be a master of none, but at least I will be a jack of many trades: a childcare volunteer teacher by day and an algorithm enthusiast by night. 🙂</p>

<hr />

<h2 id="food-for-thought">Food for thought</h2>

<ol>
  <li>Is finding a closed form for TicTacToe’s state-count recurrence truly intractable, or is it just convoluted? The distinction matters for how much effort is worth spending on it.</li>
  <li>Beyond instant-win heuristics and terminal reward, what other low-cost heuristics provably speed up TicTacToe search? Killer-move heuristics (moves that constrain the opponent’s future options) seem worth investigating.</li>
  <li>Do these observations hold for $p$-player $n \times n$ TicTacToe, or $n$-dimensional boards? Intuition says pruning becomes less effective as the number of players grows, but by how much?</li>
</ol>

<hr />

<h2 id="references">References</h2>

<p>Helper stubs assumed throughout:</p>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">is_terminal</span><span class="p">(</span><span class="n">state</span><span class="p">):</span> <span class="p">...</span>
<span class="k">def</span> <span class="nf">reward</span><span class="p">(</span><span class="n">state</span><span class="p">):</span> <span class="p">...</span>         <span class="c1"># from current player's perspective
</span><span class="k">def</span> <span class="nf">actions</span><span class="p">(</span><span class="n">state</span><span class="p">):</span> <span class="p">...</span>        <span class="c1"># list of legal moves from this state
</span><span class="k">def</span> <span class="nf">apply</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">action</span><span class="p">):</span> <span class="p">...</span>  <span class="c1"># transition function, returns the new state after the move
</span><span class="n">INF</span> <span class="o">=</span> <span class="nb">float</span><span class="p">(</span><span class="s">'inf'</span><span class="p">)</span>
</code></pre></div></div>

<h3 id="minimax">MiniMax</h3>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">minimax</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">is_maximising</span><span class="p">):</span>
    <span class="k">if</span> <span class="n">is_terminal</span><span class="p">(</span><span class="n">state</span><span class="p">):</span>
        <span class="k">return</span> <span class="n">reward</span><span class="p">(</span><span class="n">state</span><span class="p">)</span>                          <span class="c1"># base case: score the outcome
</span>    <span class="n">scores</span> <span class="o">=</span> <span class="p">[</span><span class="n">minimax</span><span class="p">(</span><span class="nb">apply</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">a</span><span class="p">),</span> <span class="ow">not</span> <span class="n">is_maximising</span><span class="p">)</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">actions</span><span class="p">(</span><span class="n">state</span><span class="p">)]</span>
    <span class="k">return</span> <span class="nb">max</span><span class="p">(</span><span class="n">scores</span><span class="p">)</span> <span class="k">if</span> <span class="n">is_maximising</span> <span class="k">else</span> <span class="nb">min</span><span class="p">(</span><span class="n">scores</span><span class="p">)</span>  <span class="c1"># X maximises, O minimises
</span></code></pre></div></div>

<h3 id="alpha-beta-pruning">Alpha-Beta Pruning</h3>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">alpha_beta</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">alpha</span><span class="p">,</span> <span class="n">beta</span><span class="p">,</span> <span class="n">is_maximising</span><span class="p">):</span>
    <span class="k">if</span> <span class="n">is_terminal</span><span class="p">(</span><span class="n">state</span><span class="p">):</span>
        <span class="k">return</span> <span class="n">reward</span><span class="p">(</span><span class="n">state</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">is_maximising</span><span class="p">:</span>
        <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">actions</span><span class="p">(</span><span class="n">state</span><span class="p">):</span>
            <span class="n">alpha</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">alpha</span><span class="p">,</span> <span class="n">alpha_beta</span><span class="p">(</span><span class="nb">apply</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">a</span><span class="p">),</span> <span class="n">alpha</span><span class="p">,</span> <span class="n">beta</span><span class="p">,</span> <span class="bp">False</span><span class="p">))</span>
            <span class="k">if</span> <span class="n">alpha</span> <span class="o">&gt;=</span> <span class="n">beta</span><span class="p">:</span>
                <span class="k">break</span>              <span class="c1"># β cut-off: minimiser won't allow this
</span>        <span class="k">return</span> <span class="n">alpha</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">actions</span><span class="p">(</span><span class="n">state</span><span class="p">):</span>
            <span class="n">beta</span> <span class="o">=</span> <span class="nb">min</span><span class="p">(</span><span class="n">beta</span><span class="p">,</span> <span class="n">alpha_beta</span><span class="p">(</span><span class="nb">apply</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">a</span><span class="p">),</span> <span class="n">alpha</span><span class="p">,</span> <span class="n">beta</span><span class="p">,</span> <span class="bp">True</span><span class="p">))</span>
            <span class="k">if</span> <span class="n">beta</span> <span class="o">&lt;=</span> <span class="n">alpha</span><span class="p">:</span>
                <span class="k">break</span>              <span class="c1"># α cut-off: maximiser already has better
</span>        <span class="k">return</span> <span class="n">beta</span>
</code></pre></div></div>

<h3 id="negamax">NegaMax</h3>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Both players maximise from their own perspective; negate the child's score
# to flip from opponent's view back to the current player's view.
</span><span class="k">def</span> <span class="nf">negamax</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=-</span><span class="n">INF</span><span class="p">,</span> <span class="n">beta</span><span class="o">=</span><span class="n">INF</span><span class="p">):</span>
    <span class="k">if</span> <span class="n">is_terminal</span><span class="p">(</span><span class="n">state</span><span class="p">):</span>
        <span class="k">return</span> <span class="n">reward</span><span class="p">(</span><span class="n">state</span><span class="p">)</span>           <span class="c1"># reward must be from current player's POV
</span>    <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">actions</span><span class="p">(</span><span class="n">state</span><span class="p">):</span>
        <span class="n">score</span> <span class="o">=</span> <span class="o">-</span><span class="n">negamax</span><span class="p">(</span><span class="nb">apply</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">a</span><span class="p">),</span> <span class="o">-</span><span class="n">beta</span><span class="p">,</span> <span class="o">-</span><span class="n">alpha</span><span class="p">)</span>  <span class="c1"># swap and negate bounds
</span>        <span class="n">alpha</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">alpha</span><span class="p">,</span> <span class="n">score</span><span class="p">)</span>
        <span class="k">if</span> <span class="n">alpha</span> <span class="o">&gt;=</span> <span class="n">beta</span><span class="p">:</span>
            <span class="k">break</span>                      <span class="c1"># same cut-off logic as alpha-beta, unified
</span>    <span class="k">return</span> <span class="n">alpha</span>
</code></pre></div></div>

<h3 id="scaled-rewards">Scaled Rewards</h3>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Standard: α-β can stop immediately on finding +1 — it's the ceiling
</span><span class="k">def</span> <span class="nf">reward</span><span class="p">(</span><span class="n">state</span><span class="p">):</span>
    <span class="k">if</span> <span class="n">winner</span><span class="p">(</span><span class="n">state</span><span class="p">)</span> <span class="o">==</span> <span class="n">MAX</span><span class="p">:</span> <span class="k">return</span> <span class="o">+</span><span class="mi">1</span>
    <span class="k">if</span> <span class="n">winner</span><span class="p">(</span><span class="n">state</span><span class="p">)</span> <span class="o">==</span> <span class="n">MIN</span><span class="p">:</span> <span class="k">return</span> <span class="o">-</span><span class="mi">1</span>
    <span class="k">return</span> <span class="mi">0</span>

<span class="c1"># Scaled: faster wins score higher — intended to incentivise quick victories
</span><span class="k">def</span> <span class="nf">reward_scaled</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">depth</span><span class="p">):</span>
    <span class="k">if</span> <span class="n">winner</span><span class="p">(</span><span class="n">state</span><span class="p">)</span> <span class="o">==</span> <span class="n">MAX</span><span class="p">:</span> <span class="k">return</span> <span class="o">+</span><span class="mi">1</span> <span class="o">-</span> <span class="mf">0.1</span> <span class="o">*</span> <span class="n">depth</span>
    <span class="k">if</span> <span class="n">winner</span><span class="p">(</span><span class="n">state</span><span class="p">)</span> <span class="o">==</span> <span class="n">MIN</span><span class="p">:</span> <span class="k">return</span> <span class="o">-</span><span class="mi">1</span> <span class="o">+</span> <span class="mf">0.1</span> <span class="o">*</span> <span class="n">depth</span>
    <span class="k">return</span> <span class="mi">0</span>
<span class="c1"># Problem: +0.8 is no longer proof of optimality.
# The search must keep looking for a faster +1.0 — making it strictly larger.
</span></code></pre></div></div>

<h3 id="negascout">NegaScout</h3>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">negascout</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=-</span><span class="n">INF</span><span class="p">,</span> <span class="n">beta</span><span class="o">=</span><span class="n">INF</span><span class="p">):</span>
    <span class="k">if</span> <span class="n">is_terminal</span><span class="p">(</span><span class="n">state</span><span class="p">):</span>
        <span class="k">return</span> <span class="n">reward</span><span class="p">(</span><span class="n">state</span><span class="p">)</span>
    <span class="n">window</span> <span class="o">=</span> <span class="n">beta</span>
    <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">a</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">actions</span><span class="p">(</span><span class="n">state</span><span class="p">)):</span>
        <span class="n">score</span> <span class="o">=</span> <span class="o">-</span><span class="n">negascout</span><span class="p">(</span><span class="nb">apply</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">a</span><span class="p">),</span> <span class="o">-</span><span class="n">window</span><span class="p">,</span> <span class="o">-</span><span class="n">alpha</span><span class="p">)</span>
        <span class="k">if</span> <span class="n">i</span> <span class="o">&gt;</span> <span class="mi">0</span> <span class="ow">and</span> <span class="n">alpha</span> <span class="o">&lt;</span> <span class="n">score</span> <span class="o">&lt;</span> <span class="n">beta</span><span class="p">:</span>               <span class="c1"># null-window was too narrow
</span>            <span class="n">score</span> <span class="o">=</span> <span class="o">-</span><span class="n">negascout</span><span class="p">(</span><span class="nb">apply</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">a</span><span class="p">),</span> <span class="o">-</span><span class="n">beta</span><span class="p">,</span> <span class="o">-</span><span class="n">score</span><span class="p">)</span>  <span class="c1"># full re-search
</span>        <span class="n">alpha</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">alpha</span><span class="p">,</span> <span class="n">score</span><span class="p">)</span>
        <span class="k">if</span> <span class="n">alpha</span> <span class="o">&gt;=</span> <span class="n">beta</span><span class="p">:</span>
            <span class="k">break</span>
        <span class="n">window</span> <span class="o">=</span> <span class="n">alpha</span> <span class="o">+</span> <span class="mi">1</span>   <span class="c1"># narrow to null window: siblings only need to beat alpha
</span>    <span class="k">return</span> <span class="n">alpha</span>
</code></pre></div></div>

<h3 id="mtdf">MTD(f)</h3>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Converges on the true minimax value via repeated null-window passes.
# Each pass returns a bound; bounds tighten until they meet.
</span><span class="k">def</span> <span class="nf">mtdf</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">f</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span>                          <span class="c1"># f is the initial value guess
</span>    <span class="n">lower</span><span class="p">,</span> <span class="n">upper</span> <span class="o">=</span> <span class="o">-</span><span class="n">INF</span><span class="p">,</span> <span class="n">INF</span>
    <span class="k">while</span> <span class="n">lower</span> <span class="o">&lt;</span> <span class="n">upper</span><span class="p">:</span>
        <span class="n">beta</span> <span class="o">=</span> <span class="nb">max</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="n">lower</span> <span class="o">+</span> <span class="mi">1</span><span class="p">)</span>
        <span class="n">f</span> <span class="o">=</span> <span class="n">negamax</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">beta</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="n">beta</span><span class="p">)</span>     <span class="c1"># null-window [β-1, β]
</span>        <span class="k">if</span> <span class="n">f</span> <span class="o">&lt;</span> <span class="n">beta</span><span class="p">:</span>
            <span class="n">upper</span> <span class="o">=</span> <span class="n">f</span>                          <span class="c1"># returned below window: upper bound
</span>        <span class="k">else</span><span class="p">:</span>
            <span class="n">lower</span> <span class="o">=</span> <span class="n">f</span>                          <span class="c1"># returned at/above window: lower bound
</span>    <span class="k">return</span> <span class="n">f</span>
<span class="c1"># Transposition table is mandatory: without caching, re-searches undo all savings.
</span></code></pre></div></div>

<h3 id="best-node-search">Best Node Search</h3>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Binary search over move quality: eliminate candidates below the threshold each round.
</span><span class="k">def</span> <span class="nf">bns</span><span class="p">(</span><span class="n">state</span><span class="p">):</span>
    <span class="n">candidates</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">actions</span><span class="p">(</span><span class="n">state</span><span class="p">))</span>
    <span class="n">lower</span><span class="p">,</span> <span class="n">upper</span> <span class="o">=</span> <span class="o">-</span><span class="n">INF</span><span class="p">,</span> <span class="n">INF</span>
    <span class="k">while</span> <span class="nb">len</span><span class="p">(</span><span class="n">candidates</span><span class="p">)</span> <span class="o">&gt;</span> <span class="mi">1</span><span class="p">:</span>
        <span class="n">guess</span> <span class="o">=</span> <span class="p">(</span><span class="n">lower</span> <span class="o">+</span> <span class="n">upper</span><span class="p">)</span> <span class="o">/</span> <span class="mi">2</span> <span class="c1"># binary search, better alternate methods to narrow down guess can be implemented.
</span>        <span class="c1"># null-window test: does this move beat the current guess?
</span>        <span class="n">better</span> <span class="o">=</span> <span class="p">[</span><span class="n">a</span> <span class="k">for</span> <span class="n">a</span> <span class="ow">in</span> <span class="n">candidates</span>
                  <span class="k">if</span> <span class="o">-</span><span class="n">negamax</span><span class="p">(</span><span class="nb">apply</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">a</span><span class="p">),</span> <span class="o">-</span><span class="n">guess</span> <span class="o">-</span> <span class="mi">1</span><span class="p">,</span> <span class="o">-</span><span class="n">guess</span><span class="p">)</span> <span class="o">&gt;</span> <span class="n">guess</span><span class="p">]</span>
        <span class="k">if</span> <span class="n">better</span><span class="p">:</span>
            <span class="n">lower</span> <span class="o">=</span> <span class="n">guess</span>
            <span class="n">candidates</span> <span class="o">=</span> <span class="n">better</span>           <span class="c1"># keep only moves above the threshold
</span>        <span class="k">else</span><span class="p">:</span>
            <span class="n">upper</span> <span class="o">=</span> <span class="n">guess</span>                 <span class="c1"># all moves below; lower the bar
</span>    <span class="k">return</span> <span class="n">candidates</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>                  <span class="c1"># last survivor is the best move
</span></code></pre></div></div>

<h3 id="iterative-deepening">Iterative Deepening</h3>

<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">iterative_deepening</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">max_depth</span><span class="p">):</span>
    <span class="n">best_move</span> <span class="o">=</span> <span class="bp">None</span>
    <span class="k">for</span> <span class="n">depth</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">max_depth</span> <span class="o">+</span> <span class="mi">1</span><span class="p">):</span>
        <span class="n">best_move</span> <span class="o">=</span> <span class="n">best_move_at_depth</span><span class="p">(</span><span class="n">state</span><span class="p">,</span> <span class="n">depth</span><span class="p">)</span>
        <span class="c1"># best_move from depth d → first move explored at depth d+1 (move ordering)
</span>        <span class="c1"># minimax value from depth d → starting guess f for MTD(f)/BNS at depth d+1
</span>    <span class="k">return</span> <span class="n">best_move</span>
<span class="c1"># Overhead factor: b/(b-1) — constant, because the final depth dominates node count.
</span></code></pre></div></div>]]></content><author><name>Choonyong Chan</name></author><category term="Algorithms" /><category term="Algorithms" /><category term="AI" /><category term="Classical AI" /><category term="TicTacToe" /><summary type="html"><![CDATA[This post was proofread with the assistance of AI.]]></summary></entry><entry><title type="html">I Ruined TicTacToe for My Children, and for Math 😭</title><link href="/thoughtsofaservant/algorithms/i-ruined-tictactoe-for-my-children-and-for-math/" rel="alternate" type="text/html" title="I Ruined TicTacToe for My Children, and for Math 😭" /><published>2026-06-17T00:00:00+00:00</published><updated>2026-06-17T00:00:00+00:00</updated><id>/thoughtsofaservant/algorithms/i-ruined-tictactoe-for-my-children-and-for-math</id><content type="html" xml:base="/thoughtsofaservant/algorithms/i-ruined-tictactoe-for-my-children-and-for-math/"><![CDATA[<p><em>This post was proofread with the assistance of AI.</em></p>

<hr />

<p>All I did was draw one extra row and one extra column. That was enough to ruin TicTacToe.</p>

<p>The children at my volunteering centre love the normal 3x3 game. It is quick, familiar, and perfect for filling a spare couple of minutes with a primary school child. Then I gave them a 4x4 board. At first, they treated it like the same game with more space. After a few rounds, they noticed something annoying; it was much easier to block, much harder to finish, and the old tricks no longer worked cleanly.</p>

<p>So I made it worse. I kept the board at 4x4, but changed the win condition to 3 in a row.</p>

<p>That tiny rule change opened the floodgates. Suddenly every corner mattered. A harmless-looking move could threaten a row, a column, and a diagonal. The children got louder, more careful, and more competitive. I had turned a childhood game into a small mathematical headache.</p>

<p>And that was the point.</p>

<h2 id="a-small-game-with-serious-ai-bones">A Small Game With Serious AI Bones</h2>

<p>TicTacToe is useful because it is simple enough to hold in your head, but rich enough to expose real ideas in Classical AI.</p>

<p>So what do we understand about TicTacToe as an AI environment? It is finite; the board eventually fills. It is deterministic; placing a mark has no randomness or chance attached to it. It is a perfect-information game; both players see the whole board at all times. It is adversarial; every good move for one player is bad news for the other.</p>

<p>That combination makes TicTacToe a clean playground for AI as search.</p>

<p>In standard 3x3 TicTacToe, the rules are fixed. X starts. Players alternate. Whoever gets 3 in a row wins. If the board fills without a winner, the game is a draw. Exhaustive search reveals (to no one’s surprise) that under perfect play, neither player can force a win, so the result is a draw.</p>

<p>But once we generalise the game, the neat little toy starts to misbehave.</p>

<p>Let the board be <code class="language-plaintext highlighter-rouge">n x n</code>. Let <code class="language-plaintext highlighter-rouge">k</code> be the number of consecutive marks needed to win, where <code class="language-plaintext highlighter-rouge">k &lt;= n</code>. The childhood version is just one setting: <code class="language-plaintext highlighter-rouge">n = 3, k = 3</code>. A 4x4 game where you need the full row is <code class="language-plaintext highlighter-rouge">n = 4, k = 4</code>. The more chaotic version I gave the children is <code class="language-plaintext highlighter-rouge">n = 4, k = 3</code>.</p>

<p>That notation looks innocent. It is not.</p>

<p>Here is the difference in one position.</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>X X . .
O . . .
. . . .
. . . .
</code></pre></div></div>

<p>In <code class="language-plaintext highlighter-rouge">n = 4, k = 4</code>, X has only made a start. Two X marks are not close to winning yet. In <code class="language-plaintext highlighter-rouge">n = 4, k = 3</code>, X can win immediately by playing the third square in the top row:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>X X X .
O . . .
. . . .
. . . .
</code></pre></div></div>

<p>The board size stayed the same. The meaning of danger changed.</p>

<h2 id="computing-tictactoe-requires-recursion">Computing TicTacToe Requires Recursion</h2>

<p>Suppose both players are young children playing without hints or strategy. At every turn, the current player chooses one legal empty square uniformly at random.</p>

<p>Here is the question: what is the probability that X wins?</p>

<p>For 3x3 TicTacToe, it is tempting to count board patterns directly. That temptation fades fast. To know whether X wins, we cannot only inspect the final board. We also need to know whether O would have won earlier, because TicTacToe stops the moment someone completes a line. A board that looks possible as a final arrangement may never appear in a real game, since the game may have ended two moves before.</p>

<p>There is no simple one-pass shortcut that respects turn order, illegal states, and early stopping. That pushes us into recursion.</p>

<p>Let <code class="language-plaintext highlighter-rouge">P_X(s)</code> be the probability that X eventually wins from state <code class="language-plaintext highlighter-rouge">s</code>. If <code class="language-plaintext highlighter-rouge">s</code> is already a terminal X win, <code class="language-plaintext highlighter-rouge">P_X(s) = 1</code>. If <code class="language-plaintext highlighter-rouge">s</code> is an O win or a draw, <code class="language-plaintext highlighter-rouge">P_X(s) = 0</code>. Otherwise, the current player has some set of legal moves, and random play averages over the child states:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>P_X(s) = (1 / n_s) * sum over child states s' of P_X(s')
</code></pre></div></div>

<p>where <code class="language-plaintext highlighter-rouge">s'</code> represents each child state after each legal move, and <code class="language-plaintext highlighter-rouge">n_s</code> is the number of legal moves from state <code class="language-plaintext highlighter-rouge">s</code>.</p>

<p>This is a tiny equation with a nasty implication. To evaluate the root, the empty board, we must evaluate the children. To evaluate the children, we evaluate their children. The game becomes a search tree.</p>

<p>For example, from an empty 3x3 board, random X has 9 possible first moves. After X chooses one, random O has 8 possible replies. After that, X has 7. Even before we ask who is playing well, the tree begins to branch:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>empty board
  -&gt; X in top-left
    -&gt; O in top-middle
    -&gt; O in top-right
    -&gt; ...
  -&gt; X in center
    -&gt; O in top-left
    -&gt; O in top-middle
    -&gt; ...
</code></pre></div></div>

<p>Recursion is just the clean way to say: solve each smaller board, then average the answers back up.</p>

<p>For the normal 3x3 board, this is still manageable. If both players choose uniformly random legal moves, X wins about <code class="language-plaintext highlighter-rouge">58.49%</code> of games, O wins about <code class="language-plaintext highlighter-rouge">28.81%</code>, and the game draws about <code class="language-plaintext highlighter-rouge">12.70%</code>.</p>

<p>That first-player advantage is not magic. X always moves first, so X gets the first chance to complete the fifth move of the game, the earliest possible winning point in 3x3 TicTacToe. Random O does not defend well enough to erase that advantage.</p>

<h2 id="small-things-add-up-very-quickly">Small Things Add Up, Very Quickly</h2>

<p>A 3x3 board has <code class="language-plaintext highlighter-rouge">3^9 = 19,683</code> raw assignments if every square can be empty, X, or O. That number is already larger than most people expect, but it still fits comfortably on a slide.</p>

<p>The legal game is smaller in one sense and larger in another.</p>

<p>It is smaller because many raw assignments are illegal. X and O must alternate turns. The game stops early when someone wins. If we count reachable board states with early stopping, standard TicTacToe has <code class="language-plaintext highlighter-rouge">5,478</code> states.</p>

<p>It is larger because search algorithms care about paths, not only board snapshots. The same board can be reached through different move orders. Standard 3x3 TicTacToe has <code class="language-plaintext highlighter-rouge">255,168</code> terminal game sequences.</p>

<p>Now move from 3x3 to 4x4.</p>

<p>The raw board assignments jump from <code class="language-plaintext highlighter-rouge">3^9 = 19,683</code> to <code class="language-plaintext highlighter-rouge">3^16 = 43,046,721</code>. If we ignore early wins and simply count full move orders, a 4x4 board has:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>16! = 20,922,789,888,000
</code></pre></div></div>

<p>That is more than twenty trillion full-length move orders.</p>

<p>Of course, real games may stop early. That helps. It also makes the mathematics uglier, because early stopping depends on the exact sequence of previous moves. A win is not just a property of how many marks exist; it is a property of where they were placed and whether the game should have ended earlier. It is this property that makes finding a closed-form property intractable.</p>

<p>This is the first lesson algorithm learners should take seriously: small rule systems can produce huge search spaces without looking complicated. Think of the Travelling Salesman Problem or cryptography.</p>

<h2 id="the-value-of-k-changes-the-strategy-of-the-whole-game">The Value of <code class="language-plaintext highlighter-rouge">k</code> Changes the Strategy of the Whole Game</h2>

<p>The number of winning lines on an <code class="language-plaintext highlighter-rouge">n x n</code> board with <code class="language-plaintext highlighter-rouge">k</code> in a row is:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>2n(n - k + 1) + 2(n - k + 1)^2
</code></pre></div></div>

<p>The first term counts horizontal and vertical windows. The second counts the two diagonal directions.</p>

<p>For standard 3x3 TicTacToe, <code class="language-plaintext highlighter-rouge">n = 3, k = 3</code>, so there are <code class="language-plaintext highlighter-rouge">8</code> winning lines. That matches what we learn as children: three rows, three columns, two diagonals.</p>

<p>For 4x4 with <code class="language-plaintext highlighter-rouge">k = 4</code>, there are only <code class="language-plaintext highlighter-rouge">10</code> winning lines. Four rows, four columns, two long diagonals. The board has more space, but each win still needs a full-length line. Blocking is cheap: one opponent mark ruins an entire candidate line.</p>

<p>For 4x4 with <code class="language-plaintext highlighter-rouge">k = 3</code>, the count jumps to <code class="language-plaintext highlighter-rouge">24</code> winning lines. The board did not change. The win condition did.</p>

<p>On a 10x10 board, the contrast is sharper. With <code class="language-plaintext highlighter-rouge">k = 10</code>, there are <code class="language-plaintext highlighter-rouge">22</code> winning lines. With <code class="language-plaintext highlighter-rouge">k = 3</code>, there are <code class="language-plaintext highlighter-rouge">288</code>.</p>

<p>For a fixed board size, smaller <code class="language-plaintext highlighter-rouge">k</code> means more possible winning windows. Larger <code class="language-plaintext highlighter-rouge">k</code> means fewer windows, because each candidate line needs more uninterrupted space.</p>

<p>This is why my 4x4, 3-in-a-row version felt so different to the children. More winning windows means more local threats. A move can matter in several directions at once. Forks become easier to create. Defence becomes less obvious because blocking one threat may leave another one open.</p>

<p>The top row of a 4x4 board already shows the problem:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>. X X .
</code></pre></div></div>

<p>With <code class="language-plaintext highlighter-rouge">k = 4</code>, this row is not an immediate emergency. X still needs both empty ends to complete the full row. With <code class="language-plaintext highlighter-rouge">k = 3</code>, either end is a winning move:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>X X X .   or   . X X X
</code></pre></div></div>

<p>Diagonals create the same effect. This position is harmless in the 4-in-a-row version, but one move from over in the 3-in-a-row version:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>X . . .
. X . .
. . . .
. . . .
</code></pre></div></div>

<p>If X plays the third diagonal square, the game ends:</p>

<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>X . . .
. X . .
. . X .
. . . .
</code></pre></div></div>

<p>That is why children who were comfortable with normal TicTacToe suddenly began missing threats. The board did not look crowded, but the number of short lines hiding inside it had grown.</p>

<p>This does not prove that X always wins as the board grows. That would be too strong. What it does show is that <code class="language-plaintext highlighter-rouge">k</code> is not a minor rule parameter. It changes the shape of the search problem.</p>

<h2 id="where-the-problem-suddenly-changes-phase-transition">Where the Problem Suddenly Changes: Phase Transition</h2>

<p>Algorithm designers care about these turning points.</p>

<p>When <code class="language-plaintext highlighter-rouge">k</code> is close to <code class="language-plaintext highlighter-rouge">n</code>, wins are long and fragile. A single blocking mark can spoil a whole line. Draws become more plausible because players have enough room to interrupt one another.</p>

<p>When <code class="language-plaintext highlighter-rouge">k</code> is small compared with <code class="language-plaintext highlighter-rouge">n</code>, winning windows multiply. The board becomes full of short local races. It starts to resemble games like Gomoku, where threats and counter-threats stack quickly.</p>

<p>Somewhere between those extremes, the game changes character. It may shift from draw-heavy to win-heavy. It may shift from easy to search exhaustively to completely impractical. That kind of sharp behavioural change is often called a phase transition.</p>

<p>I point at a useful algorithmic question: as <code class="language-plaintext highlighter-rouge">n</code> grows, how should <code class="language-plaintext highlighter-rouge">k</code> grow if we want the game to stay balanced and analyzable?</p>

<p>A rough way to see the pressure is to look at the number of winning windows. If <code class="language-plaintext highlighter-rouge">k = n</code>, the count grows only linearly with <code class="language-plaintext highlighter-rouge">n</code>. If <code class="language-plaintext highlighter-rouge">k</code> stays fixed, the count grows quadratically with the board. Those are very different worlds.</p>

<p>That is the kind of question that turns a children’s game into research fuel.</p>

<h2 id="why-this-matters-for-ai">Why This Matters for AI</h2>

<p>Classical AI did not begin with neural networks. A lot of it began with search.</p>

<p>Take a game state. Generate legal actions. Apply one action to produce a new state. Check whether the new state is terminal. Repeat until the tree ends, then work backward to choose the best move.</p>

<p>That is the heart of tree search (think Breadth-First Search, BFS, and Depth-First Search, DFS). Making the search tree aware of two opposing players gives us MiniMax. Add alpha-beta pruning and you can ignore branches that cannot affect the final decision. Add better move ordering and the pruning improves. Add a transposition table and you avoid re-solving positions you have already seen. These are not decorative tricks; they are the difference between a solver that finishes and a solver that drowns.</p>

<p>Exact search has a hard ceiling. If the tree is too large, the correctness of BFS or DFS does not help you much. An optimal algorithm that cannot return before dinner is not useful during a game.</p>

<p>That is where Monte Carlo Tree Search enters the story. Instead of expanding every possible future, it samples futures and spends more effort where the search looks promising. AlphaZero-style systems go one step further: they use neural networks to guide search toward moves worth considering.</p>

<p>The machine does not become magical. It becomes selective.</p>

<p>And that is the second lesson. Real-world AI often hits the same wall that bigger TicTacToe hits: too many possible states, too many action sequences, too many futures to enumerate exactly. Planning, routing, scheduling, protein folding, game playing, program synthesis; once the branching factor grows, brute force collapses.</p>

<p>TicTacToe just lets us watch the collapse happen on a board small enough to draw by hand.</p>

<h2 id="what-i-took-away">What I Took Away</h2>

<p>I started with a game the children already loved. I made the board bigger. Then I changed one number.</p>

<p>That was enough to move from playground strategy to deep recursion, combinatorial explosion, phase-transition intuition, and explaining the limits of exact AI search.</p>

<p>The funny part is that the children understood the important bit before the math appeared. They could feel that the game had changed. More space did not simply mean more freedom. A smaller <code class="language-plaintext highlighter-rouge">k</code> did not simply mean an easier win. The rules interacted, and the board became harder to reason about.</p>

<p>That is algorithm design in miniature.</p>

<p>In the next post, I will take this broken version of TicTacToe and treat it the way classical AI would: as a search problem. We will start with MiniMax, sharpen it with alpha-beta pruning, and see exactly where exact search begins to run out of breath.</p>]]></content><author><name>Choonyong Chan</name></author><category term="Algorithms" /><summary type="html"><![CDATA[This post was proofread with the assistance of AI.]]></summary></entry><entry><title type="html">Machines Are Becoming Sophisicated Cyberattackers. Is Singapore Ready?</title><link href="/thoughtsofaservant/opinion/machines-are-becoming-sophisicated-cyberattackers-is-singapore-ready/" rel="alternate" type="text/html" title="Machines Are Becoming Sophisicated Cyberattackers. Is Singapore Ready?" /><published>2026-04-19T00:00:00+00:00</published><updated>2026-04-19T00:00:00+00:00</updated><id>/thoughtsofaservant/opinion/machines-are-becoming-sophisicated-cyberattackers-is-singapore-ready</id><content type="html" xml:base="/thoughtsofaservant/opinion/machines-are-becoming-sophisicated-cyberattackers-is-singapore-ready/"><![CDATA[<p><em>This post was proofread with the assistance of AI.</em></p>

<hr />

<h2 id="i-you-are-already-in-the-fight">I. You Are Already in the Fight</h2>

<p>Have you dismissed a software update notification on your phone this week? Clicked a link in an SMS from “DBS” about a transaction you did not make? Streamed a Korean drama on a website you had never heard of? Reused the same password across your Singpass, banking, and e-commerce accounts? You are not alone, and you have never been a more attractive target.</p>

<p>Last year, Singaporeans lost S$1.1 billion to scams (Cyber Security Agency of Singapore [CSA], 2025), a 70% jump from last year, bringing cumulative losses since 2019 past S$3.4 billion. A single malware-enabled cryptocurrency scam cost one victim S$125 million in a single stroke. Behind each figure is a retiree whose CPF was drained, a small business whose accounts were emptied overnight, a family whose carefully-built nest egg simply vanished.</p>

<p>And that was the world <em>before</em> the machines got smart.</p>

<p>Within 9 days in April 2026, 3 announcements landed in rapid succession, from Anthropic, OpenAI, and our own CSA, that should have dominated front pages across Singapore. They did not. Most Singaporeans scrolled past them. That omission may prove expensive.</p>

<p>This article explains to the everyday Singaporean what happened, why it matters to you, and what must change (quickly!) if Singapore is to emerge from the coming decade as a global hub of secure computing rather than a cautionary tale.</p>

<hr />

<h2 id="ii-a-nation-under-siege">II. A Nation Under Siege</h2>

<p>First, let be clear-eyed about where Singapore already stood, even before frontier AI entered the picture. We were, by any reasonable measure, one of the most heavily targeted countries.</p>

<p>Organisations operating here faced an average of 2,272 cyber attacks per week in 2025, 17% per cent higher than the year before (CrowdStrike, 2026). Singapore was ranked the 7th most attacked country globally in the 4th quarter of 2024. SecurityScorecard found that every single one of Singapore’s top 100 publicly listed companies had at least one compromised third-party provider. Phishing cases rose 49% in 2024 alone; ransomware incidents climbed 21%, hammering small and medium enterprises [SMEs] in professional services hardest of all (CSA, 2025).</p>

<p>The threat actors are no amateurs. Most Singaporeans will remember the 2018 SingHealth breach that exposed 1.5 million patient records, including those of Prime Minister Lee Hsien Loong, and shook public confidence in our digital systems. What fewer people know is that the pressure has only intensified since. In July 2025, CSA publicly disclosed that UNC3886, a sophisticated China-linked Advanced Persistent Threat group, had been quietly targeting Singapore’s critical information infrastructure [CII] since late 2021, using zero-day exploits and rootkits designed to burrow deep into telecommunications and defence systems. In February 2026, Operation Cyber Guardian, a multi-agency response, was mounted specifically to counter UNC3886’s intrusions into our telecommunications networks.</p>

<p>To Singapore’s credit, the national architecture built to defend against these threats is genuinely formidable. CSA, established in 2015, now oversees roughly 500 personnel and enforces standards across 11 CII sectors under the Cybersecurity Act and its 2024 amendments, which expanded regulatory reach to cloud workloads, virtual systems, and third-party-owned infrastructure (Baker McKenzie, 2026). GovTech operates a round-the-clock Government Cybersecurity Operations Centre [GCSOC]. The Defence Science and Technology Agency [DSTA] builds bespoke military cyber solutions. The Digital and Intelligence Service [DIS], stood up in 2022 as the Singapore Armed Forces’ [SAF] 4th service branch, consolidates digital defence, cyber operations, and intelligence under a unified command. Singapore achieved the highest tier in the International Telecommunication Union’s 2024 Global Cybersecurity Index.</p>

<p>On paper, this is a nation that takes cybersecurity seriously. And on paper, it is well-prepared.</p>

<p>But paper defences are no match for what arrived in April.</p>

<hr />

<h2 id="iii-the-models-that-changed-everything">III. The Models That Changed Everything</h2>

<p>On 7 April 2026, Anthropic, the American AI company behind the Claude models, announced Project Glasswing, a consortium formed with Amazon Web Services [AWS], Apple, Google, Microsoft, Cisco, CrowdStrike, NVIDIA, Palo Alto Networks, and several others. The occasion was the unveiling of Claude Mythos Preview, a frontier AI model the company had determined was too dangerous for public release (Anthropic, 2026).</p>

<p>What had Mythos done? Operating autonomously, without human steering, it had discovered thousands of previously unknown security flaws in every major operating system and every major web browser on earth. It found a 27-year-old remotely exploitable bug in OpenBSD, a system long considered one of the most security-hardened in the world and used to run firewalls and critical infrastructure. It unearthed a 16-year-old vulnerability in FFmpeg, a media processing tool embedded in virtually every video application on the planet, in a line of code that automated testing tools had hit 5 million times without ever catching the problem. It chained together multiple vulnerabilities in the Linux kernel, the software that runs most of the world’s servers, to escalate from an ordinary user account to total control of the machine.</p>

<p>Anthropic was blunt about the implications: “AI models have reached a level of coding capability where they can surpass all but the most skilled humans at finding and exploiting software vulnerabilities.” It then added a warning that should land with weight in every boardroom (and living room) in Singapore: “Given the rate of AI progress, it will not be long before such capabilities proliferate, potentially beyond actors who are committed to deploying them safely” (Anthropic, 2026).</p>

<p>The benchmarks validates the leap. On CyberGym, a standardised test of offensive cybersecurity capability, Mythos scored 83.1%, compared to 66.6% for its predecessor, Opus 4.6. On Firefox JavaScript engine exploit development, Mythos succeeded 181 times where the previous model managed just 2 across hundreds of attempts. During testing, the model broke out of its sandbox and sent an unsolicited email to a researcher, a vivid demonstration of autonomous behaviour that no one had programmed it to exhibit.</p>

<p>Anthropic is not alone. One week later, on 14 April, OpenAI announced GPT-5.4-Cyber, a variant of its latest model deliberately fine-tuned for cybersecurity tasks, with refusal boundaries lowered to enable binary reverse engineering and other advanced defensive workflows. OpenAI classified GPT-5.4 as “high” cyber capability under its own preparedness framework, an unprecedented admission from a company that has historically been measured about model risks. “Cyber risk is already here and accelerating, but we can act… Safeguards cannot wait for a single future threshold” (OpenAI, 2026).</p>

<p>The next day, on 15 April, CSA issued a formal advisory titled <em>Risks Associated with Frontier AI Models</em>. The language was measured; the message was not. These frontier AI models, the advisory noted, “can reportedly reduce the time taken to identify vulnerabilities and engineer exploits, cutting short the duration from months to hours” (CSA, 2026). CSA urged organisations to patch every high-critical vulnerability on internet-facing systems, enforce multi-factor authentication across all administrative interfaces, review cloud configurations, segment networks against lateral movement, and deploy AI-powered vulnerability detection of their own.</p>

<p>Anthony Grieco, Cisco’s Chief Security and Trust Officer and a Glasswing partner, captured what those 9 days had accomplished: “AI capabilities have crossed a threshold that fundamentally changes the urgency required to protect critical infrastructure from cyber threats, and there is no going back. The old ways of hardening systems are no longer sufficient” (Anthropic, 2026).</p>

<p>Within a single week, the most sophisticated offensive cybersecurity capabilities, the kind that once required nation-state resources and years of elite training, became replicable by any computer running autonomously, in hours, for the cost of cloud computing credits.</p>

<hr />

<h2 id="iv-singapores-hand-strong-cards-structural-gaps">IV. Singapore’s Hand: Strong Cards, Structural Gaps</h2>

<p>Given what is now arriving, how does Singapore stand?</p>

<p>The genuinely good news is that we enter this new era better prepared than almost any comparable nation. The institutional architecture described above is real, not ornamental. Beyond it, the DIS Sentinel Programme trains approximately 800 students annually in penetration testing, network forensics, and, from this year, applied AI-for-cyber modules, developed with AI Singapore (Ministry of Defence [MINDEF], 2025). Budget 2026 committed to training 100,000 AI-savvy workers by 2029 and offered citizens on training courses 6 months of free access to premium AI tools. Singapore has invested over S$400 million in quantum technology through 2030, including Singtel’s quantum key distribution network, a hedge against the day AI-accelerated cryptanalysis makes conventional encryption obsolete.</p>

<p>Diplomatically, Singapore chaired the United Nations Open-Ended Working Group on ICT Security from 2021 to 2025, hosts the ASEAN-Singapore Cybersecurity Centre of Excellence, and has trained more than 900 senior ASEAN officials. By any reasonable standard, we are a regional leader.</p>

<p>But honesty demands we name the weaknesses too. Four stand out.</p>

<p><em>1. Talent</em>. Singapore’s cybersecurity workforce grew from roughly 4,000 professionals in 2016 to 12,000 in 2022 — yet approximately 4,000 positions remain unfilled. Cybersecurity job postings rose 57% from 2024 to 2025. The Ministry of Manpower has placed cybersecurity roles on its 2026 Shortage Occupation List, a tacit admission that domestic supply cannot meet demand. Worse, the shortage is no longer merely about headcount. The SANS Institute’s 2026 Cybersecurity Workforce Research Report, which featured CSA as a case study, found that, for the first time, skills gaps have overtaken headcount shortages as the industry’s top workforce challenge, with AI-for-cyber the single largest deficit (SANS Institute, 2026). Bitdefender’s 2025 assessment found that 64% of Singapore’s cybersecurity professionals are experiencing burnout, and 53% plan to leave their roles within a year, both figures well above global averages (Bitdefender, 2025). A nation cannot defend itself with exhausted people heading for the exit.</p>

<p><em>2.Vendor dependence</em>. Not a single Singapore-based company sits among the 12 Project Glasswing launch partners. Our cybersecurity ecosystem, while growing, remains heavily reliant on American, Israeli, and European security products. The CyberSG R&amp;D Programme Office at Nanyang Technological University [NTU] has received S$62 million, a respectable sum, but modest next to the US$29.5 million DARPA spent on its AI Cyber Challenge alone, or the US$4.4 billion in venture funding that flowed into Israeli cybersecurity in 2025 (NTU Singapore, 2023; Ynet News, 2025). We largely buy our defence from abroad. In an era of sharpening great-power competition, that dependency is strategic vulnerability.</p>

<p><em>3. The quiet nationalisation of frontier AI itself</em>. Analysts strategic think tanks have predicted and are starting to observe that major global powers are increasingly treating their most capable AI models as sovereign assets, tools reserved for national decisions on economics, defence, and intelligence, and wielded as diplomatic levers to draw smaller states into alignment. The evidence is already on the record. The United States’ January 2025 AI Diffusion Framework placed 120 countries, Singapore among them, into a three-tier system capping advanced AI chip imports, and, for the first time, applied formal export controls on the model weights of the most capable AI systems themselves, through a new Export Control Classification Number for AI models (CSIS, 2025). Enforcement actions across 2025 and 2026 specifically targeted entities in Singapore and Malaysia alleged to have diverted restricted Nvidia chips onward to China, with Washington pressuring regional governments to tighten their own licensing regimes (Sun, 2026). Anthropic itself framed its Project Glasswing partnership as a reason why “the US and its allies must maintain a decisive lead in AI technology” (Anthropic, 2026), language that makes plain how cybersecurity capability is now inseparable from great-power AI strategy. For all Singapore’s computing and policy prowess, it enters this contest as a middle power. Full sovereign control of the AI stack, from compute to foundation models to the cybersecurity applications that sit on top of them, lies beyond Singapore’s reach (Cambrian Research, 2025). The realistic strategy is not to match superpowers on their own terms but to preserve agency within Singapore’s competition: investing in targeted sovereign capabilities where scale permits, diversifying access pathways across multiple vendors and jurisdictions, and refusing the pressure toward simple alignment that an AI-bifurcated world will increasingly exert.</p>

<p><em>4. The softest target of all: the citizen, the small business, the household</em>. Which is where this story stops being abstract.</p>

<hr />

<h2 id="v-why-this-is-personal">V. Why This Is Personal</h2>

<p>Before we discuss national policy responses, an uncomfortable conversation is overdue, one aimed squarely at the reader.</p>

<p>Have you been delaying the software updates on your phone and your laptop, the ones that pop up at inconvenient moments and that you dismiss with a sigh? Those updates are not cosmetic nor corporate trickery. They patch precisely the kind of vulnerabilities that Mythos has now proven a machine can discover in hours, at scale, autonomously. Every unpatched device is an open door. CSA’s April 15 advisory is explicit: “AI-powered attacks can weaponise newly disclosed vulnerabilities within hours of publication, making rapid patch deployment critical to preventing mass exploitation” (CSA, 2026). You are no longer racing human hackers working weekends. You are racing AI models that never sleep.</p>

<p>Have you clicked a link in a Telegram or Instagram advertisement promising cheap branded hauls, flash-sale sure-win Pokemon boster packs, or unbelievable investment returns? AI-generated phishing messages already achieve click-through rates of 54%, compared to 12% for human-crafted attempts. The spelling errors and awkward phrasing that used to be telltale signs are gone. Frontier models write in fluent, contextually appropriate English, Mandarin, Malay, and Tamil. They adapt to Singaporeans’ profiles, browsing histories, and social graphs. The next “DBS™ bank notification” you receive may have been optimised by AI to look exactly like something you would trust.</p>

<p>Have you streamed a drama on a website that was not Netflix, Viu, Disney+, or mewatch, but a “free” site cluttered with pop-ups and auto-playing advertisements? Such sites are among the most reliable distribution channels for malware in Southeast Asia. The moment a frontier AI model is used to engineer a novel browser exploit, every visitor to such a site becomes a potential victim of a drive-by compromise that needs no click, no download, no consent. Your device is infected by the mere act of loading the page.</p>

<p>Have you reused the same password across your Singpass, your bank, your Shopee and Lazada accounts, and your work login? A single credential breach, combined with an AI-accelerated credential-stuffing attack, is now sufficient to drain accounts, impersonate you on social media, and extract sensitive data from your employer. The attacker no longer needs to target you specifically. The attacker targets everyone, simultaneously, at machine speed, and you happen to be in the snare.</p>

<p>This is what changes when the machines learn to hack: the cost of carelessness rises precipitously, and the margin for error collapses.</p>

<p>Small and medium enterprises [SMEs], which form the backbone of Singapore’s economy, face this same dynamic at organisational scale, and with far higher stakes. CSA data shows that over 80% of organisations, many of them SMEs, suffer at least one cyber incident annually (CSA, 2025). The World Economic Forum [WEF] projects the global economic impact of cyberattacks will surge from US$8.44 trillion in 2022 to US$23.84 trillion by 2027. SMEs, lacking dedicated security teams, proper budgets, or specialist expertise, will absorb a disproportionate share of that damage (World Economic Forum, 2024). A single successful ransomware attack can close a neighbourhood clinic, a family-owned home catering business, or a third-party vendor whose breach cascades up into the multinational bank it services.</p>

<p>The new Cyber Resilience Centre, offering SME helplines, diagnostic clinics, and CISO-as-a-Service, is a step forward. Three homegrown startups, AgileMark, Scantist, and StrongKeep, backed by the S$20 million CyberSG TIG Centre at the National University of Singapore [NUS], are building affordable, accessible tools tailored for exactly this segment (CyberSG TIG Centre, 2026). These are the right instruments. The question is whether they reach the SMEs at Woodlands and Ubi as quickly as they reach the MNCs at Raffles Place, and whether SME owners recognise the urgency before an attack, rather than after one.</p>

<hr />

<h2 id="vi-crisis-as-catalyst-three-imperatives">VI. Crisis as Catalyst: Three Imperatives</h2>

<p>There is a version of this story that ends badly: a nation well-prepared for yesterday’s threats but overwhelmed by tomorrow’s. There is another version, arguably more consistent with Singapore’s historical instinct, in which an external shock catalyses transformation that would otherwise have taken a decade.</p>

<p>Singapore has done this before. The abrupt withdrawal of British forces in 1968 forced the creation of a national defence capability from scratch, giving rise to the Singapore Armed Forces. The SARS crisis of 2003 produced a public health infrastructure whose value was proven during COVID-19. Perpetual water dependency drove investment in NEWater and desalination that made Singapore a global model for water security. In each case, a perceived vulnerability became the foundation of a national strength.</p>

<p>Frontier AI in cybersecurity presents the same structural opportunity, if Singapore moves with the urgency the moment demands. Three imperatives stand out.</p>

<p><strong>1. Build the local cybersecurity workforce as a national project, not a market outcome.</strong> The DIS Sentinel Programme, today training 800 students per year, should be scaled to 2000 or more, with structured post-National-Service pathways directly into private-sector cybersecurity careers. The model to study is Israel’s Unit 8200, whose alumni form the nucleus of a multi-billion-dollar cybersecurity industry, representing only 7% of Israel’s tech sector by number but attracting 36% to 38% of total tech investment (Startup Nation Central, 2025). Defence Cyber Chief Colonel Clarence Cai put the imperative clearly when the Sentinel Programme announced its new AI modules: “Cyber operations are already being transformed by AI. There is a need for Sentinel Programme, as a cyber youth programme, to prepare a new generation of defenders who are grounded in the fundamentals of cyber and also able to reflexively use AI for cyber operations” (MINDEF, 2025). Minister of State for Defence Desmond Choo, addressing the largest-ever cohort of Senior Military Experts in January 2026, added what should be this decade’s organising principle: “People remain our decisive advantage” (MINDEF, 2026).</p>

<p>CSA has trained over 22,000 individuals since 2020 (SANS Institute, 2026). The next 22,000 thousand must be trained in half the time, with applied AI-for-cyber skills at the core, or Singapore will find itself defending against machine-speed threats with human-speed teams.</p>

<p><strong>2. Create non-negotiable urgency for both SMEs and citizens.</strong> CSA’s advisory recommendations are not suggestions. They are survival instructions. A national SME Cyber Voucher programme, modelled on the existing Productivity Solutions Grant, could subsidise AI-powered endpoint protection, managed security operations, and automated vulnerability scanning for SMEs with fewer than 200 employees. The cost would be a small fraction of the S$1.1 billion Singaporeans already lose to scams every year.</p>

<p>For citizens, CSA’s existing outreach must be equally direct. Update your devices when prompted. Do not click on links in unsolicited messages, no matter how authentic they appear. Do not stream from pirate sites. Use unique, strong passwords. Enable two-factor authentication on every account that offers it. Report suspicious activity to ScamShield without delay. These are not IT department problems. In the age of frontier AI, they are civic duties, on the same plane as locking your HDB flat door at night.</p>

<p><strong>3. Position Singapore as the world’s most secure computing hub.</strong> This is not merely defensive; it is economic strategy. As Anthropic commits US$100 million in usage credits through Project Glasswing and OpenAI scales its Trusted Access for Cyber [TAC] programme, the global market for AI-secured infrastructure is forming in real time. Singapore’s strengths: regulatory maturity, data-centre density, a projected cybersecurity market reaching US$6.41 billion by 2031, and geopolitical neutrality, make it the natural home for organisations that demand both connectivity and demonstrable security.</p>

<p>But “demonstrable” is the operative word. The provisions for Entities of Special Cybersecurity Interest and Foundational Digital Infrastructure in the amended Cybersecurity Act should be commenced as soon as practicable, not stretched across a multi-year timeline (Baker McKenzie, 2026). CSA’s March 2026 announcement that the Centre for Strategic Infocomm Technologies [CSIT] will develop indigenous threat detection tools for critical infrastructure owners is a welcome signal of sovereign ambition, but the resourcing must match the ambition. And Singapore should actively seek partnership or observer status in initiatives like Project Glasswing; not because it needs access to Mythos itself, but because the intelligence-sharing, standard-setting, and advance warning such partnerships confer are worth more than any single tool.</p>

<hr />

<h2 id="vii-the-choice-before-us">VII. The Choice Before Us</h2>

<p>Anthropic’s own Project Glasswing announcement concluded with a sentence that deserves quoting at full weight: “The work of defending the world’s cyber infrastructure might take years; frontier AI capabilities are likely to advance substantially over just the next few months. For cyber defenders to come out ahead, we need to act now” (Anthropic, 2026).</p>

<p>Singapore has every ingredient required to lead: institutional maturity, regulatory sophistication, a highly educated workforce, a government that moves quickly when it chooses to, and a citizenry that has historically responded to existential challenges with pragmatism and resolve. What remains is the choice to treat this moment as what it actually is, not a technical problem for IT departments, but a national challenge that demands a national response.</p>

<p>So here is the call, plainly stated, to every reader of this article.</p>

<p><strong>To Government:</strong> fund the Sentinel Programme’s expansion, accelerate the remaining Cybersecurity Act provisions, bankroll an SME Cyber Voucher scheme, and pursue a seat at the international table where the rules of AI-enabled security are being written.</p>

<p><strong>To businesses, especially SMEs:</strong> assume breach, patch everything, enforce multi-factor authentication (or stronger Identity and Access Management [IAM]), segment your networks, adopt the affordable locally-built tools that the CyberSG TIG Centre has already brought to market, and do not wait for an incident to force the investment.</p>

<p><strong>To every Singaporean reading this:</strong> update your devices tonight, stop clicking unknown links, stop streaming from shady sites, use different passwords for different accounts, turn on two-factor authentication, and understand, as clearly as you understand that your HDB door needs to be locked when you leave home, that digital hygiene is now a matter of national security as much as personal security.</p>

<p>AI models are already at the gates. The question is no longer whether Singapore will be targeted, it already is, 2,272 times a week, but whether each of us, in our role, will act before the window closes.</p>

<p>The clock is running. It does not run slowly.</p>

<hr />

<h3 id="references">References</h3>

<p>Anthropic. (2026, April 7). <em>Project Glasswing: Securing critical software for the AI era.</em> https://www.anthropic.com/glasswing</p>

<p>Baker McKenzie. (2026, March 31). <em>Singapore: Cybersecurity licensing framework updates, new threat detection tools.</em> https://www.bakermckenzie.com/en/insight/publications/2026/03/singapore-cybersecurity-licensing-framework-updates-threat-detection-tools</p>

<p>Bitdefender. (2025). <em>2025 Cybersecurity Assessment Report: Singapore findings.</em> Bitdefender.</p>

<p>Cambrian Research. (2025, August 11). <em>Singapore’s AI Strategy and the Limits of Digital Sovereignty.</em> https://cambrianr.substack.com/p/singapores-ai-strategy-and-the-limits</p>

<p>Centre for Strategic and International Studies. (2025, April 2). <em>Understanding U.S. allies’ current legal authority to implement AI and semiconductor export controls.</em> https://www.csis.org/analysis/understanding-us-allies-current-legal-authority-implement-ai-and-semiconductor-export</p>

<p>CrowdStrike. (2026). <em>2026 Global Threat Report: AI accelerated adversaries.</em> https://www.crowdstrike.com/en-us/press-releases/2026-crowdstrike-global-threat-report/</p>

<p>Cyber Security Agency of Singapore. (2025). <em>Singapore Cyber Landscape 2024/2025.</em> https://www.csa.gov.sg/resources/publications/singapore-cyber-landscape-2024-2025/</p>

<p>Cyber Security Agency of Singapore. (2026, April 15). <em>Advisory on risks associated with frontier AI models.</em> https://www.csa.gov.sg</p>

<p>CyberSG TIG Centre. (2026, March 23). <em>Singapore cybersecurity firms showcase SME-focused innovations to counter rising cyber threats at RSAC 2026 Conference.</em> Media OutReach Newswire.</p>

<p>Ministry of Defence, Singapore. (2025, November 28). <em>SAF expands Cybersecurity Student Talent Development Programme to include AI skills in 2026.</em> https://www.mindef.gov.sg/news-and-events/latest-releases/28nov25-nr2/</p>

<p>Ministry of Defence, Singapore. (2026, January 21). <em>Speech by Minister of State for Defence, Mr Desmond Choo, for the 30/26 SAF Senior Military Expert Appointment Ceremony.</em> https://www.mindef.gov.sg/news-and-events/latest-releases/21jan26-speech/</p>

<p>NTU Singapore. (2023). <em>S$62 million CyberSG R&amp;D Programme Office.</em> https://www.ntu.edu.sg/news/detail/sgd62-million-cybersg-r-d-programme-office</p>

<p>OpenAI. (2026, April 14). <em>Trusted access for the next era of cyber defense.</em> https://openai.com</p>

<p>SANS Institute. (2026). <em>2026 SANS GIAC Cybersecurity Workforce Research Report.</em> SANS Institute.</p>

<p>Startup Nation Central. (2025). <em>Israeli cybersecurity is defining the future in 2025.</em> https://startupnationcentral.org/hub/blog/israeli-cybersecurity-is-defining-the-future-in-2025/</p>

<p>Sun, M. (2026, April). <em>Manacled Manus: The limits of “Singapore washing” for China AI. Asia Times.</em> https://asiatimes.com/2026/04/manacled-manus-the-limits-of-singapore-washing-for-china-ai/</p>

<p>World Economic Forum. (2024). <em>Global Cybersecurity Outlook 2024.</em> WEF.</p>

<p>Ynet News. (2025). <em>Record $4.4B flows into Israeli cybersecurity as global VCs outpace locals in 2025 boom.</em> https://www.ynetnews.com/business/article/rjggjusz11g</p>

<hr />

<p><em>The views expressed are the author’s own.</em></p>]]></content><author><name>Choonyong Chan</name></author><category term="Opinion" /><category term="Opinion" /><category term="Cybersecurity" /><summary type="html"><![CDATA[This post was proofread with the assistance of AI.]]></summary></entry><entry><title type="html">Every Scam Site Leaves One Trace Before It Goes Live. We Built a Tool to Catch It.</title><link href="/thoughtsofaservant/cybersecurity/brandsentinel-a-live-feed-of-malicious-domains-before-the-first-victim-is-claimed/" rel="alternate" type="text/html" title="Every Scam Site Leaves One Trace Before It Goes Live. We Built a Tool to Catch It." /><published>2026-03-27T00:00:00+00:00</published><updated>2026-03-27T00:00:00+00:00</updated><id>/thoughtsofaservant/cybersecurity/brandsentinel-a-live-feed-of-malicious-domains-before-the-first-victim-is-claimed</id><content type="html" xml:base="/thoughtsofaservant/cybersecurity/brandsentinel-a-live-feed-of-malicious-domains-before-the-first-victim-is-claimed/"><![CDATA[<p><em>This post was proofread with the assistance of AI. The source code backing this story is open-sourced and available on <a href="https://github.com/choonyongchan/BrandSentinel">GitHub</a>.</em></p>

<hr />

<p>When we pointed CertStream at two influential Singapore organisations, it surfaced over ten thousands of suspicious domains in a single day. No human team could triage that volume before the threats became active. So we built a machine to do it — one that classifies domains in real time, flags the dangerous ones before the first victim ever loads the page, and hands analysts a curated shortlist instead of a firehose. This is the story of how that tool came to be, what it took to build it, and what it taught us about defending brands at machine speed.</p>

<hr />

<p>Before I get into the technical details, I want to take a moment to acknowledge the people who made this work possible.</p>

<p>During my internship, I had the privilege of learning from the Asia-Pacific Digital Risk Protection team at a leading cybersecurity company. They showed me what it really looks like to defend a brand in the wild — the volume, the pace, the judgment calls analysts make under pressure, and the genuine satisfaction of taking down a scam site before it harms someone. This project is built on everything they taught me, and I dedicate it to them.</p>

<hr />

<h2 id="the-problem-we-found-the-domains-now-what">The Problem: We Found the Domains. Now What?</h2>

<p>In <a href="./2026-03-19-hunting-scam-domains-before-they-strike-with-certstream.md">my previous post</a>, I described how Certificate Transparency logs — and specifically CertStream, the real-time WebSocket feed built by Cali Dog Security — give us a powerful early warning system. Every domain that obtains an HTTPS certificate is publicly logged the moment it does, and brand-targeted scam domains are no exception. That means we can see them at the point of certificate issuance, often before the site is even reachable.</p>

<p>The problem we were left with was a different one entirely: <strong>volume</strong>.</p>

<p>CertStream processes millions of certificate events every day. Even after filtering to domains that contain your brand keywords, you’re looking at thousands of candidates per day for a single brand. A human analyst cannot manually investigate each one fast enough for the intelligence to be actionable. By the time they’ve worked through the queue, the scam domains they flagged are either weaponised or gone.</p>

<p>We had solved the sourcing problem. We had created a new one.</p>

<h2 id="the-implication-speed-is-not-a-nice-to-have">The Implication: Speed Is Not a Nice-to-Have</h2>

<p>Let me be direct about the stakes. In threat intelligence, there are two kinds of errors. <strong>False positives</strong> — flagging a benign domain as malicious — waste analyst time and erode trust in the tool. Annoying, but recoverable. <strong>False negatives</strong> — missing a real scam domain — mean a live phishing site with your brand’s logo on it is collecting victims’ credentials, authorising fraudulent transactions, and destroying customer trust, with nobody at your organisation aware it exists.</p>

<p>False negatives are not a metric. They are harm to real people.</p>

<p>The implication is this: an automated classification layer that works around the clock, surfaces threats with high recall, and hands analysts only what truly requires human judgment is not a convenience — it is a prerequisite for any serious DRP operation. That is what we set out to build.</p>

<h2 id="brandsentinel-a-live-feed-of-malicious-domains">BrandSentinel: A Live Feed of Malicious Domains</h2>

<h3 id="methodology">Methodology</h3>

<p>BrandSentinel is an open-source Digital Risk Protection pipeline. It ingests domains from multiple threat intelligence feeds continuously, classifies them using an ensemble of heuristics, and writes verdicts to per-category output files — all while the analyst monitors a live dashboard in Prefect’s UI.</p>

<p><strong>Ingestion.</strong> Ten source workers run as independent async tasks, each polling a different feed on its own schedule:</p>

<table>
  <thead>
    <tr>
      <th>Source</th>
      <th>Frequency</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>CertStream</td>
      <td>Live (CT log stream)</td>
    </tr>
    <tr>
      <td>URLhaus</td>
      <td>Every 5 minutes</td>
    </tr>
    <tr>
      <td>PhishTank</td>
      <td>Every 1 hour</td>
    </tr>
    <tr>
      <td>OpenPhish</td>
      <td>Every 12 hours</td>
    </tr>
    <tr>
      <td>CERT Polska, Phishing.Database, Phishing Army, Botvrij.eu, DigitalSide</td>
      <td>Configurable</td>
    </tr>
    <tr>
      <td>Manual import file</td>
      <td>Every 30 seconds</td>
    </tr>
  </tbody>
</table>

<p>Every domain that any source produces is placed onto a shared, deduplicated <code class="language-plaintext highlighter-rouge">asyncio.Queue</code>. If a domain has been seen before — across any source — it is silently dropped. The pipeline never processes the same domain twice.</p>

<p><strong>Filtering.</strong> A keyword-based filter checks each domain against the brand configurations in <code class="language-plaintext highlighter-rouge">config.yaml</code>. Domains that do not contain a brand keyword are classified as <code class="language-plaintext highlighter-rouge">IRRELEVANT</code> and logged immediately. This is the first gate, and it is intentionally cheap — regex matching costs nothing compared to what comes next.</p>

<p><strong>Classification.</strong> For every domain that passes the filter, BrandSentinel concurrently fetches the HTTP response (following redirects, capturing page content), resolves DNS records, and retrieves the TLS certificate — all in parallel. This enriched context is then passed through a pipeline of 15 heuristics.</p>

<p>The heuristics run lazily. If any single heuristic produces a <em>definitive</em> verdict — such as matching a known brand favicon hash on a non-canonical domain, or detecting a phishing kit directory structure — the pipeline short-circuits and assigns that verdict immediately, without running the remaining checks. For contributing signals, scores are accumulated and normalised. The final verdict is one of four categories:</p>

<ul>
  <li><strong>SCAM</strong> — High confidence. Investigate and initiate takedown.</li>
  <li><strong>INCONCLUSIVE</strong> — Suspicious. Requires analyst review.</li>
  <li><strong>BENIGN</strong> — Passed all checks. Continue monitoring passively.</li>
  <li><strong>IRRELEVANT</strong> — Not related to the monitored brand. Dropped.</li>
</ul>

<p>A few heuristics are worth calling out explicitly because they do the heaviest lifting:</p>

<ul>
  <li><strong>Inactive domain</strong>: An HTTP timeout, connection refusal, or non-200 response is a strong signal — scam domains are frequently pre-registered before they’re weaponised, or already retired.</li>
  <li><strong>Parking detection</strong>: The page content or DNS fingerprint matches a known domain parking service. These are typically benign, but they are watched — parked domains can be activated at any time.</li>
  <li><strong>Brand lookalike</strong>: Levenshtein distance between the registered domain and the brand’s canonical domain detects typosquatting and combosquatting that a keyword filter alone would miss.</li>
  <li><strong>Favicon hash matching</strong>: A SHA-1 hash of the site’s favicon is compared against a curated list of brand favicon hashes. A match on a non-canonical domain is a near-definitive SCAM signal — an attacker has copied the brand’s own visual identity.</li>
  <li><strong>Forms exfiltration</strong>: Login forms or input fields that submit to a different domain than the one being analysed are a reliable indicator of a credential-harvesting phishing page.</li>
</ul>

<p><strong>Output.</strong> Verdicts are written continuously to <code class="language-plaintext highlighter-rouge">scam.txt</code>, <code class="language-plaintext highlighter-rouge">inconclusive.txt</code>, <code class="language-plaintext highlighter-rouge">benign.txt</code>, and <code class="language-plaintext highlighter-rouge">irrelevant.txt</code>. The entire pipeline is orchestrated by Prefect, which provides a live UI for observing flow runs, retrying failed tasks, and inspecting per-domain logs without touching the terminal.</p>

<h3 id="results">Results</h3>

<p>The version I am describing here is the current, fully instrumented release. But the insight that validated the approach came from something far simpler.</p>

<p>An early prototype ran only two heuristics: the <strong>inactive domain check</strong> and the <strong>parking detection check</strong>. No lookalike scoring, no favicon hashing, no content analysis — just those two. Even with only CertStream as the data source, the results were striking.</p>

<p>Of every hundred domains flagged by the brand keyword filter, roughly <strong>ninety-five were resolved by those two heuristics alone</strong>. The inactive check cleared out domains that had been registered but not yet deployed — or already retired. The parking check cleared out benign infrastructure that had been registered speculatively by domain investors, not attackers. What remained — the five or so domains that were live, active, and neither parked nor obviously benign — was the set that actually warranted human attention.</p>

<p>In operational terms, this meant a DRP analyst who previously faced a hundred manual lookups per day was reduced to reviewing fewer than ten. The pipeline had not replaced the analyst. It had given them back their day.</p>

<h3 id="discussion">Discussion</h3>

<p>The two-heuristic result is encouraging, but it is not the ceiling — it is the floor. The more sources you ingest from, and the more precisely your heuristics are calibrated, the tighter the classification becomes.</p>

<p>This brings us back to the two-error asymmetry. <strong>False positives</strong> are an inconvenience: the analyst reviews a benign domain and clears it. The cost is time. <strong>False negatives</strong> are a failure: a scam domain is classified as benign or irrelevant and goes unactioned. The cost is victims. BrandSentinel is deliberately calibrated toward recall over precision — when evidence is ambiguous, the verdict is <code class="language-plaintext highlighter-rouge">INCONCLUSIVE</code>, not <code class="language-plaintext highlighter-rouge">BENIGN</code>. The analyst sees it. The domain does not slip through.</p>

<p>As research matures, more can be layered in. New ingestion sources plug in by implementing a single <code class="language-plaintext highlighter-rouge">Source</code> base class. New heuristics extend <code class="language-plaintext highlighter-rouge">HeuristicBase</code> and register themselves; the orchestrator handles the rest. Commercial feeds — URLScan.io, OTX, VirusTotal — can join the source pool without touching the classification logic. An ML-based scoring layer can replace or complement the weighted heuristic sum for teams with labelled datasets. The architecture was designed to absorb these improvements without structural change.</p>

<h2 id="who-should-use-this">Who Should Use This</h2>

<p><strong>Digital Risk Protection teams</strong> are the primary audience. BrandSentinel is not a replacement for experienced analysts — it is the triage layer that lets them work on what matters. Instead of spending their shift manually checking whether <code class="language-plaintext highlighter-rouge">mybank-secure-login.xyz</code> resolves to anything, analysts can focus on the pre-scored shortlist that BrandSentinel surfaces: live domains, with suspicious content, with brand-identical favicons, submitting credentials to a foreign server. That is a very different kind of work.</p>

<p><strong>Small and medium enterprises</strong> are the second audience, and perhaps the more important one. Most SMEs do not have a DRP team. They may not even know how actively their brand is being impersonated online. Running BrandSentinel — even in its simplest configuration, with CertStream and a handful of brand keywords — produces a sobering and actionable picture of digital threat exposure. For an SME trying to decide whether to invest in a commercial DRP solution, that picture is exactly the evidence they need.</p>

<h2 id="what-comes-next">What Comes Next</h2>

<p>BrandSentinel is open source. It is a gift to the cybersecurity community, in the same spirit as the team that first gave me this perspective. If you work in threat intelligence, digital risk, or brand protection, I hope you find it useful — and I hope you make it better. Open an issue, submit a heuristic, add a feed.</p>

<p>The threat is automated. The defence should be too.</p>

<h2 id="what-i-took-away">What I Took Away</h2>

<p><strong>1. Open-source intelligence is a treasure trove — especially for those who need it most.</strong></p>

<p>Open-source intelligence will not beat a dedicated proprietary threat intelligence platform on visibility, reliability, or breadth of coverage. The commercial vendors have more sources, better enrichment, and dedicated teams curating signal quality. That is simply the reality.</p>

<p>But here is the counterpoint: for the organisations most exposed to brand-impersonation attacks — small and medium enterprises — a commercial DRP subscription is often out of reach. The same businesses that cannot absorb the cost of a cyberattack are frequently the ones that cannot afford the tooling to prevent one. That asymmetry is where open-source intelligence earns its keep.</p>

<p>The insight BrandSentinel reinforced for me is that the value of OSINT is not in any single source — it is in the fusion. CertStream alone produces noise. URLhaus alone has coverage gaps. PhishTank alone misses the newest campaigns. But a pipeline that ingests all of them, deduplicates across sources, and applies a consistent classification layer produces something genuinely useful: a picture of your brand’s threat exposure that would otherwise require a commercial contract to see. For an SME deciding whether to invest in a DRP solution, BrandSentinel running for a week is a more persuasive argument than any sales deck.</p>

<p><strong>2. Building a real-time, scalable ETL pipeline in Python is harder than it looks — and the right answer is to iterate, not over-engineer.</strong></p>

<p>I designed BrandSentinel to be event-driven from the start. The first version used Python <code class="language-plaintext highlighter-rouge">async</code> functions throughout, with Redis Pub/Sub as the message bus between ingestion and classification. For a small proof-of-concept, this is clean and simple — everything lives in one file, the architecture is easy to reason about, and the latency between domain observation and verdict is low.</p>

<p>The problem became obvious under load. Python’s threading model means CPU-bound work competes with I/O on a single thread, and the volume of domains BrandSentinel processes is enough to make that a real bottleneck. Worse, several libraries I relied on — DNS resolution in particular — do not expose async interfaces. Every blocking call was a stall. Converting the entire codebase to async is not just tedious; it is a maintenance burden that multiplies with every new heuristic.</p>

<p>My first instinct was to reach for a microservice architecture: split ingestion, classification, and output into separate processes, use a proper message queue, and let each component scale independently. The design is sound on paper. In practice, the overhead of inter-process communication directly conflicts with BrandSentinel’s primary goal, which is the speed of scam domain detection. Latency introduced at the architecture level is just as damaging as latency introduced by slow heuristics.</p>

<p>The solution I landed on was Prefect — a workflow orchestration framework that let me run the full pipeline from a single Python file while genuinely parallelising I/O-bound work across tasks. No async contortions. No microservice topology. No operational complexity beyond spinning up the Prefect server. The pipeline became both faster and easier to reason about than either of its predecessors.</p>

<p>The lesson here is not that Prefect is the answer to every pipeline problem. The lesson is to resist the temptation to design for theoretical scale before you understand where the real bottlenecks are — and to search aggressively for existing tools before building your own solutions to solved problems. Iterative solutions are not a sign of weak engineering. They are a sign of engineering focused on the right outcome: in this case, getting a scam domain classified and surfaced to an analyst as fast as possible.</p>

<hr />

<p><em>If you’re running BrandSentinel against your own brand, I’d genuinely like to hear what you find. What heuristics are generating the most signal? What sources are you missing? Leave a comment below.</em></p>]]></content><author><name>Choonyong Chan</name></author><category term="Cybersecurity" /><summary type="html"><![CDATA[This post was proofread with the assistance of AI. The source code backing this story is open-sourced and available on GitHub.]]></summary></entry><entry><title type="html">Hunting Scam Domains Before They Strike with CertStream</title><link href="/thoughtsofaservant/cybersecurity/hunting-scam-domains-before-they-strike-with-certstream/" rel="alternate" type="text/html" title="Hunting Scam Domains Before They Strike with CertStream" /><published>2026-03-19T00:00:00+00:00</published><updated>2026-03-19T00:00:00+00:00</updated><id>/thoughtsofaservant/cybersecurity/hunting-scam-domains-before-they-strike-with-certstream</id><content type="html" xml:base="/thoughtsofaservant/cybersecurity/hunting-scam-domains-before-they-strike-with-certstream/"><![CDATA[<p><em>This post was proofread with the assistance of AI. The source code backing this story is open-sourced and available on <a href="https://github.com/choonyongchan/CertStream">GitHub</a>.</em></p>

<hr />

<p>Here’s something that genuinely surprised me: modern scam operations are run like software companies.</p>

<p>They have CI/CD pipelines. They spin up cloud infrastructure on demand. They automate the deployment of thousands of scam websites, run them briefly, then tear them down before anyone can respond. Research by <a href="https://www.group-ib.com/media-center/press-releases/classiscam-2023/">Group-IB</a> shows that organised scam syndicates have adopted the same DevOps practices your favourite tech companies use — except they’re using it to steal from people. Their investigation into <em>Classiscam</em>, a scam-as-a-service operation, found 38,000 members organised across Telegram groups with defined roles, using bots to spin up fake phishing pages on demand — a supply chain for fraud that earned an estimated $64.5 million across 251 brands in 79 countries.</p>

<p>That realisation changed how I think about cybersecurity. And it led my team to build something small, clever, and — I think — genuinely useful.</p>

<hr />

<p>A note on where this work comes from. During my internship with the Asia-Pacific Digital Risk Protection team at a global cybersecurity company, I had the chance to observe how professional analysts actually hunt threats: the feeds they rely on, the signals they trust, and the exhausting manual workload that accumulates when the tooling can’t keep up. CertStream was already on the team’s radar. What you’re reading is my attempt to understand it properly — to pull it apart, follow it back to its roots in Certificate Transparency, and make a clear case for why every practitioner doing this work should have it in their toolkit. Whatever is useful in this post, I owe to the analysts who showed me what the work actually looks like on the ground.</p>

<hr />

<h2 id="the-problem-with-playing-defence">The Problem with Playing Defence</h2>

<p>The traditional playbook for tackling scam websites goes something like this: a victim reports a suspicious link → an analyst investigates → the domain gets flagged or taken down. Clean. Logical. Thoroughly outdated.</p>

<p>Here’s why. A scam syndicate today doesn’t need a domain to last more than a few hours. They register a new domain, run a scam campaign on it, and retire it the moment it’s been accessed once — or even sooner. By the time a victim reports the link, the website is already gone. By the time an analyst looks at it, the domain resolves to nothing. The takedown request goes nowhere.</p>

<p>Throwing more web crawlers at the problem sounds appealing — just enumerate <em>all</em> the domains! — but that’s prohibitively expensive and still fundamentally reactive. We needed a way to catch scam domains before they reach their first victim, not after.</p>

<h2 id="an-unexpected-clue-in-plain-sight">An Unexpected Clue in Plain Sight</h2>

<p>Here’s where it gets interesting. Modern browsers are ruthless about unencrypted or untrusted connections. Navigate to a site without a valid HTTPS certificate and you’re greeted with a giant red warning page. For a scam site trying to look legitimate, that warning is a conversion killer.</p>

<p>This means scam domains <em>have</em> to obtain a valid HTTPS certificate. There’s no way around it.</p>

<p>And here’s the thing about HTTPS certificates: every single one is publicly logged. Certificate Transparency (CT) is an open standard that requires Certificate Authorities to record every certificate they issue into append-only public logs. The goal is accountability — anyone can audit who issued what, to whom, and when.</p>

<p>But as a side effect, CT logs are a live feed of every domain that is in the process of going online. Including scam domains.</p>

<p><a href="https://calidog.io/">Cali Dog Security</a> built <a href="https://certstream.calidog.io/">CertStream</a> to make this feed accessible. It’s a real-time WebSocket stream of certificate issuance events — open, free, and broadcasting millions of domain names every day. When I first heard about it, I immediately thought: <em>this is our early warning system</em>.</p>

<h2 id="what-we-built">What We Built</h2>

<p>The concept is straightforward: listen to the CertStream feed, and flag any domain that looks like it could be impersonating a brand or entity we’re protecting.</p>

<p>We express “looks like” as a set of regex patterns — one per line in a simple text file. To monitor for domains targeting a bank, you might write:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>.*mybank.*
.*my-bank.*
.*mybank-secure.*
</code></pre></div></div>

<p>Our tool subscribes to the CertStream WebSocket, processes each incoming certificate event, strips the domain variants (with and without <code class="language-plaintext highlighter-rouge">www.</code>, <code class="language-plaintext highlighter-rouge">*.</code>, <code class="language-plaintext highlighter-rouge">https://</code>), and checks them against the compiled pattern list. Any match gets logged to a SQLite database and exported for further analysis.</p>

<p>The flow looks like this:</p>

<blockquote>
  <p><strong>CT Server → WebSocket stream → sanitise domains → match regex → store hits → export for analysis</strong></p>
</blockquote>

<p>That’s it. No web crawlers. No enumeration. No waiting for a user to report anything. The scam domains come to us.</p>

<h2 id="results">Results</h2>

<p>When we deployed this alongside our team’s existing domain analysis tooling, the effect was immediate: we effectively <strong>doubled our search space for suspicious domains</strong> without proportionally increasing our workload. More importantly, we were now seeing domains at the moment of certificate issuance — which is often <em>before</em> the site is even reachable to the public.</p>

<p>For a Digital Risk Protection team, that’s a significant shift. Instead of responding to threats, we were meeting them at the door.</p>

<h2 id="who-should-care-about-this">Who Should Care About This</h2>

<p>If you’re on a <strong>Digital Risk Protection team</strong>, this approach is directly applicable. Swap in the brand keywords you’re protecting and you have a 24/7 monitoring feed that never sleeps.</p>

<p>If you’re a <strong>company trying to assess your own digital risk</strong>, running this against your own brand names gives you a rough — and surprisingly sobering — sense of how actively threat actors target your identity online.</p>

<p>And if you’re a <strong>security researcher or student</strong>, I’d encourage you to look at this as a template for creative problem solving: what other public, boring-sounding infrastructure quietly records everything you’d ever want to know?</p>

<p>The natural next step is a downstream analysis stage — an AI/ML classifier that quickly triages matches between genuine threats and false positives, either automatically or with a human in the loop. That’s what we’re working on next with <strong>BrandSentinel</strong>. Both projects are being open-sourced as a gift to the cybersecurity community, because good tools should be shared.</p>

<h2 id="what-i-took-away">What I Took Away</h2>

<p><strong>1. The threat landscape moves faster than you think.</strong> I knew scam campaigns were a problem. I didn’t fully appreciate that they had industrialised to the point of using orchestration frameworks and automated infrastructure. That shift changes everything about how we need to respond.</p>

<p><strong>2. Cybersecurity rewards lateral thinking.</strong> The instinct is to do more of what already works — bigger crawlers, more analysts. The better question is: <em>what does the attacker have to do that we can intercept?</em> CT logs were the answer hiding in plain sight.</p>

<p><strong>3. Open source makes this possible.</strong> CertStream is a free, open initiative. Without it, building this capability would have required months of infrastructure work. The open source community quietly lowers the barrier to entry for exactly this kind of innovation, and that’s worth celebrating.</p>

<hr />

<p><em>What other open data feeds or public infrastructure do you think are sitting underused as threat intelligence sources? I’d love to hear what you’re thinking in the comments.</em></p>]]></content><author><name>Choonyong Chan</name></author><category term="Cybersecurity" /><summary type="html"><![CDATA[This post was proofread with the assistance of AI. The source code backing this story is open-sourced and available on GitHub.]]></summary></entry></feed>