Studying physical chess literature (tactics manuals, game collections, or Kindle e-readers) has a fundamental limitation: you cannot easily test alternative move trees, verify tricky defensive resources, or check your calculation without setting up a physical board or manually typing FEN strings into an analysis app.

We built Chessloom, an offline Android application that scans printed 2D chess diagrams directly from books, screens, or Kindles using your phone camera, instantly converts them into an interactive board, and provides real-time tactical blunder prevention powered by an embedded native Stockfish engine.

Here is the engineering story of how the vision pipeline, homography rectification, and native C++ engine were implemented.

1. System Architecture

Physical Chess Book / Kindle Display
                │
                ▼
 CameraX Viewfinder + Interactive 4-Corner Draggable Overlay
                │
                ▼
 2D Projective Homography Transform ──► Rectified 800x800 Square Bitmap
                │
                ▼
 64-Cell Grid Slicer ──► Morphological Feature Extraction & Piece Classifier
                │
                ▼
 Validated FEN Board State (Interactive Review & Palette Edit)
                │
                ├─► Threat Heatmap & Hanging Piece Detector
                ├─► Blunder Radar (>150 centipawn swing detector)
                └─► Dual Engine Evaluator (Stockfish NDK + Kotlin Minimax)

2. Perspective Homography Rectification

Printed chess diagrams in paperbacks are rarely photographed dead-on. Pages curve into the book spine, and handheld cameras introduce significant perspective distortion.

Standard square bounding boxes fail completely under perspective tilt. Instead, Chessloom implements an interactive 4-Corner Draggable Quad overlay on the CameraX preview:

    P0 (Top-Left)               P1 (Top-Right)
         o─────────────────────────o
          \                       /
           \                     /     Skewed Paperback Diagram
            \                   /
             o─────────────────o
        P3 (Bottom-Left)     P2 (Bottom-Right)

Once the user positions the four corners, Android’s hardware-accelerated 2D graphics matrix rectifies the quad into a normalized $800 \times 800\text{ px}$ square grid using projective homography:

fun rectifyQuadrilateral(srcBitmap: Bitmap, corners: List<PointF>): Bitmap {
    val src = floatArrayOf(
        corners[0].x, corners[0].y, // Top-Left
        corners[1].x, corners[1].y, // Top-Right
        corners[2].x, corners[2].y, // Bottom-Right
        corners[3].x, corners[3].y  // Bottom-Left
    )
    val dst = floatArrayOf(
        0f, 0f,
        800f, 0f,
        800f, 800f,
        0f, 800f
    )
    val matrix = Matrix()
    matrix.setPolyToPoly(src, 0, dst, 0, 4)
    
    val destBitmap = Bitmap.createBitmap(800, 800, Bitmap.Config.ARGB_8888)
    val canvas = Canvas(destBitmap)
    canvas.drawBitmap(srcBitmap, matrix, Paint(Paint.FILTER_BITMAP_FLAG))
    return destBitmap
}

3. The 2D Glyph Classifier: Fast, Offline OCR

Running heavy machine learning models (like YOLO or MobileNet) on mobile devices introduces latency, drains battery, and bloats APK sizes by tens of megabytes.

Because printed book diagrams follow high-contrast 2D typography, Chessloom uses a lightweight morphological classifier:

  1. 64-Cell Extraction: Slices the $800 \times 800\text{ px}$ rectified bitmap into 64 uniform $100 \times 100\text{ px}$ sub-bitmaps.
  2. Board Checker Background Subtraction: Alternating light and dark squares are normalized by subtracting the average border luminance.
  3. Contour Density & Aspect Ratio:
    • Empty squares have near-zero central pixel variance.
    • Pawns have compact height-to-width ratios.
    • Knights exhibit asymmetric left/right contour variance.
    • Kings and Queens exhibit high top-half cross density (crowns).
  4. Instant Palette Override: In the rare event of a glyph misidentification, the user can tap any square to swap the piece or flip board orientation in under 2 seconds.

4. Dual Local Engine Architecture: C++ NDK + Pure Kotlin

For move analysis and sparring, Chessloom relies on a dual-engine architecture:

+─────────────────────────────────────────────────────────────+
|                     Android App Layer                       |
+──────────────────────────────┬──────────────────────────────+
|  Stockfish NDK (libstockfish)|  Embedded Kotlin Minimax PST  |
|  - C++17 UCI Bridge          |  - Alpha-Beta search          |
|  - Multi-threaded search     |  - PeSTO Piece-Square Tables  |
|  - Master-grade evaluation   |  - Zero-dependency fallback   |
+──────────────────────────────┴──────────────────────────────+

1. Stockfish C++ NDK Bridge

Compiled natively using Android NDK 28 and CMake into libstockfish_bridge.so. It supports arm64-v8a, armeabi-v7a, x86, and x86_64, delivering full multi-threaded UCI search depths ($> 20\text{ plies}$) directly on device without internet connectivity.

2. Pure Kotlin Minimax PST Fallback

For instant unit testing, emulator runs without NDK, and low-power battery-saver modes, Chessloom includes an embedded Kotlin Alpha-Beta engine using PeSTO Piece-Square Tables (PST) and quiescent search.

5. The 1400-Rating Improvement Suite

Intermediate players (~1400 Elo) rarely lose games due to deep 15-move positional subtleties; they lose because of tactical blindness and leaving pieces unguarded. Chessloom incorporates features tailored directly to intermediate training:

1. Threat Heatmap & Hanging Pieces

Calculates the attack ray intersection of every piece on the board. Friendly pieces attacked by enemy units with zero defenders are classified as Hanging Pieces and highlighted with pulsing red halos.

2. The Blunder Radar

When practicing against the sparring bot, candidate moves are evaluated before commitment. If a proposed move drops more than $1.50\text{ pawns}$ ($> 150\text{ centipawns}$) or allows an immediate tactical fork/pin, the app pauses the play:

Blunder Warning: Move drops $-2.40$. White has Qh5+ winning the bishop on c5. Try again!

By combining computer vision with on-device Stockfish analysis, Chessloom turns any paper chess book into an interactive, master-level training station.