Sketchbook

A Next-Generation 3D Sandbox Engine combining Three.js, Cannon.js, and "Text-to-World" Generative AI.

Features • Text-to-World AI • Showcase • Comparison • Getting Started • Controls

---

Overview

Sketchbook is not just a game engine—it is a Generative 3D Playground. It bridges the gap between natural language and interactive 3D worlds. Users can describe complex game mechanics, characters, vehicles, or scenarios, and the engine's integrated AI agent generates the necessary TypeScript code in real-time, compiles it, and injects it into the running world without a reload.

Built on the robust foundation of Three.js for rendering and Cannon.js for physics, Sketchbook offers a physically accurate, visually stunning, and infinitely extensible environment.

---

🎮 Live Demo: Play Online

📱 Download APK: Latest Release

AI Generated Showcase

Every demo below was created solely by prompting the AI. No manual coding was performed for these specific scenarios. The user simply described the desired logic, physics interactions, or game rules, and Sketchbook generated the implementation.

🌲 Fantasy Scene Generation

"Create a fantasy forest scene with an interactive talking ogre."

YTDown.com_YouTube_Creating-forest-with-talking-ogre-using_Media_JmRra54vYGU_001_720p.mp4

💥 Physics-Based Combat

"Create a car with a rocket launcher that shoots explosive projectiles."

📺 Watch Demo

carbazoka.mp4

⚽ Sports Logic

"Add a football game mechanic with a ball and goals."

📺 Watch Demo

football.mp4

🤝 Multiplayer Interaction

"Enable two-player split-screen gameplay."

📺 Watch Demo

2player.mp4

🏎️ Racing Mechanics

"Create a racing circuit with lap timing."

📺 Watch Demo

racing.mp4

🧟 AI Behavior

"Spawn zombies that chase the player."

📺 Watch Demo

zombies.mp4

---

Comparison vs Other Platforms

Sketchbook occupies a unique niche in the "Generative Game Engine" landscape by focusing on code generation over asset assembly, offering lower-level control and transparency than its SaaS competitors.

Feature	Sketchbook	BitMagic	Rosebud AI	Buildbox 4	Hiber3D
Primary Output	TypeScript Code	Game Levels	Assets & Scripts	Drag & Drop + AI	3D Worlds
Model	Open Source (Self-Hosted)	SaaS / Closed	SaaS / Web-Based	Commercial	SaaS / Web-Based
Mechanism	Code Generation (Logic)	Asset Assembly	"Vibe Coding"	No-Code Assistant	Asset Assembly
Tech Stack	Three.js + Cannon.js	Proprietary	WebGL / Phaser	Proprietary	Custom Engine
Customization	Unlimited (Full Code Access)	Platform Constraints	Web Editor	Engine Constraints	Platform Constraints
Cost	Free (Your API Key)	Freemium	Freemium	Paid License	Freemium

• BitMagic excels at quickly assembling playable levels using a vast library of pre-made assets and multiplayer "co-prompting," but the underlying logic is often hidden.
• Rosebud AI focuses on web-based games (often 2D or simple 3D) with a "Vibe Coding" interface that assists in asset generation.
• Buildbox 4 uses AI to assist its existing No-Code drag-and-drop workflow, making it great for non-coders but less flexible for custom logic.
• Sketchbook generates raw, editable TypeScript. It teaches you how the game works rather than just showing you the result, making it ideal for developers and prototyping.

🤖 Update Without a PC - AI-Powered Development

This project features automated GitHub Actions that compile and publish your code automatically on every push. You can develop and update the entire project from anywhere using AI coding assistants - no local PC setup required!

How It Works

The GitHub Actions workflow automatically:

• ✅ Builds the web application
• ✅ Compiles signed Android APK
• ✅ Generates Android App Bundle (AAB) for Google Play
• ✅ Creates a GitHub Release with downloadable artifacts
• ✅ Deploys to GitHub Pages (if configured)

Triggered on: Every push to master branch or manual workflow dispatch

Update from Anywhere with AI Assistants

You can edit code directly on GitHub using these AI-powered tools:

1. GitHub Copilot (github.com/copilot)

   • Edit files directly in your browser on GitHub.com
   • AI suggestions appear as you type
   • Commit changes directly to trigger automatic build

2. Cursor Web Agent (cursor.com)

   • Use Cursor's web interface to edit GitHub repositories
   • Full AI assistance with context awareness
   • Push changes to automatically trigger builds

3. Google Jules (jules.google.com)

   • Google's AI coding assistant
   • Can directly interact with GitHub repositories
   • Make changes that automatically trigger the build pipeline

Workflow Steps

1. Make changes using any AI assistant (or GitHub's web editor)
2. Commit to master branch
3. Automatic build starts via GitHub Actions
4. Download APK from the new release (appears in ~5-10 minutes)
5. Install on device or publish to Google Play Store

No local development environment needed - code, compile, and deploy all from the cloud! 🚀

Running Tests

---

Features

🏎️ Advanced Vehicle Physics

Sketchbook implements a high-fidelity vehicle simulation model.

• Raycast Vehicle System: Uses raycasting for wheel detection, ensuring smooth movement over uneven terrain.
• Dynamic Suspension: Configurable stiffness, rest length, and travel for realistic chassis interaction.
• Drivetrain Simulation: Supports AWD, RWD, and FWD configurations with realistic engine force and gear shifting curves.
• Air Control: Arcade-style air control mechanics allow for stunts and mid-air reorientation.

🏃 3rd Person Character Controller

A versatile character system designed for exploration and interaction.

• State Machine Architecture: Robust state handling (Idle, Walk, Run, Sprint, Jump, Falling, Driving, Entering/Exiting vehicles).
• Capsule Physics: Physics-based movement using a capsule collider preventing tunneling and ensuring proper collision response.

🤖 Generative AI (Text-to-World)

The core innovation of Sketchbook.

• Context-Aware Generation: The AI is fed the current state of the codebase, type definitions, and helper libraries, allowing it to write code that actually works within the engine's specific architecture.
• Real-time Code Injection: Generated code is executed immediately. No build steps required.
• Multi-Model Support: Plug-and-play support for Anthropic (Claude), OpenAI (GPT-4), Google (Gemini), and local models via HuggingFace.

---

Technical Architecture

Tech Stack

• Language: TypeScript
• Renderer: Three.js
• Physics: Cannon-es (Fork of cannon.js)
• Build Tool: Webpack
• UI: Vue.js (for the overlay), Dat.GUI (for debugging)
• Editor: Monaco Editor

Core Concepts

• World (src/ts/world/World.ts): The central hub. Manages the physics world, graphics scene, update loops, and entity registry.
• Entities (src/ts/core/IUpdatable.ts): Almost everything is an IUpdatable. The World iterates through these every frame.
• Characters (src/ts/characters/Character.ts): Complex composite objects containing:
  • CharacterCapsule: The physical representation.
  • CharacterState: The logic representation (StateMachine).
  • Model: The visual representation.

---

Getting Started

Prerequisites

• Node.js (v16.0.0 or higher)
• npm (v7.0.0 or higher)

Installation

1. Clone the Repository

   git clone https://github.com/friuns2/Sketchbook.git
   cd Sketchbook

2. Install Dependencies

   npm install

   Note: If you encounter peer dependency warnings, they can usually be ignored.

3. Start Development Server

   npm run dev

   The application will be accessible at http://localhost:8080.

4. Build for Production

   npm run build

   Output is generated in the build/ directory.

---

Configuration

To unlock the AI features, you must configure your API keys. Sketchbook looks for a localSettings.js file in the root directory.

1. Create localSettings.js in the project root.
2. Paste the following configuration (adjusting for your preferred provider):

// localSettings.js

// === OPTION 1: OpenRouter (Recommended for best performance) ===
// Access Claude 3.5 Sonnet, GPT-4o, etc.
settings.model.selected = "anthropic/claude-3-haiku";
settings.apiUrl = "https://openrouter.ai/api/v1";
settings.apiKey = "sk-or-v1-YOUR-OPENROUTER-KEY";

// === OPTION 2: Google Gemini ===
// settings.model.selected = "gemini-1.5-flash-latest";
// settings.apiKey = "AIzaSy-YOUR-GEMINI-KEY";

// === OPTION 3: Hugging Face ===
// settings.apiKey = "hf_YOUR-HUGGINGFACE-TOKEN";

Note: localSettings.js is git-ignored to prevent accidental key leakage.

---

Build

Character Mode

Key	Action
W, A, S, D	Move Character
Shift	Sprint
Space	Jump
F	Enter Vehicle (Driver)
G	Enter Vehicle (Passenger)
E	Interact / Use Item
V	Toggle First/Third Person
Shift + C	Toggle Free Camera
Alt + ←/→	Undo / Redo AI Generation

Vehicle Mode

Key	Action
W	Throttle
S	Brake / Reverse
A, D	Steer Left / Right
Space	Handbrake
F	Exit Vehicle
X	Switch Seat
V	Camera View

Editor & AI

Key	Action
` (Backtick)	Open AI Prompt Bar
Ctrl + Enter	Run Code (in Editor)

---

Credits

Core Engineering: Ján Bláha (Swift502) Original Concept: A 3D portfolio project evolving into an AI sandbox. Libraries: Three.js, Cannon-es, PeerJS, Vue.js, Monaco Editor.

---

License

Copyright (c) 2024. All rights reserved. (License TBD - Check repository root for updates)