Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Git
.git
.gitignore

# Docker
Dockerfile*
docker-compose*.yml
.dockerignore

# Node.js development files
ui/node_modules
ui/.vite
ui/coverage

# Python
__pycache__
*.pyc
*.pyo
*.pyd
.Python
env
pip-log.txt
pip-delete-this-directory.txt
.tox
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.log
.git
.mypy_cache
.pytest_cache
.hypothesis

# IDE
.vscode
.idea
*.swp
*.swo
*~

# OS
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db

# Documentation
README.md
LICENSE
*.md
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#=====================================================================#
# Server & Setup Configuration #
#=====================================================================#

PUID=1000
PGID=1000
94 changes: 94 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
name: React UI CI

on:
push:
branches: [ main, develop, better-hostname-handling ]
paths: [ 'ui/**' ]
pull_request:
branches: [ main ]
paths: [ 'ui/**' ]

jobs:
test-and-build:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./ui

strategy:
matrix:
node-version: [18, 20]

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Setup Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
cache-dependency-path: ui/package-lock.json

- name: Install dependencies
run: npm ci

- name: Lint check
run: npm run lint

- name: Prettier format check
run: npm run format:check

- name: TypeScript type check
run: npm run type-check

- name: Run tests with coverage
run: npm run test:coverage

- name: Upload coverage reports
if: matrix.node-version == 20
uses: codecov/codecov-action@v3
with:
directory: ./ui/coverage
flags: unittests
name: codecov-umbrella

- name: Build production
run: npm run build

- name: Check build size
run: |
echo "Build size analysis:"
du -sh dist/
ls -la dist/

- name: Upload build artifacts
if: matrix.node-version == 20
uses: actions/upload-artifact@v4
with:
name: dist-${{ github.sha }}
path: ui/dist/
retention-days: 7

security-audit:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./ui

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
cache-dependency-path: ui/package-lock.json

- name: Install dependencies
run: npm ci

- name: Run security audit
run: npm audit --audit-level=moderate
17 changes: 17 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
node_modules/
__pycache__/
.DS_Store
.env
npm-debug.log*
yarn-debug.log*
yarn-error.log*
node.zip
.vscode/
.claude/

# Ignore Models for testing
data/*
!data/comfy/user/default/workflows

# Ignore generated project files
gpu_config.json
149 changes: 149 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

ComfyUI-Distributed is a Python extension for ComfyUI that enables distributed and parallel processing across multiple GPUs and machines. It allows users to scale image/video generation and upscaling workflows by leveraging multiple GPU resources locally, remotely, or in the cloud.

## Architecture

### Core Components

**Main Modules:**
- `distributed.py` - Core distributed processing nodes and workflow coordination
- `distributed_upscale.py` - Specialized distributed upscaling functionality
- `__init__.py` - Node registration, ComfyUI integration, and execution patching

**Worker System:**
- **Master**: Main ComfyUI instance that coordinates work distribution
- **Workers**: ComfyUI instances that process tasks (local, remote, or cloud-based)
- `worker_monitor.py` - Monitors master process and terminates workers if master dies

**Utilities (`utils/`):**
- `config.py` - Configuration management and worker setup
- `process.py` - Process lifecycle management and monitoring
- `network.py` - HTTP client/server communication utilities
- `image.py` - Tensor/PIL image conversion utilities
- `async_helpers.py` - Async operation wrappers for ComfyUI integration
- `constants.py` - Shared timeout and configuration constants
- `usdu_managment.py` / `usdu_utils.py` - Ultimate SD Upscale distributed processing

**Frontend (`web/`):**
- `main.js` - Primary UI integration with ComfyUI
- `ui.js` - Worker management interface and controls
- `apiClient.js` - Backend API communication
- `workerUtils.js` - Worker process management utilities

### Key Design Patterns

**Distributed Processing:**
- Jobs are distributed to available workers with load balancing
- Results are collected and aggregated on the master
- Supports both parallel generation (multiple seeds) and distributed upscaling (tile-based)

**Process Management:**
- Workers are spawned as separate ComfyUI processes on different ports
- Master-worker communication via HTTP API
- Automatic worker cleanup when master terminates

**ComfyUI Integration:**
- Custom nodes register through `NODE_CLASS_MAPPINGS`
- Execution validation is patched for dynamic output nodes
- Frontend integrates with ComfyUI's existing UI framework

## Development Commands

### Testing
```bash
# No automated tests - manual testing through ComfyUI workflows
# Use the provided workflow JSON files in /workflows for testing different features
```

### Linting
```bash
# No specific linting commands configured
# Follow Python PEP 8 standards for new code
```

### Configuration
Worker configuration is managed through a JSON config file. The system auto-generates local worker configs on first launch.

### Key Workflow Files
- `/workflows/distributed-txt2img.json` - Basic parallel image generation
- `/workflows/distributed-wan.json` - Parallel video generation
- `/workflows/distributed-upscale.json` - Distributed image upscaling
- `/workflows/distributed-upscale-video.json` - Distributed video upscaling

## Important Implementation Details

### Worker Management
- Workers are auto-discovered for local GPUs on first launch
- Each worker runs on a unique port (8189, 8190, etc.)
- Workers require `--enable-cors-header` flag when using remote/cloud workers
- Process monitoring ensures workers terminate when master dies

### Memory and Performance
- Batch processing limited by `MAX_BATCH` constant (default 20 items)
- Heartbeat monitoring with configurable timeout (default 60s)
- Automatic VRAM cleanup between worker tasks

### Network Communication
- HTTP-based master-worker communication
- Chunked transfer for large image data
- Configurable timeouts for different operation types

### Error Handling
- Graceful worker failure handling with automatic retry
- Process cleanup on master termination
- Validation patching for ComfyUI's execution system

## Planning Document Standards

All planning documents in this repository should follow a consistent, problem-focused approach:

### Document Structure
**Required Sections:**
- **Overview**: Brief description of what the plan aims to achieve
- **Current State**: Current problems and limitations being addressed
- **Project Phases**: Organized phases with problems and tasks
- **Success Criteria**: Measurable outcomes for functional, technical, and UX requirements
- **How to Use This Plan**: Standard guidance for collaborative implementation

### Phase Organization
**Each Phase Should Include:**
- **Problems to Solve**: Clear problem statements (not solutions)
- **Tasks**: Actionable items with checkboxes [ ] for tracking progress
- **Focus**: Problems rather than prescriptive implementation details

### Planning Philosophy
1. **Problem-Focused**: Identify problems to solve, not specific solutions
2. **Collaborative**: Encourage discussion of implementation options
3. **Flexible**: Allow adaptation based on discovery during implementation
4. **Trackable**: Clear tasks that can be checked off as completed
5. **Iterative**: Support refinement based on what we learn

### Example Phase Format
```markdown
### Phase X: Descriptive Name 📝 PLANNED
**Problems to Solve:**
- Problem statement 1
- Problem statement 2
- Problem statement 3

**Tasks:**
- [ ] Task that addresses the problems
- [ ] Another task that addresses the problems
- [ ] Final task for this phase
```

### Status Indicators
- ✅ COMPLETED - Phase has been successfully implemented
- 📝 PLANNED - Phase is defined but not yet started
- 🔄 IN PROGRESS - Phase is currently being worked on

This approach ensures plans remain collaborative tools rather than rigid specifications, allowing teams to work together to find the best solutions for each identified problem.

## Integration Notes

This is a ComfyUI custom node extension. Development should follow ComfyUI's node development patterns and be tested within a ComfyUI environment. The extension requires multiple NVIDIA GPUs or cloud GPU access to be fully functional.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ ComfyUI Distributed supports three types of workers:
```bash
git clone https://github.com/robertvoy/ComfyUI-Distributed.git
```

2. **Restart ComfyUI**
- If you'll be using remote/cloud workers, add `--enable-cors-header` to your launch arguments on the master

Expand Down
8 changes: 7 additions & 1 deletion __init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,13 @@ def patched_execute(self, prompt, prompt_id, extra_data={}, execute_outputs=[]):
NODE_DISPLAY_NAME_MAPPINGS as UPSCALE_DISPLAY_NAME_MAPPINGS
)

WEB_DIRECTORY = "./web"
# Check environment variable to determine UI version
CD_UI_VERSION = os.environ.get('CD_UI_VERSION', 'react')

if CD_UI_VERSION == 'legacy':
WEB_DIRECTORY = "./web"
else:
WEB_DIRECTORY = "./dist"

ensure_config_exists()

Expand Down
Loading
Loading