Local server, mocked responses and testing of notebooks

Makefile

@@ -1,4 +1,4 @@
-.PHONY:  test test-github jupyter install
+.PHONY:  test test-github jupyter install mock-server
 
 SHELL := /bin/bash
 uname=$(shell uname -s)
@@ -54,6 +54,13 @@ docs-deploy:
 	    mkdocs gh-deploy && \
 	    deactivate
 
+# run mock server for local development and testing
+mock-server:
+	@echo "Starting mock server..."
+	@source $(CURDIR)/venv/bin/activate && \
+	    python scripts/run_mock_server.py $(if $(PORT),$(PORT),8000) && \
+	    deactivate
+
 
 test-github-live:
 	pytest -v --ignore=tests/test_config.py --ignore=tests/test_context.py

docs/dd/how-to/proteins.md

@@ -7,16 +7,6 @@ The [`Protein` class](../ref/protein.md)  is the primary way to work with protei
 ## Constructing a protein
 
 
-### From a common name
-
-A protein can be constructed by a common name or text identifier as follows:
-
-```python
-from deeporigin.drug_discovery import Protein
-protein = Protein.from_name("HIV-1 protease")
-```
-The top hit on the PDB will be retrieved. 
-
 ### From a file
 
 A protein can be constructed from a file:

docs/dev/mock-server.md

@@ -0,0 +1,109 @@
+# Mock Server for Local Development
+
+The mock server is a local test server that mimics the DeepOrigin Platform API. It's useful for local development and testing without making real API calls to the platform.
+
+## Overview
+
+The mock server (`tests/mock_server.py`) provides mock responses for all API endpoints used by the DeepOriginClient. It runs locally using FastAPI and uvicorn, and serves responses based on fixture data stored in `tests/fixtures/`.
+
+## Running the Mock Server
+
+### Standalone Script
+
+To run the mock server standalone for local development:
+
+```bash
+python scripts/run_mock_server.py [PORT]
+```
+
+Where `PORT` is the port number to run the server on (default: 8000).
+
+Example:
+
+```bash
+python scripts/run_mock_server.py 8000
+```
+
+The server will start and display:
+
+```bash
+Mock server running at http://127.0.0.1:8000
+Press Ctrl+C to stop...
+```
+
+Press `Ctrl+C` to stop the server.
+
+### Using in Tests
+
+The mock server is automatically started when running tests with the `--mock` flag:
+
+```bash
+pytest --mock
+```
+
+This starts the server for the duration of the test session and automatically stops it when tests complete.
+
+## Configuring Your Client
+
+To use the mock server with your code, configure the `DeepOriginClient` to point to the mock server URL:
+
+```python
+from deeporigin.platform.client import DeepOriginClient
+
+client = DeepOriginClient(
+    token="test-token",  # Any token works with the mock server
+    org_key="deeporigin",  # Use any org_key
+    base_url="http://127.0.0.1:8000",  # Mock server URL
+    env="local",
+)
+```
+
+## Available Endpoints
+
+The mock server implements the following endpoints:
+
+- **Files API**: List, upload, download, and delete files
+- **Tools API**: List tools and tool definitions
+- **Functions API**: List functions and run function executions
+- **Executions API**: List executions, get execution details, cancel/confirm executions
+- **Clusters API**: List available clusters
+- **Organizations API**: List organization users
+- **Health Check**: `/health` endpoint
+
+## Fixtures
+
+The mock server uses fixture files from `tests/fixtures/` to provide realistic responses. Key fixtures include:
+
+- `execution_example.json`: Template for execution responses
+- `molprops_serotonin.json`: Molecular properties data for testing
+- `abfe/progress-reports.json`: ABFE progress report data
+
+## Extending the Mock Server
+
+To add new endpoints or modify existing ones, edit `tests/mock_server.py`:
+
+1. Add a new route handler in the `_setup_routes()` method
+2. Optionally add fixture files in `tests/fixtures/` if you need realistic data
+3. Use `_load_fixture()` to load JSON fixtures
+
+Example:
+
+```{.python notest}
+@self.app.get("/tools/{org_key}/custom-endpoint")
+def custom_endpoint(org_key: str) -> dict[str, Any]:
+    """Custom endpoint handler."""
+    fixture_data = self._load_fixture("custom_fixture")
+    return fixture_data
+```
+
+## Limitations
+
+The mock server is designed for testing and development purposes. It has some limitations:
+
+- Authentication is not validated (any token works)
+- File storage is in-memory and lost when the server stops
+- Some complex API behaviors may not be fully replicated
+- Rate limiting and other production features are not implemented
+
+For production use, always use the real DeepOrigin Platform API.
+

mkdocs.yaml

@@ -85,6 +85,7 @@ nav:
 - Developing: 
   - Installing: dev/install.md
   - Clients: dev/clients.md
+  - Mock server: dev/mock-server.md
   - Job widget: dev/job-widget.md
 - Changelog: changelog.md
 - Support: https://www.support.deeporigin.com/servicedesk/customer/portals

scripts/run_mock_server.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+"""Standalone script to run the mock server for local development and testing.
+
+This script starts a local mock server that mimics the DeepOrigin Platform API.
+It's useful for local development and testing without making real API calls.
+
+Usage:
+    python scripts/run_mock_server.py [OPTIONS]
+
+Options:
+    PORT: Port number to run the server on (default: 8000)
+    --abfe-duration SECONDS: Duration for ABFE executions in seconds (default: 300)
+
+Examples:
+    python scripts/run_mock_server.py 8000
+    python scripts/run_mock_server.py --abfe-duration 600
+    python scripts/run_mock_server.py 8080 --abfe-duration 120
+"""
+
+from __future__ import annotations
+
+import argparse
+import importlib.util
+from pathlib import Path
+
+# Load mock_server module directly using importlib instead of modifying sys.path
+# This is cleaner than sys.path manipulation and avoids polluting the import system
+project_root = Path(__file__).parent.parent
+mock_server_path = project_root / "tests" / "mock_server.py"
+if not mock_server_path.exists():
+    raise FileNotFoundError(
+        f"Could not find mock_server.py at {mock_server_path}. "
+        "Make sure you're running this script from the project root."
+    )
+spec = importlib.util.spec_from_file_location("mock_server", mock_server_path)
+if spec is None or spec.loader is None:
+    raise ImportError(f"Could not create module spec for {mock_server_path}")
+mock_server_module = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(mock_server_module)
+MockServer = mock_server_module.MockServer
+
+
+def main() -> None:
+    """Run the mock server."""
+    parser = argparse.ArgumentParser(
+        description="Run the DeepOrigin Platform API mock server",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog=__doc__,
+    )
+    parser.add_argument(
+        "port",
+        type=int,
+        nargs="?",
+        default=8000,
+        help="Port number to run the server on (default: 8000)",
+    )
+    parser.add_argument(
+        "--abfe-duration",
+        type=float,
+        default=300.0,
+        help="Duration for ABFE executions in seconds (default: 300)",
+    )
+
+    args = parser.parse_args()
+
+    # Create MockServer instance
+    server = MockServer(port=args.port)
+
+    # Set execution durations
+    server._mock_execution_durations["deeporigin.abfe-end-to-end"] = args.abfe_duration
+
+    # Run with uvicorn directly - this blocks, which is fine for a dev script
+    import uvicorn
+
+    print(f"Starting mock server on http://127.0.0.1:{args.port}")
+    print(f"ABFE execution duration: {args.abfe_duration} seconds")
+    print("Press Ctrl+C to stop...")
+    print()
+    uvicorn.run(server.app, host="127.0.0.1", port=args.port, log_level="info")
+
+
+if __name__ == "__main__":
+    main()

src/drug_discovery/abfe.py

@@ -189,8 +189,6 @@ def _get_ligands_to_run(
         else:
             ligand_hashes_already_run = set()
 
-        print(ligand_hashes_already_run)
-
         # no re-run, remove already run ligands
         ligands_to_run = [
             ligand

src/functions/sysprep.py

@@ -44,8 +44,8 @@ def run_sysprep(
         "use_cache": use_cache,
     }
 
-    protein.upload()
-    ligand.upload()
+    protein.upload(client=client)
+    ligand.upload(client=client)
 
     response = client.functions.run(
         key="deeporigin.system-prep",

src/platform/job.py

@@ -181,17 +181,6 @@ def sync(self):
             self._attributes = result
             self.status = result.get("status")
 
-            # Quick and dirty logging for analysis - append progressReport to file
-            try:
-                progress_data = json.loads(result["progressReport"])
-            except Exception:
-                progress_data = result["progressReport"]
-
-            log_file = Path(f"{self._id}.txt")
-            with open(log_file, "a") as f:
-                f.write(json.dumps(progress_data, indent=2))
-                f.write("\n\n")
-
     def _get_running_time(self) -> Optional[int]:
         """Get the running time of the job.
 
@@ -419,7 +408,7 @@ def show(self):
         rendered_html = self._render_job_view()
         display(HTML(rendered_html))
 
-    def watch(self):
+    def watch(self, *, interval: float = 5.0):
         """Start monitoring job progress in real-time.
 
         This method initiates a background task that periodically updates
@@ -458,7 +447,7 @@ async def update_progress_report():
             """Update and display job progress at regular intervals.
 
             This coroutine runs in the background, updating the display
-            with the latest job status and progress every 5 seconds.
+            with the latest job status and progress every `interval` seconds.
             It automatically stops when the job reaches a terminal state.
             """
             try:
@@ -487,7 +476,7 @@ async def update_progress_report():
                         )
 
                     # Always sleep 5 seconds before next attempt
-                    await asyncio.sleep(5)
+                    await asyncio.sleep(interval)
             finally:
                 # Perform a final non-blocking refresh and render to clear spinner
                 if self._display_id is not None: