Merge Tutorial Branch Into Main (#665)

* Refactor Web-UI  Events, Tasks and Notifications (#660)

* refactor web-ui tasks, events, notifications - use EventStream for sending events

* Bug Fixes - SSE behavior, login.

* modify security token flow to support URL-based login

* resolve 'open button'  confusion

* resolve OPERATION -> OPERATIONAL in all entities

* make valid/invalid all caps

* remove is valid: (true/false) from dataset details and export dataset. modify export dataset ui

* fix Registering Model -> Registering Container

* remove the approved flag in benchmark details

* remove report file field if it does not exist - temp (bug exists)

* remove hashes from entities details

* remove the additional N/A text - add "Owner" before owner ID

* add navbar color (green) and add footer

* fix footer - stick at the bottom

* allow users to select containers based on their types instead of writing the IDs - benchmark registration

* change name, description, and location maxlength - apply migrations

* apply ui changes - footer and navbar

* modify reload time into 1 second - medperf login and logout

* auto approve benchmark for tutorial

* autopopulate some mock benchmarks for tutorials

* fix line length problem

* fix association dropdown ui

* fix hidden error - RuntimeError: Event object at ... is bound to a different event loop

* fix print logs delays

* add codespaces scripts

---------

Co-authored-by: hasan7n <[email protected]>

* pin click version

* fix codespaces script

* clear notifications after logging out

* remove generated_uid from dataset details page

* move print_webui_props to utils

* WebUI E2E Tests (#664)

* Add html attributes required for tests

* fix missing function in common.js

* fix compatibility/workflow tests popup modal title

* fix auto-scroll lag issue

* Add E2E tests for webui

* ci: add GitHub workflow for WebUI E2E tests

* fix tests issues

* fix webui-ci

* fix typo in local-auth env that caused errors while seeding db

* remove auto-login when getting tutorial files - add profile activation test

* remove logout button duplicate, add comment

---------

Co-authored-by: hasan7n <[email protected]>

* add EventsManager class to manage events/buffering.
add 'webui' prefix for maxlogmessages, add other webui configs

* remove the name validator from entities

* remove chrome-setup from auth-ci

---------

Co-authored-by: hasan7n <[email protected]>

Author: mhmdk0

.devcontainer/webui/devcontainer.json

@@ -0,0 +1,10 @@
+{
+  "image": "docker.io/mlcommons/medperf-codespaces:latest",
+    "features": {
+		"ghcr.io/devcontainers/features/docker-in-docker:2": {
+			"moby": false
+		}
+	},
+	"postStartCommand": "bash /workspaces/medperf/tutorials_scripts/codespaces/webui/env_start.sh",
+	"postAttachCommand": "bash /workspaces/medperf/tutorials_scripts/codespaces/webui/run_server_detached.sh"
+}

.github/workflows/auth-ci.yml

@@ -25,13 +25,6 @@ jobs:
       uses: actions/setup-python@v2
       with:
         python-version: '3.9'
-    
-    - name: Setup Chrome
-      run: |
-        sudo apt-get install -y wget
-        wget -O chrome.deb https://dl.google.com/linux/chrome/deb/pool/main/g/google-chrome-stable/google-chrome-stable_126.0.6478.126-1_amd64.deb
-        sudo apt update && sudo apt install ./chrome.deb -y --allow-downgrades
-        rm chrome.deb
 
     - name: Install dependencies
       working-directory: .

.github/workflows/unittests.yml

@@ -36,7 +36,7 @@ jobs:
         # Ignore E231, as it is raising warnings with auto-generated code.
         flake8 . --count --max-complexity=10 --max-line-length=127 --ignore F821,W503,E231 --statistics --exclude=examples/,"*/migrations/*",cli/medperf/templates/
     - name: Test with pytest
-      working-directory: ./cli
+      working-directory: ./cli/medperf/tests
       run: |
         pytest
     - name: Set server environment vars

.github/workflows/webui-ci.yml

@@ -0,0 +1,53 @@
+name: WebUI Integration workflow
+
+on: pull_request
+
+jobs:
+  setup:
+    name: webui-integration-test
+    runs-on: ubuntu-latest
+    steps:
+    - name: Checkout repository
+      uses: actions/checkout@v2
+
+    - name: Set up Python 3.9
+      uses: actions/setup-python@v2
+      with:
+        python-version: '3.9'
+    
+    - name: Install dependencies
+      working-directory: .
+      run: |
+        python -m pip install --upgrade pip
+        pip install -e cli/
+        pip install -r cli/test-requirements.txt
+        pip install -r server/requirements.txt
+        pip install -r server/test-requirements.txt
+
+    - name: Set server environment vars
+      working-directory: ./server
+      run: cp .env.local.local-auth .env
+
+    - name: Run postgresql server in background
+      working-directory: ./server
+      run: sh run_dev_postgresql.sh && sleep 6
+
+    - name: Run django server in background with generated certs
+      working-directory: ./server
+      run: sh setup-dev-server.sh & sleep 6
+
+    - name: Reset and seed database with tutorial data
+      working-directory: ./server
+      run: sh reset_db.sh && python seed.py --demo tutorial
+
+    - name: Set up tutorial files
+      working-directory: .
+      run: bash tutorials_scripts/setup_webui_tutorial.sh
+
+    - name: Run fastapi server in background
+      working-directory: .
+      run: medperf_webui & sleep 6
+
+    - name: Run webUI tutorial E2E tests with pytest
+      working-directory: ./cli/medperf/web_ui/tests/e2e/
+      run: pytest

cli/cli_tests.sh

@@ -173,15 +173,6 @@ print_eval medperf benchmark submit --name bmk --description bmk --demo-url $DEM
 checkFailed "Benchmark submission failed"
 BMK_UID=$(medperf benchmark ls | grep bmk | tail -n 1 | tr -s ' ' | cut -d ' ' -f 2)
 echo "BMK_UID=$BMK_UID"
-
-# Approve benchmark
-echo $ADMIN
-echo $MOCK_TOKENS_FILE
-ADMIN_TOKEN=$(jq -r --arg ADMIN $ADMIN '.[$ADMIN]' $MOCK_TOKENS_FILE)
-echo $ADMIN_TOKEN
-checkFailed "Retrieving admin token failed"
-print_eval "curl -sk -X PUT $SERVER_URL$VERSION_PREFIX/benchmarks/$BMK_UID/ -d '{\"approval_status\": \"APPROVED\"}' -H 'Content-Type: application/json' -H 'Authorization: Bearer $ADMIN_TOKEN' --fail-with-body"
-checkFailed "Benchmark approval failed"
 ##########################################################
 
 echo "\n"

cli/medperf/config.py

@@ -232,6 +232,13 @@
 ui = "CLI"
 webui = "WEBUI"
 
+# WebUI-related config
+webui_max_log_messages = 200  # Max nb of messages that will appear in LogPanel in WebUI
+webui_max_chunk_age = 2.0  # Max 2 seconds as age of a chunk
+webui_max_chunk_length = 20  # Max 20 events in a chunk
+webui_max_chunk_size = 64 * 1024  # Max 64 Bytes as chunk size
+
+
 default_profile_name = "default"
 testauth_profile_name = "testauth"
 test_profile_name = "local"

cli/medperf/entities/benchmark.py

@@ -22,7 +22,7 @@ class Benchmark(Entity, ApprovableSchema, DeployableSchema):
     what models to run and how to evaluate them.
     """
 
-    description: Optional[str] = Field(None, max_length=20)
+    description: Optional[str] = Field(None, max_length=256)
     docs_url: Optional[HttpUrl]
     demo_dataset_tarball_url: str
     demo_dataset_tarball_hash: Optional[str]

cli/medperf/entities/dataset.py

@@ -22,8 +22,8 @@ class Dataset(Entity, DeployableSchema):
     data preparation output.
     """
 
-    description: Optional[str] = Field(None, max_length=20)
-    location: Optional[str] = Field(None, max_length=20)
+    description: Optional[str] = Field(None, max_length=256)
+    location: Optional[str] = Field(None, max_length=128)
     input_data_hash: str
     generated_uid: str
     data_preparation_mlcube: Union[int, str]

cli/medperf/entities/schemas.py

@@ -11,7 +11,7 @@
 class MedperfSchema(BaseModel):
     for_test: bool = False
     id: Optional[int]
-    name: str = Field(..., max_length=64)
+    name: str = Field(..., max_length=128)
     owner: Optional[int]
     is_valid: bool = True
     created_at: Optional[datetime]
@@ -76,12 +76,6 @@ def empty_str_to_none(cls, v):
             return None
         return v
 
-    @validator("name", pre=True, always=True)
-    def name_max_length(cls, v, *, values, **kwargs):
-        if not values["for_test"] and len(v) > 20:
-            raise ValueError("The name must have no more than 20 characters")
-        return v
-
     class Config:
         allow_population_by_field_name = True
         extra = "allow"

cli/medperf/entities/training_exp.py

@@ -21,7 +21,7 @@ class TrainingExp(Entity, MedperfSchema, ApprovableSchema, DeployableSchema):
     what models to run and how to evaluate them.
     """
 
-    description: Optional[str] = Field(None, max_length=20)
+    description: Optional[str] = Field(None, max_length=256)
     docs_url: Optional[HttpUrl]
     demo_dataset_tarball_url: str
     demo_dataset_tarball_hash: Optional[str]