Fix/global agent bug

.github/copilot-instructions.md

@@ -50,4 +50,8 @@ Make code changes only if you have high confidence they can solve the problem. W
 Confirm the root cause is fixed. Review your solution for logic correctness and robustness. Iterate until you are extremely confident the fix is complete and all tests pass.
 
 7. Final Reflection and Additional Testing
-Reflect carefully on the original intent of the user and the problem statement. Think about potential edge cases or scenarios that may not be covered by existing tests. Write additional tests that would need to pass to fully validate the correctness of your solution. Run these new tests and ensure they all pass. Be aware that there are additional hidden tests that must also pass for the solution to be successful. Do not assume the task is complete just because the visible tests pass; continue refining until you are confident the fix is robust and comprehensive.
+Reflect carefully on the original intent of the user and the problem statement. Think about potential edge cases or scenarios that may not be covered by existing tests. Write additional tests that would need to pass to fully validate the correctness of your solution. Run these new tests and ensure they all pass. Be aware that there are additional hidden tests that must also pass for the solution to be successful. Do not assume the task is complete just because the visible tests pass; continue refining until you are confident the fix is robust and comprehensive.
+
+VERSIONING
+Application versioning remains in `application/single_app/config.py`.
+Deployer and CI/CD versioning lives separately in `deployers/version.txt`; when files under `deployers/` are modified, increment `deployers/version.txt` as part of the same change, defaulting to a patch bump unless a deliberate minor or major compatibility change is intended.

.github/instructions/update_deployer_version.instructions.md

@@ -0,0 +1,19 @@
+---
+applyTo: 'deployers/**'
+---
+
+# Deployer Version Management
+
+When a change modifies files under `deployers/`, include an update to `deployers/version.txt` in the same change.
+
+## Rules
+
+- Keep the deployer version separate from `application/single_app/config.py`.
+- `deployers/version.txt` must contain only a plain semantic version string in the format `X.Y.Z`.
+- Default to a patch increment when a deployer change is made: `1.0.0` -> `1.0.1`.
+- Use a minor or major increment only when the deployer workflow or CI/CD compatibility contract changes intentionally.
+- If the only deployer file being changed is `deployers/version.txt`, do not add an extra bump beyond the intended version update.
+
+## Applies To
+
+This rule covers deployer scripts, `azure.yaml`, `.azure` environment helpers, Bicep/Terraform deployer files, and other deployment workflow assets under `deployers/`.

CLAUDE.md

@@ -62,6 +62,8 @@ return render_template('page.html', settings=public_settings)
 - Version is stored in `config.py`: `VERSION = "X.XXX.XXX"`
 - When incrementing, only change the third segment (e.g., `0.238.024` -> `0.238.025`)
 - Include the current version in functional test file headers and documentation files
+- Deployer CI/CD logic version is tracked separately in `deployers/version.txt`
+- When modifying files under `deployers/`, increment `deployers/version.txt`; default to a patch bump unless a deliberate deployment compatibility change warrants a minor or major increment
 
 ## Documentation Locations
 

application/single_app/Dockerfile

@@ -18,6 +18,22 @@ RUN update-ca-trust enable \
 
 ENV PYTHONUNBUFFERED=1
 
+RUN set -eux; \
+    tdnf install -y curl ca-certificates; \
+    printf '%s\n' \
+        '[microsoft-azurelinux-3-ms-non-oss]' \
+        'name=Microsoft Azure Linux 3 Non-OSS' \
+        'baseurl=https://packages.microsoft.com/azurelinux/3.0/prod/ms-non-oss/$basearch' \
+        'enabled=1' \
+        'gpgcheck=1' \
+        'gpgkey=https://packages.microsoft.com/keys/microsoft.asc' \
+        > /etc/yum.repos.d/microsoft-ms-non-oss.repo; \
+    mkdir -p /opt/microsoft/msodbcsql18; \
+    touch /opt/microsoft/msodbcsql18/ACCEPT_EULA; \
+    tdnf install -y unixODBC unixODBC-devel msodbcsql18; \
+    tdnf clean all; \
+    rm -rf /var/cache/tdnf
+
 RUN set -eux; \
     echo "nonroot:x:${GID}:" >> /etc/group; \
     echo "nonroot:x:${UID}:${GID}:nonroot:/home/nonroot:/bin/bash" >> /etc/passwd; \
@@ -29,6 +45,24 @@ RUN set -eux; \
 
 RUN mkdir -p /app/flask_session && chown -R ${UID}:${GID} /app/flask_session
 RUN mkdir /sc-temp-files && chown -R ${UID}:${GID} /sc-temp-files
+# Preserve the package-selected unixODBC driver registry path for the distroless runtime.
+RUN set -eux; \
+    driver_config_path="$(odbcinst -j | while IFS= read -r line; do case "$line" in DRIVERS*) printf '%s\n' "${line##*: }"; break ;; esac; done)"; \
+    test -n "${driver_config_path}"; \
+    case "${driver_config_path}" in \
+        */odbcinst.ini) driver_config_file="${driver_config_path}" ;; \
+        *) driver_config_file="${driver_config_path}/odbcinst.ini" ;; \
+    esac; \
+    driver_config_dir="${driver_config_file%/odbcinst.ini}"; \
+    test -f "${driver_config_file}"; \
+    mkdir -p /odbc-runtime/usr/lib64 /odbc-runtime/opt "/odbc-runtime${driver_config_dir}" /odbc-runtime/etc; \
+    cp -a "${driver_config_file}" "/odbc-runtime${driver_config_dir}/"; \
+    if [ "${driver_config_dir}" != "/etc" ]; then cp -a "${driver_config_file}" /odbc-runtime/etc/; fi; \
+    cp -a /opt/microsoft /odbc-runtime/opt/; \
+    cp -a /usr/lib64/libodbc.so* /odbc-runtime/usr/lib64/; \
+    cp -a /usr/lib64/libodbcinst.so* /odbc-runtime/usr/lib64/; \
+    cp -a /usr/lib64/libodbccr.so* /odbc-runtime/usr/lib64/; \
+    cp -a /usr/lib64/libltdl.so* /odbc-runtime/usr/lib64/
 
 WORKDIR /app
 
@@ -47,13 +81,15 @@ COPY --from=builder /home/nonroot /home/nonroot
 COPY --from=builder /etc/passwd /etc/passwd
 COPY --from=builder /etc/group /etc/group
 COPY --from=builder /usr/lib/python3.12 /usr/lib/python3.12
+COPY --from=builder /odbc-runtime/ /
 
 USER ${UID}:${GID}
 
 COPY --from=builder --chown=${UID}:${GID} /app /app
 COPY --from=builder --chown=${UID}:${GID} /sc-temp-files /sc-temp-files
 
 ENV HOME=/home/nonroot \
+    LD_LIBRARY_PATH="/usr/lib64:/opt/microsoft/msodbcsql18/lib64" \
     PATH="/home/nonroot/.local/bin:$PATH" \
     PYTHONIOENCODING=utf-8 \
     LANG=C.UTF-8 \

application/single_app/app.py

@@ -54,12 +54,14 @@
 from route_frontend_notifications import *
 
 from route_backend_chats import *
+from route_backend_search import *
 from route_backend_conversations import *
 from route_backend_documents import *
 from route_backend_groups import *
 from route_backend_users import *
 from route_backend_group_documents import *
 from route_backend_models import *
+from route_backend_workflows import *
 from route_backend_safety import *
 from route_backend_feedback import *
 from route_backend_settings import *
@@ -79,6 +81,7 @@
 from route_backend_thoughts import register_route_backend_thoughts
 from route_backend_speech import register_route_backend_speech
 from route_backend_tts import register_route_backend_tts
+from route_backend_collaboration import register_route_backend_collaboration
 from route_enhanced_citations import register_enhanced_citations_routes
 from plugin_validation_endpoint import plugin_validation_bp
 from route_openapi import register_openapi_routes
@@ -873,9 +876,15 @@ def list_semantic_kernel_plugins():
 # ------------------- API Chat Routes --------------------
 register_route_backend_chats(app)
 
+# ------------------- API Search Routes ------------------
+register_route_backend_search(app)
+
 # ------------------- API Conversation Routes ------------
 register_route_backend_conversations(app)
 
+# ------------------- API Collaboration Routes -----------
+register_route_backend_collaboration(app)
+
 # ------------------- API Documents Routes ---------------
 register_route_backend_documents(app)
 
@@ -891,6 +900,9 @@ def list_semantic_kernel_plugins():
 # ------------------- API Model Routes -------------------
 register_route_backend_models(app)
 
+# ------------------- API Workflow Routes ----------------
+register_route_backend_workflows(app)
+
 # ------------------- API Safety Logs Routes -------------
 register_route_backend_safety(app)
 

application/single_app/background_tasks.py

@@ -15,7 +15,14 @@
 from config import cosmos_settings_container, exceptions
 from functions_appinsights import log_event
 from functions_debug import debug_print
+from functions_personal_workflows import (
+    compute_next_run_at,
+    get_due_personal_workflows,
+    get_personal_workflow,
+    update_personal_workflow_runtime_fields,
+)
 from functions_settings import get_settings, update_settings
+from functions_workflow_runner import run_personal_workflow
 
 
 def _get_lock_holder_id():
@@ -303,12 +310,109 @@ def run_retention_policy_loop():
         time.sleep(300)
 
 
+def check_due_workflows_once():
+    """Execute interval-based personal workflows that are due."""
+    settings = get_settings()
+    if not settings.get('allow_user_workflows', True):
+        return []
+
+    due_workflows = get_due_personal_workflows(limit=20)
+    if not due_workflows:
+        return []
+
+    results = []
+    for workflow in due_workflows:
+        workflow_id = str(workflow.get('id') or '').strip()
+        user_id = str(workflow.get('user_id') or '').strip()
+        if not workflow_id or not user_id:
+            continue
+
+        lock_document = acquire_distributed_task_lock(f'workflow_run_{workflow_id}', lease_seconds=900)
+        if not lock_document:
+            continue
+
+        refreshed_workflow = None
+        try:
+            refreshed_workflow = get_personal_workflow(user_id, workflow_id)
+            if not refreshed_workflow:
+                continue
+            if refreshed_workflow.get('trigger_type') != 'interval' or not refreshed_workflow.get('is_enabled', False):
+                continue
+
+            next_run_at = refreshed_workflow.get('next_run_at')
+            if next_run_at:
+                try:
+                    if datetime.fromisoformat(next_run_at) > datetime.now(timezone.utc):
+                        continue
+                except Exception:
+                    pass
+
+            started_at = datetime.now(timezone.utc).isoformat()
+            update_personal_workflow_runtime_fields(
+                user_id,
+                workflow_id,
+                {
+                    'status': 'running',
+                    'last_run_started_at': started_at,
+                    'last_run_trigger_source': 'scheduled',
+                    'last_run_error': '',
+                },
+            )
+
+            result = run_personal_workflow(refreshed_workflow, trigger_source='scheduled')
+            update_fields = dict(result.get('workflow_updates') or {})
+            update_fields['status'] = 'idle'
+            update_fields['next_run_at'] = compute_next_run_at(refreshed_workflow, from_time=datetime.now(timezone.utc))
+            update_personal_workflow_runtime_fields(user_id, workflow_id, update_fields)
+            results.append({'workflow_id': workflow_id, 'success': bool(result.get('success'))})
+        except Exception as exc:
+            log_event(
+                f"[WorkflowScheduler] Error executing workflow {workflow_id}: {exc}",
+                extra={
+                    'workflow_id': workflow_id,
+                    'user_id': user_id,
+                },
+                level=logging.ERROR,
+                exceptionTraceback=True,
+            )
+            if refreshed_workflow:
+                update_personal_workflow_runtime_fields(
+                    user_id,
+                    workflow_id,
+                    {
+                        'status': 'idle',
+                        'last_run_status': 'failed',
+                        'last_run_error': str(exc),
+                        'last_run_at': datetime.now(timezone.utc).isoformat(),
+                        'last_run_trigger_source': 'scheduled',
+                        'next_run_at': compute_next_run_at(refreshed_workflow, from_time=datetime.now(timezone.utc)),
+                    },
+                )
+        finally:
+            release_distributed_task_lock(lock_document)
+
+    return results
+
+
+def run_workflow_scheduler_loop():
+    """Run personal workflow scheduling checks forever."""
+    while True:
+        try:
+            check_due_workflows_once()
+        except Exception as exc:
+            print(f"Error in workflow scheduler check: {exc}")
+            log_event(f"[WorkflowScheduler] Error in workflow scheduler check: {exc}", level=logging.ERROR)
+
+        time.sleep(5)
+
+
 def start_background_task_threads():
     """Start all background task loops for the current process."""
     task_specs = [
         ('Logging timer background task started.', run_logging_timer_loop),
         ('Approval expiration background task started.', run_approval_expiration_loop),
         ('Retention policy background task started.', run_retention_policy_loop),
+        ('Workflow scheduler background task started.', run_workflow_scheduler_loop),
     ]
 
     started_threads = []