Handle execution and provisioning errors

Wetlands reports environment setup failures separately from worker-call failures. Catch the specific public exception so your application can show a useful message and choose a safe recovery.

Complete example

from __future__ import annotations

from pathlib import Path

from wetlands import (
    EnvironmentManager,
    EnvironmentSpec,
    ExecutionError,
    LocalPackage,
    PostInstallCommand,
    ProvisioningError,
)


def main(root: Path = Path("wetlands")) -> dict[str, object]:
    """Show structured worker and provisioning failures."""
    example_directory = Path(__file__).parent
    worker_spec = EnvironmentSpec(
        python="3.12.*",
        conda=("pip",),
        local=(LocalPackage(example_directory),),
    )
    remote_category = ""
    remote_target = ""
    remote_type = ""
    remote_message = ""
    remote_traceback = ""
    provisioning_stage = ""
    provisioning_command = ""
    provisioning_returncode: int | None = None
    provisioning_stderr: tuple[str, ...] = ()

    with EnvironmentManager(root=root) as manager:
        environment = manager.provision("docs-examples", worker_spec).wait_for()
        with environment.start() as pool:
            task = pool.submit_import("example_module:raise_example_error")
            try:
                task.wait_for()
            except ExecutionError as error:
                failure = error.failure
                remote = failure.remote_exception
                remote_category = failure.category.value
                remote_target = failure.call_target or ""
                remote_traceback = failure.traceback or ""
                print(f"Category: {remote_category}")
                print(f"Target: {failure.call_target}")
                if remote is not None:
                    remote_type = remote.type_name or ""
                    remote_message = remote.message or ""
                    print(f"Remote error: {remote.type_name}: {remote.message}")
                print(remote_traceback)

        failing_spec = EnvironmentSpec(
            python="3.12.*",
            post_install=(
                PostInstallCommand(
                    (
                        "python",
                        "-c",
                        "import sys; print('deliberate setup failure', file=sys.stderr); raise SystemExit(7)",
                    )
                ),
            ),
        )
        try:
            manager.provision("docs-provisioning-error", failing_spec).wait_for()
        except ProvisioningError as error:
            failure = error.failure
            provisioning_stage = failure.stage
            provisioning_command = failure.command or ""
            provisioning_returncode = failure.returncode
            provisioning_stderr = failure.stderr_tail
            print(f"Provisioning stage: {failure.stage}")
            print(f"Safe command: {failure.command}")
            print(f"Return code: {failure.returncode}")
            print(*failure.stderr_tail[-3:], sep="\n")

        corrected = manager.provision(
            "docs-provisioning-error",
            EnvironmentSpec(python="3.12.*"),
        ).wait_for()
        with corrected.start() as pool:
            retry_result = pool.execute_import("builtins:sum", args=([20, 22],), timeout=30)

    print(f"Corrected retry result: {retry_result}")
    return {
        "remote_category": remote_category,
        "remote_target": remote_target,
        "remote_type": remote_type,
        "remote_message": remote_message,
        "remote_traceback": remote_traceback,
        "provisioning_stage": provisioning_stage,
        "provisioning_command": provisioning_command,
        "provisioning_returncode": provisioning_returncode,
        "provisioning_stderr": provisioning_stderr,
        "retry_result": retry_result,
    }


if __name__ == "__main__":
    main()

The example first catches an exception raised by worker code. It then catches a deliberately failing post-install command and provisions a corrected environment under the same name.

Representative output is:

Category: remote_exception
Target: example_module:raise_example_error
Remote error: ValueError: The worker could not process this input
Traceback (most recent call last):
...
Provisioning stage: post_install
Safe command: python -c ...
Return code: 7
deliberate setup failure
Corrected retry result: 42

Traceback paths and the exact safe command display depend on the local installation.

Worker-call failures

ExecutionError.failure is an ExecutionFailure record. Use its stable category to decide how to respond and its other fields for diagnostics.

For a remote Python exception, remote_exception contains the exception module, type, message, traceback, cause, and context. The original exception object cannot cross the worker boundary and is not re-raised in the host.

Provisioning failures

ProvisioningError.failure identifies the failed stage, safe command display, return code, bounded output tails, environment name, and any cleanup failure.

Wetlands removes an incomplete environment before the operation becomes terminal. A corrected call can therefore use the same environment name, as the example demonstrates.

Command displays and captured output are sanitized, but applications should still avoid printing secrets from their own commands.

Decide what to do

Situation	Inspect	Typical response
Preparation or provisioning failed	PreparationError or ProvisioningError and error.failure.stage	Correct connectivity, dependency, or command configuration, then retry.
Worker function raised	ExecutionError with category REMOTE_EXCEPTION	Report the remote type/message; retry only if the function and input make retry safe.
Unsupported argument or result	ValueEncodingError, ValueDecodingError, or category SERIALIZATION	Convert the value to a supported type before retrying.
Worker crashed or disconnected	Category WORKER_DIED or WORKER_CONNECTION	Record diagnostics; the pool replaces the worker, so a safe idempotent task may be retried.
Waiting timed out	Built-in TimeoutError	Continue waiting or explicitly cancel; the task was not canceled automatically.
Cancellation completed	OperationCanceled	Treat it as an expected user action when cancellation was requested.

See Errors and failure categories for the complete field reference.