Skip to content

Getting started

Simplified Execution with env.import_module

To demonstrates the most straightforward way to use Wetlands, we will create an environment, install cellpose, and run a segmentation function defined in a separate file (example_module.py) within that isolated environment.

Let's see the main script getting_started.py step by step.

We will segment the image img02.png (available here).

from pathlib import Path
image_path = Path("img02.png")
segmentation_path = image_path.parent / f"{image_path.stem}_segmentation.png"

1. Initialize the Environment Manager

We start by initializing the EnvironmentManager. We provide: - A wetlands_instance_path where Wetlands stores logs and debug information (defaults to "wetlands/"). - Optionally, a conda_path where Wetlands should look for an existing Pixi (or Micromamba) installation or where it should download and set up a new one. If not provided, it defaults to wetlands_instance_path / "pixi".

from wetlands.environment_manager import EnvironmentManager

environment_manager = EnvironmentManager()
# Or with explicit paths:
# environment_manager = EnvironmentManager(
#     wetlands_instance_path="wetlands_state",
#     conda_path="path/to/pixi/"
# )

Note

EnvironmentManager also accepts a main_conda_environment_path argument, useful if Wetlands is used in a conda environment (e.g. environment_manager = EnvironmentManager(main_conda_environment_path="/path/to/project/environment/")). Wetlands will activate this main environment and check if the installed packages satisfy the requirements when creating new environments. If the required dependencies are already installed in the main environment, EnvironmentManager.create() will return the main enviroment instead of creating a new one. The modules will be called directly, bypassing the Wetlands communication server.

Warning

On Windows, spaces are not allowed in the conda_path argument of EnvironmentManager().

2. Create (or get) an Environment and Install Dependencies

Next, we define and create the Conda environment. We give it a name ("cellpose_env") and specify its dependencies using a dictionary. Here, we require cellpose version 3.1.0, to be installed via Conda. If an environment with this name already exists, Wetlands will use it (and ignore the dependencies); otherwise, it will create it and install the specified dependencies. The create method returns an Environment object.

env = environment_manager.create(
    "cellpose_env",
    {"conda": ["cellpose==3.1.0"]}
)

Note

If a main_conda_environment_path was provided when instanciating the EnvironmentManager, Wetlands will check if cellpose==3.1.0 is already installed in the main environment and return it if it is the case. If main_conda_environment_path is not provided but the required dependencies are only pip packages, Wetlands will check if the dependencies are installed in the current python environment and return it if it is the case.

Reusing existing environments with use_existing=True

You can pass use_existing=True to create() to search for and reuse an existing environment that satisfies the dependencies. This includes the main environment. If no environment satisfies the requirements, a new one will be created. By default, use_existing=False, which always creates a new environment.

# Return main or existing environment if it satisfies the dependencies
env = environment_manager.create(
    "cellpose_env",
    {"conda": ["cellpose==3.1.0"]},
    use_existing=True  # Check if any existing env satisfies the dependencies
)

Specifying dependencies

See the dependencies page to learn more on specifying dependencies. Wetlands supports PEP 440 version specifiers, so you can use flexible version constraints like "numpy>=1.20,<2.0", "scipy~=1.5", or "pandas!=1.0.0". You can also use EnvironmentManager.create_from_config() and provide a requirements.txt, environment.yml, pyproject.toml or pixi.toml file for your dependencies.

Load an existing environment

You can also load an existing environment with environment.load("env_name", Path("Path/to/existing/environment/pyproject.toml")). See EnvironmentManager.load().

3. Launch the Environment's Communication Server

For Wetlands to execute code within the isolated environment (using import_module or execute), we need to launch its background communication server. This server runs as a separate process inside the cellpose_env and listens for commands from our main script.

env.launch()

4. Import and Execute Code in the Environment via Proxy

This is where the core Wetlands interaction happens. We use env.import_module("example_module.py") to gain access to the functions defined in example_module.py. Wetlands doesn't actually import the module into the main process; instead, it returns a proxy object. When we call a method on this proxy object (like example_module.segment(...)), Wetlands intercepts the call, sends the function name and arguments to the server running in the cellpose_env, executes the real function there, and returns the result back to the main script. File paths and other pickleable arguments are automatically transferred.

print("Importing module in environment...")
example_module = env.import_module("example_module.py")

print(f"Running segmentation on {image_path}...")
diameters = example_module.segment(str(image_path), str(segmentation_path))

print(f"Segmentation complete. Found diameters of {diameters} pixels.")

Alternatively, we could use env.execute() directly:

print(f"Running segmentation on {image_path}...")
args = (str(image_path), str(segmentation_path))
diameters = env.execute("example_module.py", "segment", args)

print(f"Segmentation complete. Found diameters of {diameters} pixels.")

Function arguments must be serializable

The arguments of the segment function will be send to the other process via multiprocessing.connection.Connection.send() so the objects must be picklable.

5. Clean Up

Finally, we tell Wetlands to shut down the communication server and clean up resources associated with the launched environment.

print("Exiting environment...")
env.exit()

print("Done.")
getting_started.py source code 
from pathlib import Path
from wetlands.environment_manager import EnvironmentManager
import urllib.request
import logging


def initialize(pip_deps=[]):
    # Initialize the environment manager
    # Wetlands will store logs and state in the wetlands_instance_path (defaults to "wetlands/")
    # Pixi/Micromamba will be installed in wetlands_instance_path/pixi by default
    logging.getLogger("wetlands").addHandler(logging.StreamHandler())
    environment_manager = EnvironmentManager()

    # Create and launch an isolated Conda environment named "cellpose"
    env = environment_manager.create("cellpose", {"conda": ["cellpose==3.1.0"], "pip": pip_deps})
    env.launch()

    # Download example image from cellpose
    image_path = Path("cellpose_img02.png")
    image_url = "https://www.cellpose.org/static/images/img02.png"

    with urllib.request.urlopen(image_url) as response:
        image_data = response.read()

    with open(image_path, "wb") as handler:
        handler.write(image_data)

    return image_path, env


if __name__ == "__main__":
    # Initialize: create the environment manager, the Cellpose conda environment, and download the image to segment
    image_path, env = initialize()

    # Import example_module in the environment
    example_module = env.import_module("example_module.py")
    # exampleModule is a proxy to example_module.py in the environment,
    # calling exampleModule.function_name(args) will run env.execute(module_name, function_name, args)
    diameters = example_module.segment(str(image_path))

    # Or use env.execute() directly to call a function in a module
    # diameters = env.execute("example_module.py", "segment", (image_path))

    # Alternatively, use env.run_script() to run an entire Python script
    # env.run_script("script.py", args=(str(image_path)))

    print(f"Found diameters of {diameters} pixels.")

    # Clean up and exit the environment
    env.exit()

Now, let's look at the example_module.py file. This code contains the actual segmentation logic and is executed inside the isolated cellpose_env when called via the proxy object.

Define the Segmentation Function

The module defines a segment function that takes input/output paths and other parameters. It uses a global variable model to potentially cache the loaded Cellpose model between calls within the same environment process lifetime.

# example_module.py
from pathlib import Path
from typing import Any, cast

model = None

def segment(
    input_image: Path | str,
    segmentation: Path | str,
    model_type="cyto",
    use_gpu=False,
    channels=[0, 0],
    auto_diameter=True,
    diameter=30,
):
    """Performs cell segmentation using Cellpose."""
    global model

    input_image = Path(input_image)
    if not input_image.exists():
        raise FileNotFoundError(f"Error: input image {input_image}"\
                                "does not exist.")

Import Dependencies (Inside the Environment)

Crucially, the necessary libraries (cellpose, numpy) are imported within this function, meaning they are resolved using the packages installed inside the isolated cellpose_env, not the main script's environment. This is important to enable the main script to import example_module.py without raising a ModuleNotFoundError. In this way, the main script can see the functions defined in example_module.py. This is only necessary when using the proxy object (env.import_module("example_module.py") then example_module.function(args)) but it is not required when using env.execute("example_module.py", "function", (args)) directly.

    print(f"[[1/4]] Load libraries and model '{model_type}'")
    import cellpose.models
    import cellpose.io
    import numpy as np

Using try catch to prevent ModuleNotFoundError

A better approach is to use a try statement at the beginning of example_module.py to fail silently when importing modules which are not accessible in the main environment, like so:

try:
    import cellpose.models
    import cellpose.io
    import numpy as np
except ModuleNotFoundError:
    pass
...

This allows: - to access the function definitions in the main environment (even if we won't be able to execute them in the main environment), - to import the modules for all functions defined in example_module.py in the cellpose_env.

Load Model and Process Image

The code proceeds to load the Cellpose model (if not already cached) and the input image. All this happens within the context of the cellpose_env.

    if model is None or model.cp.model_type != model_type:
        print("Loading model...")
        gpu_flag = str(use_gpu).lower() == 'true'
        model = cellpose.models.Cellpose(gpu=gpu_flag, model_type=model_type)

    print(f"[[2/4]] Load image {input_image}")
    image = cast(np.ndarray, cellpose.io.imread(str(input_image)))

Perform Segmentation

The core segmentation task is performed using the loaded model and image. Any exceptions raised here will be captured by Wetlands and re-raised in the main script.

    print(f"[[3/4]] Compute segmentation for image shape {image.shape}")
    try:
        kwargs: Any = dict(diameter=int(diameter)) if auto_diameter else {}
        masks, _, _, diams = model.eval(image, channels=channels, **kwargs)
    except Exception as e:
        print(f"Error during segmentation: {e}")
        raise e
    print("Segmentation finished (inside environment).")

Save Results and Return Value

The segmentation results (masks) are saved to disk, potentially renaming the output file. The function then returns the calculated cell diameters (diams). This return value is serialized by Wetlands and sent back to the main script.

    if segmentation is None:                # If segmentation is None: return all results
        return masks, flows, styles, diams

    segmentation_path = Path(segmentation)
    print(f"[[4/4]] Save segmentation to {segmentation_path}")

    cellpose.io.save_masks(image, masks, flows, str(input_image), png=True)
    default_output = input_image.parent / f"{input_image.stem}_cp_masks.png"

    if default_output.exists():
        if segmentation_path.exists():
            segmentation_path.unlink()
        default_output.rename(segmentation_path)
        print(f"Saved mask: {segmentation_path}")
    else:
        print("Warning: Segmentation mask file was not generated by cellpose.")

    return diams
example_module.py source code 
from pathlib import Path
from typing import TYPE_CHECKING, Any

model = None

if TYPE_CHECKING:
    import numpy


def segment_image(
    image: numpy.ndarray,
    model_type="cyto",
    use_gpu=False,
    channels=[0, 0],
    auto_diameter=True,
    diameter=30,
) -> Any:
    global model

    print("Loading libraries...")
    import cellpose.models  # type: ignore

    if model is None or model.cp.model_type != model_type:
        print(f"Loading model {model_type}...")
        model = cellpose.models.Cellpose(gpu=True if use_gpu == "True" else False, model_type=model_type)

    print("Compute segmentation...")
    try:
        kwargs: Any = dict(diameter=int(diameter)) if auto_diameter else {}
        masks, flows, styles, diams = model.eval(image, channels=channels, **kwargs)
    except Exception as e:
        print(e)
        raise e
    print("Segmentation finished.")
    return masks, flows, styles, diams


def segment(
    input_image: Path | str,
    model_type="cyto",
    use_gpu=False,
    channels=[0, 0],
    auto_diameter=True,
    diameter=30,
    return_segmentation=False,  # return segmentation or save it in a file
) -> Any:
    global model

    import cellpose.io  # type: ignore

    input_image = Path(input_image)
    image = cellpose.io.imread(input_image)

    masks, flows, styles, diams = segment_image(image, model_type, use_gpu, channels, auto_diameter, diameter)

    # If return_segmentation: return masks, otherwise save them and return diams
    if return_segmentation:
        return masks

    # save results as png
    cellpose.io.save_masks(image, masks, flows, str(input_image), png=True)

    return diams

Summary of Example 1 Flow:

The main script uses EnvironmentManager to prepare an isolated environment. [env.launch()][wetlands.environment_manager.Environment.launch] starts a hidden server in that environment. env.import_module() provides a proxy, and calling functions on the proxy executes the code (like example_module.segment) within the isolated environment, handling data transfer automatically. env.exit() cleans up the server process.

Next Steps