Troubleshooting

Guide to solving common problems with Polycrate: Docker, workspace, blocks, actions, workflows, SSH and performance, with problem descriptions and solution steps.

Troubleshooting

This guide helps you solve common problems with Polycrate.

Docker problems

"Cannot connect to Docker daemon"

Problem: Polycrate cannot connect to the Docker daemon.

Error: Cannot connect to the Docker daemon at unix:///var/run/docker.sock

Solutions:

  1. Check whether Docker is running: 

    docker ps

  2. Start Docker: 

    # Linux (systemd)
sudo systemctl start docker

# macOS/Windows
# Start Docker Desktop

  3. Check permissions: 

    # Linux - add the user to the docker group
sudo usermod -aG docker $USER
newgrp docker

  4. Check the socket path: 

    ls -la /var/run/docker.sock

"Image not found" or pull errors

Problem: Polycrate cannot find or download the workspace image.

Error: Error response from daemon: pull access denied for cargo.ayedo.cloud/library/polycrate

Solutions:

  1. Check your internet connection

  2. Authenticate with the registry: 

    docker login cargo.ayedo.cloud

  3. Use a different image: 

    polycrate run my-block install --image-ref <alternative-image>

  4. Skip the pull (if the image is already local): 

    polycrate run my-block install --pull false

Container does not start

Problem: The Polycrate container does not start or crashes immediately.

Solutions:

  1. Check the container logs: 

    docker logs <container-id>

  2. Check the Polycrate logs: 

    ls -la .logs/
cat .logs/<latest-log-file>

  3. Increase the log level: 

    polycrate run my-block install --loglevel 2

  4. Test the container manually: 

    docker run -it --rm cargo.ayedo.cloud/library/polycrate:latest /bin/sh

Bind mount error on macOS with OrbStack (0.36.0)

Problem: On macOS with OrbStack (or other alternative Docker runtimes) the action execution fails with a bind mount error. The error message typically contains the path /var/run/com.apple.launchd.*/Listeners.

Error: failed to create container: ... bind mount source path does not exist:
/var/run/com.apple.launchd.XXXXX/Listeners

Cause: The SSH agent socket (SSH_AUTH_SOCK) is mounted into the container. On macOS SSH_AUTH_SOCK points to a launchd socket that OrbStack cannot pass through into the Docker VM.

Solution (since 0.37.0): The SSH agent mount is disabled by default. If you enabled the mount explicitly and this problem occurs, remove the activation:

  • Remove --ssh-agent-mount from the CLI call
  • Remove sshagentmount: true from workspace.poly

Solution (0.36.x): Disable the SSH agent mount with --no-ssh-agent-mount or config.sshagentmount: false.

Action run submission fails: exit code -1 (before 0.36.0)

Problem: An action run is submitted to the Polycrate API, but the API rejects it because the exit code is -1. The API expects exit codes >= 0.

Error: exit_code: Ensure this value is greater than or equal to 0

Cause: Before version 0.36.0, Docker infrastructure errors (container create, attach, start) returned exit code -1, which was an internal sentinel value. The API only accepts POSIX-compliant exit codes (0–255).

Solution: Update to Polycrate CLI >= 0.36.0. All infrastructure errors now return exit code 1:

polycrate update 0.51.3

Workspace problems

"Not a valid workspace"

Problem: Polycrate does not recognize the directory as a workspace.

Error: Not a valid workspace: workspace.poly not found

Solutions:

  1. Check whether workspace.poly exists: 

    ls -la workspace.poly

  2. Initialize a new workspace (in the target directory): 

    mkdir -p ~/polycrate-workspaces/my-workspace
cd ~/polycrate-workspaces/my-workspace
polycrate workspace init --with-name my-workspace

  3. Specify the workspace path explicitly: 

    polycrate run my-block install --workspace /path/to/workspace

Workspace encryption fails

Problem: Encrypting or decrypting the workspace fails.

Solutions:

  1. Check the encryption status: 

    polycrate workspace status

  2. Make sure you are using the correct passphrase

  3. Check whether encrypted files exist: 

    ls -la *.enc

  4. Create a backup before encrypting: 

    cp workspace.poly workspace.poly.backup
polycrate workspace encrypt

Snapshot errors

Problem: The snapshot cannot be generated.

Solutions:

  1. Validate your workspace configuration: 

    polycrate workspace inspect

  2. Check for syntax errors in workspace.poly: 

    # Use a YAML validator
yamllint workspace.poly

  3. Test the snapshot in isolation: 

    polycrate workspace snapshot

Block problems

"Block not found"

Problem: Polycrate cannot find a block.

Error: Block 'my-block' not found in workspace

Solutions:

  1. List all available blocks: 

    polycrate block list

  2. Check the block directory: 

    ls -la blocks/

  3. Pull the block from the registry (full path!): 

    polycrate blocks pull registry.my-org.com/infra/my-block:1.0.0

  4. Check the block configuration: 

    cat blocks/my-block/block.poly

Block validation fails

Problem: Block validation fails.

Error: Block configuration validation failed

Solutions:

  1. Validate the block: 

    polycrate block validate my-block

  2. Check block.poly for syntax errors: 

    yamllint blocks/my-block/block.poly

  3. Inspect the block configuration: 

    polycrate block inspect my-block

Block push/pull fails

Problem: The block cannot be pushed to or pulled from the registry.

Solutions:

  1. Authenticate with the registry: 

    docker login cargo.ayedo.cloud
# or for a custom registry:
docker login my.registry.com

  2. Check the registry URL (full path with version!): 

    # Full registry path required
polycrate blocks pull my.registry.com/my-org/my-block:1.0.0

  3. Check the block version: 

    # In block.poly
cat blocks/my-block/block.poly | grep version

  4. Check the network connection to the registry: 

    curl -I https://cargo.ayedo.cloud

Action problems

"Action not found"

Problem: The action does not exist or cannot be found.

Error: Action 'install' not found in block 'my-block'

Solutions:

  1. List all available actions: 

    polycrate action list

  2. Inspect the block: 

    polycrate block inspect my-block

  3. Check the block configuration: 

    cat blocks/my-block/block.poly

Action fails

Problem: The action execution fails with an error.

Solutions:

  1. Increase the log level: 

    polycrate run my-block install --loglevel 2

  2. Check the logs: 

    cat .logs/<latest-log-file>

  3. Run the action locally (for debugging): 

    polycrate run my-block install --local

  4. Inspect the action: 

    polycrate action inspect my-block install

  5. Test with a snapshot (without execution): 

    polycrate run my-block install --snapshot

Ansible errors

Problem: The Ansible playbook fails.

Solutions:

  1. Check the Ansible syntax: 

    # In the block directory
ansible-playbook --syntax-check install.yml

  2. Increase Ansible verbosity: 

    polycrate run my-block install --loglevel 2

  3. Check the inventory: 

    cat inventory.yml
ansible-inventory -i inventory.yml --list

  4. Test the Ansible connection: 

    ansible all -i inventory.yml -m ping

Workflow problems

Workflow fails

Problem: The workflow execution fails.

Solutions:

  1. List all workflows: 

    polycrate workflow list

  2. Inspect the workflow: 

    polycrate workflow inspect my-workflow

  3. Run the steps individually: 

    # Run each step individually to find the failing step
polycrate run step-1-block step-1-action

  4. Use allow_failure for optional steps: 

    workflows:
  - name: my-workflow
    steps:
      - name: optional-step
        block: my-block
        action: optional-action
        allow_failure: true

SSH problems

SSH connection fails

Problem: The SSH connection to the remote host fails.

Error: SSH connection failed

Solutions:

  1. Check the SSH keys: 

    ls -la id_rsa id_rsa.pub
chmod 600 id_rsa
chmod 644 id_rsa.pub

  2. Test the SSH connection manually: 

    ssh -i id_rsa user@host

  3. Check the inventory: 

    cat inventory.yml

  4. Use an SSH passphrase ⚠️ Experimental: 

    echo "my-passphrase" > ssh-passphrase.poly
polycrate workspace encrypt
polycrate run my-block install --ssh-use-passphrase

  5. Check authorized_keys on the remote host: 

    cat ~/.ssh/authorized_keys

Performance problems

Polycrate is slow

Problem: Polycrate execution takes very long.

Solutions:

  1. Skip the image pull (if already present): 

    polycrate run my-block install --pull false

  2. Skip the image build (if there is no custom Dockerfile.poly): 

    polycrate run my-block install --build false

  3. Run locally (if possible): 

    polycrate run my-block install --local

  4. Check Docker performance: 

    docker stats

  5. Increase Docker resources (Docker Desktop): - Settings → Resources → Increase CPU/Memory

Container takes long to build

Problem: The custom container image takes a long time to build.

Solutions:

  1. Optimize your Dockerfile.poly: 

    # Dockerfile.poly
FROM cargo.ayedo.cloud/library/polycrate:latest

# Cache-friendly layer order (Ubuntu-based!)
RUN apt-get update && apt-get install -y --no-install-recommends \
    tool1 tool2 \
    && rm -rf /var/lib/apt/lists/*

  2. Use the Docker build cache: 

    # Enable the Docker Desktop build cache

  3. Skip the build if possible: 

    polycrate run my-block install --build false

Configuration problems

Environment variables in YAML do not work

Problem: The ${VAR} syntax in workspace.poly or block.poly is not resolved.

# ❌ Does NOT work - YAML does not support env var substitution
config:
  api_key: ${API_KEY}  # stays literally "${API_KEY}"

Solution: Use secrets.poly for sensitive configuration:

# workspace.poly - public configuration
name: my-workspace

blocks:
  - name: my-app
    from: registry.my-org.com/infra/app:1.0.0
    config:
      namespace: production

# secrets.poly - sensitive configuration (store encrypted!)
blocks:
  - name: my-app
    config:
      api_key: my-secret-api-key
      database_password: db-password

# Encrypt the workspace before git push
polycrate workspace encrypt

→ Workspace encryption

Merge conflicts in configuration

Problem: Configurations are not merged correctly.

Solutions:

  1. Enable merge debugging: 

    polycrate run my-block install --merge-debug

  2. Use merge v2: 

    polycrate run my-block install --merge-v2

  3. Inspect the final snapshot: 

    polycrate run my-block install --snapshot > snapshot.yml
cat snapshot.yml

Registry problems

"Registry authentication failed"

Problem: Authentication with the registry fails.

Solutions:

  1. Login via Docker: 

    docker login cargo.ayedo.cloud
Username: your-username
Password: your-password

  2. Check the Docker config: 

    cat ~/.docker/config.json

  3. Use an API key instead of a password: 

    docker login cargo.ayedo.cloud -u your-username -p your-api-key

"Rate limit exceeded"

Problem: Too many requests to the registry.

Solutions:

  1. Authenticate (higher limits for authenticated users): 

    docker login cargo.ayedo.cloud

  2. Wait and try again later

  3. Use a mirror/cache

Common error messages

"permission denied"

  • Check file permissions: chmod 755 file
  • Check Docker permissions: sudo usermod -aG docker $USER
  • Check SSH key permissions: chmod 600 id_rsa

"no space left on device"

  • Check disk space: df -h
  • Clean up Docker: docker system prune -a
  • Clean up Polycrate logs: rm -rf .logs/*
  • Clean up artifacts: polycrate run my-block prune (if the block has a prune action)

"port already in use"

  • Check used ports: netstat -tuln | grep <port>
  • Stop conflicting containers: docker ps and docker stop <container>
  • Use a different port in the configuration

"connection timeout"

  • Check the network connection
  • Check firewall rules
  • Increase timeout values
  • Check DNS resolution: nslookup host

Debugging tips

Extended logs

# Maximum log level
polycrate run my-block install --loglevel 2

# JSON format for machine processing
polycrate run my-block install --logformat json

# YAML format
polycrate run my-block install --logformat yaml

Interactive mode

# The container stays open for interactive commands
polycrate run my-block install --interactive

Analyzing the snapshot

# Generate and analyze the snapshot
polycrate run my-block install --snapshot > snapshot.yml

# Check for specific values
cat snapshot.yml | grep "key_name"

# Filter with yq
yq eval '.blocks[].config' snapshot.yml

Debugging the container

# Open a shell in the container
docker run -it --rm \
  -v $(pwd):/workspace \
  cargo.ayedo.cloud/library/polycrate:latest \
  /bin/sh

# Inside the container
cd /workspace
ls -la

Support

If you cannot solve the problem:

  1. Check the documentation: - CLI reference - Best practices

  2. Ask the community: - Discord server - GitHub Discussions

  3. Create an issue: - GitHub Issues - Include logs, configuration and reproduction steps

See also