# Troubleshooting

# Basics

At boot Cycloid worker run startup.sh (opens new window) script . This script installs the worker using Ansible.

If your worker does not appear in the list of Cycloid workers, connect via SSH to the worker and refer to the following sections.

To connect using SSH on a worker, the SSH user depend on Cloud provider base images. By default we use Debian images and usual users are:

  • Azure: user cycloid
  • AWS: user admin
  • GCP: gcloud command

# First aid

Step 1) Check the boot script status

The startup.sh script is run at boot using cloud init. The setup logs can be found in either of those 2 files:

/var/log/cloud-init-output.log
/var/log/user-data.log
1
2

Check if there are any errors, If your setup is successful you should see Ansible summary log line without any failed statement

worker-ansible-setup

Step 2) Check service status and logs

Run the following command

systemctl status concourse-worker
1

The service status should be in active/running state

worker-systemd-status

Then look at the worker logs file here /var/log/concourse-worker.log or running journalctl -eu concourse-worker

For a successful log line such as

{"timestamp":"2022-04-08T05:55:25.988837053Z","level":"info","source":"worker","message":"worker.beacon-runner.beacon.rebalance.registered","data":{"session":"4.1.58143"}}
1

However if you have one of the following common errors, refer to the step 3) to validate your configuration:

# Common solution: Scheduler host or port incorrect
{"timestamp":"...","level":"error","source":"worker","message":"worker.beacon-runner.beacon.dial.failed-to-connect-to-any-tsa","data":{"error":"all worker SSH gateways unreachable","session":"4.1.1"}}
{"timestamp":"...","level":"error","source":"worker","message":"worker.beacon-runner.beacon.exited-with-error","data":{"error":"all worker SSH gateways unreachable","session":"4.1"}}

# Common solution: Team id, TSA public key or worker key incorrect
{"timestamp":"...","level":"error","source":"worker","message":"worker.beacon-runner.failed","data":{"error":"failed to establish SSH connection with gateway: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain","session":"4"}}
1
2
3
4
5
6

Step 3) Ensure the configuration is correct

By editing the generated /var/lib/concourse/concourse-worker file you can validate with your system administrator if the highlighted lines are correct:




 
 
 



 


#!/bin/bash
...
exec /var/lib/concourse/bin/concourse worker \
  --tsa-host "scheduler.cycloid.io:32223" \
  --tsa-public-key /var/lib/concourse/host_key.pub \
  --tsa-worker-private-key /var/lib/concourse/worker_key \
  --name $(hostname) \
  --work-dir /var/lib/concourse/datas \
   \
  --team 9620fd79-7c8e-414e-9072-8e1604375b98 \
...
1
2
3
4
5
6
7
8
9
10
11
  • tsa-host refer to Cycloid scheduler host and port.
  • tsa-public-key refer to Cycloid scheduler public TSA key
  • tsa-worker-private-key refer to Cycloid worker key
  • team refer to Cycloid CI team name

TIP

You can retrieve Cycloid CI team name on the console:

  1. Click on your top right profile picture, and select Organization settings

Manage org

  1. You should find the CI team in the General tab, metadata section

# known errors

If you have encountered an error message, you can also refer to the following section known errors, which compiles known errors.

# Get help from Cycloid

If your worker is not running correctly, you can create a system report of your worker and share the encrypted result with Cycloid.

The following command will create a Cycloid worker system report and provide a link to share with the Cycloid team:

curl -sSL https://raw.githubusercontent.com/cycloidio/cycloid-utils/master/report/worker_report.sh | sudo bash
1

The report contains information like running process, services and network configuration, logs, etc. You can find more details on Github (opens new window).