# Troubleshooting
# Basics
At boot Cycloid worker run startup.sh (opens new window) script . This script installs the worker using Ansible.
If your worker does not appear in the list of Cycloid workers, connect via SSH to the worker and refer to the following sections.
To connect using SSH on a worker, the SSH user depend on Cloud provider base images. By default we use Debian images and usual users are:
- Azure: user cycloid
- AWS: user admin
- GCP: gcloud command
# First aid
Step 1) Check the boot script status
The startup.sh
script is run at boot using cloud init. The setup logs can be found in either of those 2 files:
/var/log/cloud-init-output.log
/var/log/user-data.log
2
Check if there are any errors, If your setup is successful you should see Ansible summary log line without any failed
statement
Step 2) Check service status and logs
Run the following command
systemctl status concourse-worker
The service status should be in active/running state
Then look at the worker logs file here /var/log/concourse-worker.log
or running journalctl -eu concourse-worker
For a successful log line such as
{"timestamp":"2022-04-08T05:55:25.988837053Z","level":"info","source":"worker","message":"worker.beacon-runner.beacon.rebalance.registered","data":{"session":"4.1.58143"}}
However if you have one of the following common errors, refer to the step 3) to validate your configuration:
# Common solution: Scheduler host or port incorrect
{"timestamp":"...","level":"error","source":"worker","message":"worker.beacon-runner.beacon.dial.failed-to-connect-to-any-tsa","data":{"error":"all worker SSH gateways unreachable","session":"4.1.1"}}
{"timestamp":"...","level":"error","source":"worker","message":"worker.beacon-runner.beacon.exited-with-error","data":{"error":"all worker SSH gateways unreachable","session":"4.1"}}
# Common solution: Team id, TSA public key or worker key incorrect
{"timestamp":"...","level":"error","source":"worker","message":"worker.beacon-runner.failed","data":{"error":"failed to establish SSH connection with gateway: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain","session":"4"}}
2
3
4
5
6
Step 3) Ensure the configuration is correct
By editing the generated /var/lib/concourse/concourse-worker
file you can validate with your system administrator if the highlighted lines are correct:
#!/bin/bash
...
exec /var/lib/concourse/bin/concourse worker \
--tsa-host "scheduler.cycloid.io:32223" \
--tsa-public-key /var/lib/concourse/host_key.pub \
--tsa-worker-private-key /var/lib/concourse/worker_key \
--name $(hostname) \
--work-dir /var/lib/concourse/datas \
\
--team 9620fd79-7c8e-414e-9072-8e1604375b98 \
...
2
3
4
5
6
7
8
9
10
11
tsa-host
refer to Cycloid scheduler host and port.tsa-public-key
refer to Cycloid scheduler public TSA keytsa-worker-private-key
refer to Cycloid worker keyteam
refer to Cycloid CI team name
TIP
You can retrieve Cycloid CI team name on the console:
- Click on your top right profile picture, and select
Organization settings
- You should find the CI team in the
General
tab,metadata
section
# known errors
If you have encountered an error message, you can also refer to the following section known errors, which compiles known errors.
# Get help from Cycloid
If your worker is not running correctly, you can create a system report of your worker and share the encrypted result with Cycloid.
The following command will create a Cycloid worker system report and provide a link to share with the Cycloid team:
curl -sSL https://raw.githubusercontent.com/cycloidio/cycloid-utils/master/report/worker_report.sh | sudo bash
The report contains information like running process, services and network configuration, logs, etc. You can find more details on Github (opens new window).