Ctrlk
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Troubleshooting

Each issue below includes symptom, likely causes, diagnostic steps, resolution, verification, and information to collect before contacting Hivel support.

Services Not Starting

Symptom: One or more containers fail to start or exit immediately

Likely causes:

  • Missing Docker network

  • A dependent service (Redis, PostgreSQL) not yet healthy

  • Invalid value in .env

Diagnostic commands:

docker logs <service-name>
docker ps -a
docker network ls | grep microservices_network
./deploy.sh --status

Resolution:

  1. Create the missing network if absent: docker network create microservices_network

  2. Fix any invalid .env value that docker logs points to

  3. Re-run: ./deploy.sh --service <name>

Verify the fix:

Success: Service shows as Up and health check returns healthy

Collect for Hivel support: Full docker logs <service-name> output, docker ps -a, and .env file (with all passwords redacted)

Database Connection Issues

Symptom: Services cannot connect to PostgreSQL; psql fails with "connection refused" or authentication error

Likely causes:

  • Wrong credentials in .env

  • PostgreSQL not accepting remote connections

  • Firewall blocking port 5432

Diagnostic commands:

Resolution:

  1. Confirm pg_hba.conf allows access from the host

  2. Confirm firewall allows port 5432 from the host

  3. Verify DB_USER and DB_PASSWORD in .env match PostgreSQL

Verify the fix:

Success: psql prompt opens with no error

Collect for Hivel support: Exact psql error text, and confirmation of which credential phase (Environment Configuration) is in .env (Phase 1 = superuser, Phase 2 = application user)

Registry Pull Failed

Symptom: Docker pull from ECR fails; "authentication failed" or "invalid credentials" error

Likely causes:

  • Expired or incorrect AWS credentials

  • Credentials not re-authenticated after expiry

Diagnostic commands:

Resolution:

  1. Re-run aws configure with the credentials Hivel provided

  2. Re-authenticate to ECR using the command above

Verify the fix:

Success: Output shows Login Succeeded

Collect for Hivel support: Exact error from docker login, and output of aws sts get-caller-identity (account ID only - redact sensitive data)

./deploy.sh Not Found

Symptom: "command not found" or "no such file" when running ./deploy.sh

Likely causes:

  • Not running from the correct directory

  • Installer package never downloaded or unzipped

  • deploy.sh exists but is not executable

Diagnostic commands:

Resolution:

  1. Navigate to /opt/hivel-onprem: cd /opt/hivel-onprem

  2. If deploy.sh is missing entirely, re-download and re-extract the installer package (see Project Setup)

  3. If deploy.sh exists but is not executable, run: chmod +x deploy.sh

Verify the fix:

Success: Command reference prints with no "not found" error

Collect for Hivel support: Output of pwd and ls -la in the installer directory, and confirmation that hivel-onprem was successfully downloaded and unzipped

Migration Failed

Symptom: Flyway migration exits with non-zero code; schema not initialized

Likely causes:

  • .env contains application-user credentials instead of superuser credentials

  • insightly database does not exist

Diagnostic commands:

Resolution:

  1. Confirm superuser (not application-user) credentials are in .env for migration (see Environment Configuration Phase 1)

  2. Confirm the insightly database exists (see Database Setup)

  3. Re-run migration: ./deploy.sh --service flyway-migration

Verify the fix:

Success: Exit code is 0

Collect for Hivel support: Full docker logs flyway-migration output

Wrong Credentials After Migration

Symptom: Services fail to authenticate after migration completes; connection errors appear in service logs

Likely cause: .env was never switched from superuser to application-user credentials after migration succeeded

Diagnostic command:

Resolution:

  1. Update .env to Phase 2 credentials (see Environment Configuration)

  2. Restart services: ./deploy.sh --restart

Verify the fix:

Success: All services report healthy

Collect for Hivel support: Service logs showing the original authentication failure

Port Already in Use

Symptom: "Address already in use" or "bind failed" when starting services on port 80 or 443

Diagnostic command:

Resolution:

  1. Identify the process occupying the port (PID from lsof output)

  2. Stop the process: sudo kill -9 <PID>

  3. Redeploy: ./deploy.sh --vm

Health Checks Failing

Symptom: Services show as Up but health checks fail; services are unstable or slow to start

Likely causes:

  • Service needs more startup time

  • Host is resource-constrained

Diagnostic commands (allow 2–3 minutes on first start, then check):

Resolution:

  1. Allow 2–3 minutes for initial startup

  2. If resource-constrained, confirm host meets Server Infrastructure Setup sizing

  3. Inspect logs for specific errors: ./deploy.sh --logs <service-name>

Log rotation not working

Symptom: Log files not rotating; /var/log/hivel/ directory is missing or empty; cron job not present

Diagnostic commands:

Resolution:

Re-run the setup script:

bash

Success: Cron job appears in crontab -l output; log directory contains subdirectories per service

Collect for Hivel support: Output of sudo crontab -l and ls -la /var/log/hivel/

Need More Help?

If none of these steps resolve your issue:

  1. Collect the diagnostics listed for your issue

  2. Redact all sensitive data (passwords, tokens, private IPs)

  3. Contact support@hivel.ai with your findings

Back to File Locations Quick Reference

Backup and Restore

Last updated