ScootKit - Issues with bots hosted on Server 17, 20 and 24 – Incident details

Executive Summary

On January 10, 2026, SCNX experienced a major outage affecting Bot-Hosts #17, #20, and #24. The incident resulted in connectivity failures, preventing hosted bots from connecting to Discord and internal SCNX infrastructure. To resolve the issue, our engineering team provisioned a new server (Bot-Host #38) and migrated all affected workloads. The incident was marked fully resolved on January 11, 2026, following the successful migration of residual data.

Root Cause Analysis

The disruption was caused by a networking issue originating from the hosting provider for the affected servers. This network failure severed the connection between the bot hosts, Discord’s API, and other parts of the SCNX ecosystem. Due to the severity and nature of the networking problems, it remains unclear if operations can resume on the affected hardware in the future.

Impact

• Affected Services: Bots hosted on SCNX Bot-Hosts #17, #20, and #24.

• User Experience:

  • Bots were unable to connect to Discord, resulting in downtime.

  • In some cases, bots appeared online but suffered from slow response times or failed to load the dashboard.

• Downtime: The total downtime for individual bots was limited to less than 5 hours, depending on when the specific bot was automatically migrated to the new host.

Timeline of Events

All times are in the time zone reported in the system logs.

January 10, 2026

• 8:35 PM: The team begins investigating reports of bots on Servers 17, 20, and 24 being unable to connect to Discord.

• 9:58 PM: Bot-Host #38 is provisioned as a temporary host to ensure service quality. Automatic migration of affected bots begins.

• 10:58 PM: Migration for all bots from Server 24 is completed.

• 11:06 PM: Migration for all bots from Server 20 is completed.

• 11:20 PM: All active bots are successfully moved to Bot-Host #38 and confirmed to be running without issues.

January 11, 2026

• 4:09 PM: Work begins on discontinuing the affected hosts while a permanent solution is negotiated with the provider.

• 6:50 PM: Residual data from Bot-Host #17 is migrated, and the decommissioning process begins.

• 10:24 PM: Residual data from Bot-Hosts #20 and #24 is migrated, and the decommissioning process begins.

• 10:26 PM: Incident is marked as Resolved.

Resolution and Next Steps

Service was restored by migrating all active bots to a new node, Bot-Host #38. Following the restoration of service, all remaining data was migrated off the affected infrastructure.

To prevent recurrence:

1. Infrastructure Decommissioning: Bot-Hosts #17, #20, and #24 are currently being decommissioned as we assess the situation.

2. Provider Review: We are in active discussions with the hosting provider to assess their continued suitability for our operations.

3. Mitigation Improvement: We are reviewing our internal processes to ensure future connectivity issues can be mitigated even more effectively.

We apologize for the inconvenience caused and appreciate your patience during this migration.

Since all data has been migrated, we're marking this incident resolved. The total downtime for bots was less than 5 hours, depending on when they were moved automatically.

We apologize for the inconvenience caused and are working on ensuring that such issues can be mitigated more effectively in the future.

For the affected bot hosts 20, 24 and 17, we're talking to the hosting provider to assess their continued suitability for our operations.

Remaining data from Bot-Host #20 and Bot-Host #24 have been migrated, and we're in the process of decommissioning the Bot-Hosts for now. It remains unclear whether we will be able to resume operations on that bot host in the future - for now, all bots previously hosted on Bot-Host #20 and Bot-Host #24 has been migrated to Bot-Host #38.

Remaining data from Bot-Host #17 has been migrated, and we're in the process of decommissioning the Bot-Host for now. It remains unclear whether we will be able to resume operations on that bot host in the future - for now, all bots previously hosted on Bot-Host #17 has been migrated to Bot-Host #38.

We're currently working on discontinuing the affected Bot-Hosts for now, while we work with the provider to either find a solution for the networking problem or deactivate the servers for now. If your bot is experiencing issues, please contact our Customer Support, we will help you get it running as soon as possible.

We have moved all bots to the new Bot-Host #38, where they should be running without issues. We're working on restoring full access.

All bots from Server 20 have been moved to our new Bot-Host and are now running without issues.

All bots from Server 24 have been moved to our new Bot-Host and are now running without issues.

We've added Bot-Host 38 as a temporary Bot-Host to move bots to to ensure hosting quality. We're working on moving affected bots automatically, but if you want your bot moved faster, please ping @scderox on our Discord. Likewise, we apologize for the issues and are working hard to get them fixed.