ScootKit - Issues with the @SCNX-Bot – Incident details

All systems operational

Issues with the @SCNX-Bot

Resolved
Major outage
Started 10 months agoLasted about 1 hour

Affected

SCNX

Major outage from 9:08 AM to 10:20 AM

@SCNX Bot

Major outage from 9:08 AM to 10:20 AM

Updates
  • Resolved
    UTC
    Resolved

    We are pleased to confirm that the incident affecting the @SCNX-Bot is now fully resolved, and all services have returned to normal operation as of 12:20 PM CEST.

    Earlier this morning, our monitoring systems detected a disruption that was impacting some of the backend features of the @SCNX-Bot. Our engineering team immediately investigated and traced the root cause to a performance degradation in a database cluster dedicated to the bot's administrative functions. This resulted in the temporary unavailability of tools such as backups and analytics.

    We want to emphasize that the impact was strictly limited to these supplementary administrative features. All critical, core functionalities, including the operation of all custom bots and customer-facing hosted experiences, were completely unaffected and remained 100% operational throughout this period.

    Our team was able to resolve the database bottleneck by deploying an optimization, which allowed services to recover quickly without broader impact. To prevent a recurrence, we are already implementing more sensitive alerting on our database metrics and are conducting a full review to further strengthen our failover procedures.

    We sincerely apologize for any inconvenience the temporary loss of these administrative tools may have caused. While we have confirmed a full recovery, please do not hesitate to contact our customer support team if you continue to experience any persistent issues. We appreciate your patience and understanding.

  • Monitoring
    UTC
    Monitoring

    We have implemented a fix for the @SCNX-Bot ahead of schedule. Our team was able to resolve the underlying problem without the need for the planned infrastructure migration.

    All services for the @SCNX-Bot, including backups and analytics, have been restored and should now be operating normally.

    Our team will continue to monitor the system closely to ensure the stability of the fix. We will post a final "Resolved" message once we are confident the issue is fully behind us. If you continue to experience any issues, please contact our support team.

  • Identified
    UTC
    Identified

    We have identified the root cause of the issue affecting the @SCNX-Bot.

    To restore full functionality, we are currently performing emergency maintenance which involves migrating the bot's infrastructure. This is a necessary step to ensure service stability.

    During this maintenance period, features like backups and analytics for the @SCNX-Bot will remain unavailable. Custom bots and hosted experiences continue to operate normally.

    We appreciate your patience as our team works to complete this migration. Our next update will be posted by 12:30 PM CEST.

  • Investigating
    UTC
    Investigating

    We are aware of an ongoing issue preventing access to several features of the @SCNX-Bot, including backups and analytics. We understand this may disrupt your workflow and we apologize for the inconvenience.

    Please be assured that all custom bots and hosted experiences are functioning normally and are not impacted.