Dear Customers,
On May 20, 2025, between 9:20 AM and 12:34 PM, we experienced a significant disruption to our systems. During this period, our services were, in many cases, completely unavailable, marking the most severe incident at Stackfield in over six years.
What happened?
The root cause was related to our databases. To optimize data access, databases use what are known as indexes, which must be created and maintained manually. These indexes enable queries to be executed quickly and efficiently. However, in rare cases, particularly with very large volumes of data anomalies can occur.
In this case, incorrect interpretations of execution plans, combined with insufficient indexing, led to query processing times increasing from milliseconds to several seconds. This performance degradation triggered a cascading effect that ultimately overwhelmed our systems.
Despite having redundant systems in place to ensure availability, the problem propagated across all systems due to mirrored setups and identical configurations on all instances.
After determining the root cause, we were able to promptly fix the issue.
We want to emphasize that at no point was the integrity or security of your data at risk. All data remains fully intact and accessible.
Our next steps:
We are currently optimizing our monitoring systems to detect performance anomalies of this kind earlier and more accurately. This enhanced monitoring process will be fully implemented within this week.
While such incidents are extremely rare, we take them very seriously. We sincerely apologize for the inconvenience this has caused.
Thank you for your understanding and continued trust.
Cristian Mudure
CEO – Stackfield
PS: You can check the current status of our systems at any time here:
https://status.stackfield.com