Starting at ~17:55 UTC, we experienced an elevated 500 rate affecting delete_by_filter, and filtered queries as a result of a recent change deployed to filtering. This change was reverted, and errors fully subsided by ~18:30 UTC. This incident affected approximately 38.2% of delete_by_filter requests in aws-us-east-1, and a small percentage of filtered queries in other regions.
Resolved
Starting at ~17:55 UTC, we experienced an elevated 500 rate affecting delete_by_filter, and filtered queries as a result of a recent change deployed to filtering. This change was reverted, and errors fully subsided by ~18:30 UTC. This incident affected approximately 38.2% of delete_by_filter requests in aws-us-east-1, and a small percentage of filtered queries in other regions.
Monitoring
Investigating elevated 500 error rate on filtered queries, delete_by_filter operations.
