On April 12th, we received reports from some of our customers about end-users encountering 504 errors when using our "/send" API endpoint. As we investigated the issue, we discovered that the root cause was a Microsoft update on April 11th (i.e. what they internally call “Patch Tuesdays”) that prevented third-party API users from accessing their user folders if they had at least one public folder. This issue affected a small percentage of our accounts, estimated to be less than 2%.
Microsoft acknowledged the incident on April 13th and started rolling back the updates. However, due to the staged rollout of patches on their online servers, combined with the low usage of Public folders by their customers, the impact was not immediately apparent. We identified the issue and developed a mitigation solution by adding an optimization in our middleware component responsible for Exchange Server connectivity to prevent listing folders in scenarios that were contributing to the problem. We deployed our solution promptly, without waiting for Microsoft to complete their rollback, which took them almost nine days. You can view the Microsoft status page with the complete timeline here (Office 365 Admin account required): Some users may be unable to access, view, add, or select public folders and calendars in Outlook on the web.
We apologize for any inconvenience this may have caused and assure you that we have taken all necessary steps to resolve the issue and prevent a similar incident from occurring in the future. We will continue monitoring the situation closely and take all necessary precautions to ensure our customers' services are not disrupted.