2024-07 Technology Blog Posts
Who is this article for?
Admins, Module Developers and advanced Users configuring and testing new projects or changes.
A module developer license and area access is required for testing.
For current Technology Updates, sign up to the Community and Subscribe to Technology Updates
No-downtime US Production Update (2.59.2) Thursday, July 18, 2024, and Maintenance
(July 12, 2024)
Platform Update
We are planning a no-downtime update of US Production on the evening of Thursday, July 18. You can review these release notes for an overview of the changes. This release will move to the European and Canadian Production areas on Wednesday, July 31.
Action
Please report any unexpected behavior that you notice in your US Config (Build, Test, and UAT) areas, which we plan to update with this new 2.59.2 version on Friday, July 12. (We will postpone the Production update if we're not confident in its quality.) This release will reach the European and Canadian Config areas on Wednesday, July 24.
Maintenance
- No-downtime software maintenance on US Production servers: between Tuesday, July 16, and Thursday, July 18
Issue affecting US Production DevonWay applications July 19, 2024
(July 19, 2024)
We are aware of an issue currently affecting our US hosted data centers, which is causing an unplanned downtime across all applications and services running in these data centers, including TrakWay.
We are working to understand the issue and will restore service as soon as possible. We'll provide updates as we have more information.
Restored: Issue affecting US Production DevonWay applications July 19, 2024
(July 19, 2024)
Applications and services hosted in our US Production data centers are running again, as of 2:16 am CDT. The initial evidence suggests that this unplanned downtime was likely related to the flawed CrowdStrike update affecting many data centers and companies, as described in the news, though we haven't finished our investigation.
Note that we are still working to validate and bring online some redundant resources within our data centers and that a small subset of batch jobs are still not running but we are working to recover them.
We'll continue to monitor the state of our data centers and to investigate the cause, length, and impact of this unplanned downtime. We'll provide more details as we have them.
-- Carl
Update: Issue affecting US Production DevonWay applications July 19, 2024
(July 20, 2024)
As an update on the issue we experienced yesterday, we have confirmed this was caused by complications related to the CrowdStrike flawed update. A subset of hosts and data center services were affected by these updates and, due to the widespread nature of the impacts, took longer than expected to remediate.
As of 5:25 am CDT Friday morning (July 19), the remaining batch jobs were running as expected, and we have validated and brought online all redundant resources as planned.
We will continue to evaluate the event and will incorporate all possible lessons learned or improvements within our control. As always, if you have any questions or concerns, please don't hesitate to contact us.
-- Robert