Users unable to login to service
Incident Report for NetDocuments UK
Postmortem

On 14 April at approximately 7:48 am BST, NetDocuments experienced a disruption with the UK Service.   During the event customers would have been unable to authenticate to the Service or unable to reach the login page altogether.

We were alerted to the start of the issue via our automated monitoring infrastructure and were able to begin investigating immediately.  By approximately 10:20 am BST the Service had been stabilized and operations had returned to normal.

An investigation found the root cause was related to a software defect in the recently deployed 23.7 version of the Service affecting the retrieval of cached information.   Once the cause was identified, steps were taken to roll the Service back to the previous version of 23.6 in order to stabilize the Service and allow for the defect to be corrected before the new version is deployed again.

As a part of ongoing and continuous improvement, NetDocuments is evaluating and adding additional test cases to enhance the Beta and Deployment processes to prevent a similar defect from being able to affect the Service in this way again.

 

We apologize for any inconvenience this disruption may have caused.

 

Thank you,

The NetDocuments Team

Posted Apr 18, 2023 - 00:20 BST

Resolved
This incident is now resolved. We will conduct a postmortem of this event, which will be posted within 2 business days.

If you continue to experience these issues, please open a ticket with our Support team for further assistance.
Log into support.netdocuments.com to submit a support request for your issue.
However, if you need to urgently contact Technical Support via the phone use the following information:
US customers call 801-226-6882 or 866-NETDOCS (638-3627)
EU customers call +44(0) 2034.556770
Posted Apr 14, 2023 - 10:58 BST
Monitoring
Our engineering team has implemented a fix to resolve the issue with logging into the Netdocuments service and are monitoring. If you continue to experience these issues, please open a ticket with our Support team for further assistance. We will conduct a postmortem of this event, which will be posted within 2 business days.
We prefer that you go to support.netdocuments.com to submit a support request for your issue, so we can manage it more efficiently and provide you with access to your cases.
However, if you need to contact Technical Support via the phone use the following information:
US customers call 801-226-6882 or 866-NETDOCS (638-3627)
EU customers call +44(0) 2034.556770
Posted Apr 14, 2023 - 10:00 BST
Update
We are continuing to work on a fix for this issue. We will provide a further update in 15 mins.
Posted Apr 14, 2023 - 09:37 BST
Identified
Our engineering team has identified the issue causing users to be unable to login to the service. We are currently doing work to help mitigate the issue. We will provide a further update in 15 mins.
Posted Apr 14, 2023 - 09:20 BST
Update
This issue is preventing users from connecting to the service. We are actively working to mitigate this issue and will provide a further update in 15 minutes.
Posted Apr 14, 2023 - 09:01 BST
Investigating
Our engineering team has identified an issue which may cause degraded performance. We are currently doing maintenance to help mitigate the issue.
Posted Apr 14, 2023 - 08:17 BST
This incident affected: Platform.