Are you struggling with SharePoint? Have the search and workflows been giving you a headache? Well, you're not alone, and there are a few things you can try to remedy the situation. Keep reading for a first-hand account of an issue one of our engineers helped a client through. We hope that by sharing our troubleshooting experience, we can help keep your systems running.
Recently we had a client contact us because their search and workflows had stopped working in their SharePoint Server 2013 Enterprise deployment. We had helped resolve a similar issue about a year before where workflows had stopped running due to an expired SSL certificate. After adjusting the certificate information to use a valid certificate and restarting the workflows, it was resolved. The Windows logs, in this case, indicated a certificate and tokenization problem. So, as we suspected, this was another certificate error.
The First Attempt to Fix
We attempted to restart the farm and received an error related to missing certificates. A review of the certificates showed they expired in 2018. This meant we could not get a standard cert that would span the gap of time needed to “fool” the application and replace the expired cert.
We would have to use an open SSL certificate. The other issue we ran into was their farm was way behind in patches. We took the following steps, which typically resolve the issue:
- Removed Service Bus Manager
- Removed Workflow Service Manager
- Installed and Configured New Service Bus Farm
- Added New SBHost using previously configured DB string
- Restored Workflow Farm using the “New” Farm Cmdlets but retaining the workflow instances and resources DBs to keep the existing workflows
- Registered Service URI
- Added Certs to the trusted CA store
- Added Trusted CA certs to the SharePoint Store in CA to create a trust relationship as well
- Configured and checked namespace settings for both Service Bus and Workflow Manager certutil.exe -generateSSTFromWU roots.sst
- Confirmed service URI and attempted an activities update
- Ran Update=WFHost
However, throughout the process, we ran into several errors. At this point, we recommended the client open a ticket with Microsoft as all our troubleshooting could not bring this to a successful fix. After multiple attempts and working with Microsoft, the workflow manager and workflows were removed, reinstalled, and new certs applied. Unfortunately, even then, the workflows would still throw an error.
What ultimately fixed the issue?
There is a folder on the file system that holds certificate-type information. The farm account appears to have lost access to read and write to this folder:
"<Windows drive>\ProgramData\Microsoft\Crypto\RSA\MachineKeys"
Which is a form of cert caching. Once the client granted permissions to the folder, workflows, and search, it began working correctly.
We’re not really sure how the farm account lost permissions to the folder, but it’s certainly something to check if you are dealing with expired certificate issues.
Be sure to check out some of our other Microsoft blogs for more tips and tricks!
Need Help?
At Ascend, we have experts in Microsoft, Cybersecurity, Cloud Infrastructure, and more to help organizations like yours run smoothly. Have questions? Need help building a scalable platform? Reach out to one of our experts to start the conversation.
