Recovering from Exchange 2013 Website 404 Errors (ECP & OWA)

Last week we were commissioned by a client to implement a new Windows 2012 server, build Active Directory, and setup Exchange 2013.  It was a one pre-existing server solution with a small number of mailboxes.  A standard setup with Exchange configured for mailboxes, client access (OWA), Autodiscover, and Active Sync.

Everything went smoothly, certificates were installed, OWA, ECP, ActiveSync, and Outlook were all functioning for the client.  We put forward a few post implementation suggestions surrounding firewall configuration and backups, and then closed the project.

A little while later the client contacts us as their mail has stopped flowing, they can’t access the Exchange Admin Centre site or OWA, and Outlook is unable to connect to Exchange.  On the server the Exchange services all exist and are running, but they had to do some emergency maintenance on their server and now everything appears to be non-functional.

These are the troubleshooting and recovery steps we went through:

Trying to access either https://domain/ecp or https://domain/owa throws a 404 error in the browser.

 

Looking inside IIS, the files for these virtual directories exist and don’t appear to have been manipulated.

So, first things first (after reviewing some logs, confirming the services exist, etc.), we decide to try a simple “Programs and Features / Change” option on the Exchange installation to see if we can do a quick repair.  We receive an error that the installation cannot be run from here, and that the media is required.

 

Next, we mount the media and try to re-run the installation.  Very strange behavior here in that we can get to the screen displaying the roles already installed, but the ‘Next’ button, although not grayed out, does not appear to be functional as no action takes place when we click it.

 

Next, from an elevated command line we attempt the standard Exchange recovery options suggested on many sites regarding similar behavior in Exchange:

setup /m:RecoverServer /IAcceptExchangeServerLicenseTerms

 

This fails part way though the recovery, stating that we already have Exchange roles installed…

 

It would appear that somewhere along the line, Exchange has been left in a state where the OS or Exchange Installation checks both believe the roles are up and functioning.  The services are all installed, running, and not throwing any obvious errors, but we have no access to manage Exchange, work with mailboxes, and mail is not flowing.

Finally we decide to try an in-place upgrade of Exchange, even though we are using the same version that was originally installed:

setup /m:Upgrade /IAcceptExchangeServerLicenseTerms

**Note – The DNS error here is because the client has the DC/Exchange server setup with a public IP address and the public zone is different than the internal AD zone being used (IP has been removed).

 

This takes some time, but does end up completing successfully.  After a reboot, functionality is restored and the mail starts flowing.

The only “post-upgrade” issue we had to address was to reassign the public certificate back to the Exchange services in the Exchange Admin Centre.

 

Hopefully this article can save someone stress in the future.