In the evening of 8/1/2023, a network configuration change was made to our Azure Network Infrastructure.  That network change had a side effect of causing new web services to become disconnected from other required resources.

XVWeb web services are built to scale under load, with new instances created automatically in response to demand. Since new services were not needed overnight, problems only began shortly before 5:00am PDT on 8/2/2023.  As XVWeb demand increased, the scale-out failed as new instances of the web service were unable to start.  This left the base set of web services to handle the increasing load which caused images to be slow in loading. 

The incorrect network configuration was identified and corrected shortly after 6:15am PDT.  Once that was corrected, XVWeb scaling succeeded and performance returned to normal levels.