A lot of exceptions in Azure App Insights

paulvader

Vote:

Hi everyone, we have an Epi site with CMS 11.15.0 and Commerce 13.18.2, hosted in an Azure appservice. It is registering a lot of exceptions in Azure Insights, and we just don't know where are those coming from. The ones with most occurrences and with no easy explanation are:

1. CheckVirtualFileExists

This one happens on every .aspx request that does not exist. We use the 404 Geta handler, and it redirects to our "not found page" correctly, but it logs a lot of these exceptions in azure.

System.Web.HttpException:
   at System.Web.UI.Util.CheckVirtualFileExists (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.Compilation.BuildManager.GetVPathBuildResultInternal (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.Compilation.BuildManager.GetVPathBuildResultWithNoAssert (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.Compilation.BuildManager.GetVirtualPathObjectFactory (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.Compilation.BuildManager.CreateInstanceFromVirtualPath (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.UI.PageHandlerFactory.GetHandlerHelper (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.UI.PageHandlerFactory.GetHandler (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+MaterializeHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+<>c__DisplayClass285_0.<ExecuteStepImpl>b__0 (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+StepInvoker.Invoke (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+StepInvoker+<>c__DisplayClass4_0.<Invoke>b__0 (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at Microsoft.AspNet.TelemetryCorrelation.TelemetryCorrelationHttpModule.OnExecuteRequestStep (Microsoft.AspNet.TelemetryCorrelation, Version=1.0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.HttpApplication+<>c__DisplayClass284_0.<OnExecuteRequestStep>b__0 (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+StepInvoker.Invoke (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication.ExecuteStepImpl (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication.ExecuteStep (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)

From the IIS logs on azure:

2020-07-20 22:23:38 AZAPP-NXS-EPI-CMS-PREP-01__E284 GET /search.aspx 404;https://azapp-nxs-epi-cms-prep-01__e284:80/search.aspx?page=11&category=953&X-ARR-LOG-ID=77c41081-2434-4707-a9e4-966b3c94b9c3 443

Testing the same url, but in my local environment (note no 404):

2020-07-20 23:13:43 127.0.0.1 GET /search.aspx page=11&category=953 80 - 127.0.0.1 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/83.0.4103.116+Safari/537.36 - 200 0 0 1141

2. Server cannot append header after HTTP headers have been sent.

System.Web.HttpException:
   at System.Web.HttpResponse.AppendHeader (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpResponseWrapper.AppendHeader (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at EPiServer.Web.MediaHandlerBase.CheckIsModifiedAndAddETag (EPiServer.Framework.AspNet, Version=11.15.0.0, Culture=neutral, PublicKeyToken=8fe83dea738b45b7)
   at EPiServer.Web.MediaHandlerBase.NotModifiedHandling (EPiServer.Framework.AspNet, Version=11.15.0.0, Culture=neutral, PublicKeyToken=8fe83dea738b45b7)
   at EPiServer.Web.BlobHttpHandler.ProccessBlobRequest (EPiServer.Cms.AspNet, Version=11.15.0.0, Culture=neutral, PublicKeyToken=8fe83dea738b45b7)
   at EPiServer.Web.BlobHttpHandler.ProcessRequestAsyncInternal (EPiServer.Cms.AspNet, Version=11.15.0.0, Culture=neutral, PublicKeyToken=8fe83dea738b45b7)
   at EPiServer.Web.BlobHttpHandler.System.Web.IHttpAsyncHandler.BeginProcessRequest (EPiServer.Cms.AspNet, Version=11.15.0.0, Culture=neutral, PublicKeyToken=8fe83dea738b45b7)
   at System.Web.HttpApplication+CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+<>c__DisplayClass285_0.<ExecuteStepImpl>b__0 (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+StepInvoker.Invoke (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+StepInvoker+<>c__DisplayClass4_0.<Invoke>b__0 (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at Microsoft.AspNet.TelemetryCorrelation.TelemetryCorrelationHttpModule.OnExecuteRequestStep (Microsoft.AspNet.TelemetryCorrelation, Version=1.0.5.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35)
   at System.Web.HttpApplication+<>c__DisplayClass284_0.<OnExecuteRequestStep>b__0 (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication+StepInvoker.Invoke (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication.ExecuteStepImpl (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
   at System.Web.HttpApplication.ExecuteStep (System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)

This one is the most random exception. Happens a lot when trying to access to images. I had a chat with Epi Support, they asked me to review my redirects, which I did and error is still happening. This can't be reproduced in my local environment either.

These two exceptions happen a lot during the day, this is just a report from the past 6 hours:

Hope any of you have seen this before and have some insight on to what might be happening. We use azure deployment slots to decrease downtime, not sure if that can be an issue.

Thanks!

#225583

Jul 20, 2020 23:36

Quan Mai

Vote:

For the first issue, it looks like your Azure web.config is missing handler for aspx? Something like

#225589

Jul 21, 2020 7:53

Stefan Holm Olsen

Vote:

Hi Paul

Regarding issue 2. Are you by any chance using the ImageProcessor library in your solution?

I ask because that library tend to raise that exception if the site definition are configured wrong.

#225591

Jul 21, 2020 9:03

paulvader

Vote:

Hi Stefan, thanks for your reply. No, we don't use the ImageProcessor. What's weird is that we had a release 3 weeks ago, the issue stopped, then we had a release last week, the problem started happening again. It is like something gets initialized in the wrong order or something, but we still have no clue on what is going on. There was nothing major on both releases, nothing related to media handling or redirects (which are the likely cause of the problem).

#225606

Jul 21, 2020 14:52

paulvader

Vote:

Hi Quan, that was really helpful. I added the below handler to the staging environment and it is not reporting such exceptions now. Will try in production later.

<add name="AspxHandler" preCondition="integratedMode" verb="GET" path="*.aspx" type="EPiServer.Web.StaticFileHandler, EPiServer.Framework.AspNet" />

Thanks, will wait a little bit before accepting the answer just to see if someone has something about my second problem.

#225607

Jul 21, 2020 15:15

Quan Mai - Jul 21, 2020 15:21

Glad it helped. Btw you can accept multiple answers - not saying you have to accept it now :)

Vote:

In terms of "HTTP headers have been sent" error: This is usually caused when the application is performing a redirect while having the headers already sent.

Do you have any code that sets headers? E-tag or modified for images?

#226151

Aug 05, 2020 18:21

paulvader - Aug 06, 2020 16:01

Hi Mari, thanks for your reply. I've checked all our redirects and headers. They look ok. Also, this only happens in production so it has been very hard to solve. And what is most weird is that it stopped happening on a deploy, then started again on the next one. I am thinking maybe something is being initialized in the wrong order or something.

paulvader

Vote:

Hi all, seems like upgrading Epi can help with the "Headers have been sent" issue. More details here

https://world.episerver.com/documentation/Release-Notes/ReleaseNote/?releaseNoteId=CMS-16110

Cheers!

#226422

Aug 12, 2020 15:36

Try our conversational search powered by Generative AI!

A lot of exceptions in Azure App Insights