I came across the following post which addresses the same issue i am experiencing. I am currently on EPiServer.CMS.Core 8.11.0 but seems like this issue is not fixed or broken again.
I have added the appSetting as suggested in my web.config and also configured Azure portal by adding a "EPiServerDB" setting but its still picking up the connection string from my web.config file.
Is there anything else i need to configure?
Do you have a separate connectionStrings.config file referred to from your web.config? I recall that could cause some issues.