DXC - Different physical files for multiple sites using same sour

DXC - Different physical files for multiple sites using same source code - robots.txt

Sandeep Sahoo

Vote:

We have a website we are building which is deployed to the DXC and has multiple sites using different domains -

www.abc.com

www.def.com

...

and so on

All the sites use the same source code and we have created various start/home pages in the cms, and each site points to its own home page.

All seems to be working fine. We have however come accross an issue of actual phyiscal files in the code which need to be different.

We have to add the robot.txt file to the project but this needs to be unique to each site/domain we have created.

In a normal scenario, as this is a phyiscal file in the root folder it is quite simplet to add.

However in this case where we need to keep multiple versions it is not possible as we have one only common code base.

Could someone please advice the quickest way forward to add this at each domain level.

Kind Regards

Sandeep

#194174

Edited, Jun 15, 2018 1:10

Murtaza Ali

Vote:

Hi Sandeep,

You can add rewrite rules

  <!-- Rewrite robots file requests for sites -->
        <rule name="Robots" enabled="true" stopProcessing="true" patternSyntax="Wildcard" xdt:Transform="Insert">
          <match url="robots.txt" />
          <conditions>
            <add input="{HTTP_HOST}" pattern="www.*.*" ignoreCase="true" />
          </conditions>
          <action type="Rewrite" url="/Resources/Robots/robots.{C:1}.txt" />
        </rule>

You will have

/resources/Robots/robots.abc.txt

/resources/Robots/robots.def.txt

so if user comes to abc.com/robots.txt it will be redirected to abc.com/resources/robots/robots.abc.com

this way it will resolve your physical file issue.

Cheers

Ali, Murtaza

#194176

Jun 15, 2018 6:34

Darren Sunderland

Vote:

For a robots.txt file there is an option to use the plug-in POSSIBLE.RobotsTxtHandler to allow this to be configured dynamically for each site in your solution, rather than using static files. This will present the site administrators with access to a text input field to manage the content of the file.

If the administrators of your sites have requirements beyond a simple text entry field then you would need to develop a bespoke handler, but in most circumstances the POSSIBLE.RobotsTxtHandler plug-in will suffice.

#194183

Edited, Jun 15, 2018 10:12

Sandeep Sahoo

Vote:

Hi Murtaza Thanks for the reply.  Will the crawlers be able to read the file if there is a redirect in place. I am not sure. Also we need to place some other config and feed files in the root which are read by third party apps. We have seen issues with automated programs reading the file if there are redirects in place. Is there any other option ? We can actually write code to dynamically show output via a controler but would want to avoid that as it would create a lot of work for something very simple. Kind Regards Sandeep

#194184

Jun 15, 2018 10:16

Johan Petersson

Vote:

You don't need a physical file at all. You can ccreate a controller and map it to whatever URL you want. Then you can put logic inside the controller/action to render different contentent depending on the current domain/website.

#194194

Jun 15, 2018 13:03

Sandeep Sahoo

Vote:

Thanks Johan I had just mentioned the same in my comment above. I was hoping for some asy cheeat which would save some dev time. Kind Regards Sandeep

#194218

Jun 15, 2018 22:16

David Knipe

Vote:

Hey Sandeep

By the sounds of it POSSIBLE.RobotsTxtHandler will do exactly what you need (as Darren has described above). Just install it and go, no dev time needed apart from the install itself.

https://nuget.episerver.com/package/?id=POSSIBLE.RobotsTxtHandler

David

#194244

Jun 17, 2018 21:01

Try our conversational search powered by Generative AI!

DXC - Different physical files for multiple sites using same source code - robots.txt