: How to implement HTTPS redirection for static resources such as robots.txt in legacy ASP.NET website? We switched to HTTPS in our legacy ASP.NET site about a month ago. We had already implemented

@Barnes591

Posted in: #301Redirect #AspNet #Aspx #Https #RobotsTxt

We switched to HTTPS in our legacy ASP.NET site about a month ago. We had already implemented redirection with HTTP status code 301 'Moved permanently' for outdated aspx resources in the app's Global.asax file before switching to HTTPS, so we added a few more lines to the redirect algorithm in the Application_BeginRequest method to redirect all requests to aspx pages to their HTTPS counterparts. The core part of the current version of this algorithm looks like this ("example.com" is used instead of the real domain):

Sub Application_BeginRequest(ByVal sender As Object, ByVal e As EventArgs)
Dim host_requested As String = Request.Url.AbsoluteUri

' Save the original request string to know whether we need to redirect
Dim host_to_redirect As String = host_requested

' Prepare for analysis - remove the protocol and domain
For Each sRoot In m_asPossibleRoots
If host_to_redirect.StartsWith(sRoot, StringComparison.OrdinalIgnoreCase) Then
host_to_redirect = host_to_redirect.Substring(sRoot.Length)
Exit For
End If
Next

' Remove "home/" if any
Dim sHome As String = "home/"
If host_to_redirect.StartsWith(sHome, StringComparison.OrdinalIgnoreCase) Then
host_to_redirect = host_to_redirect.Substring(sHome.Length)
End If

' Redirect old product pages and articles
If host_to_redirect.StartsWith("outdated-resource-folder1", StringComparison.OrdinalIgnoreCase) Then
host_to_redirect = "resource1.aspx"
ElseIf host_to_redirect.StartsWith("outdated-resource-folder2", StringComparison.OrdinalIgnoreCase) Then
host_to_redirect = "resource2.aspx"
' ElseIf
' ... processing other outdated resources
End If

' ... other manipulations with host_to_redirect

' Add the required protocol and domain
host_to_redirect = "https://example.com/" + host_to_redirect

' If we need to redirect, do it
If (host_to_redirect <> host_requested) Then
' also provides a 301 HTTP status code in the response
Response.RedirectPermanent(host_to_redirect)
End If
End Sub


We also added the HTTPS version of our website to Google Webmaster Tools and similar resources working with our website. All works Ok, but recently it has turned out that one of the web-analytics tool we use determines the main protocol of our resource (HTTP or HTTPS) by sending requests to example.com/robots.txt and reading the protocol in the response. Sure, all these requests to robots.txt on our website are still processed as HTTP because we didn't (and couldn't) code redirection for this non-ASP.NET resource in Global.asax.

Having all this, the main question is the following: Do we need to implement HTTPS redirection for robots.txt if we switched to HTTPS in our ASP.NET website? May it affect SEO and similar things? We didn't bother about this as it seems this is not needed for Google & Bing WMT, Google Analytics and similar tools.

If we need to implement HTTPS redirections for robots.txt though, do we need to do this for other non-ASP.NET resources like sitemap.xml and the like? And what is the best method to implement this in our ASP.NET project? As I understand, we can do this only in web.config, but this may mean that we need to move the HTTPS redirection logic part from Global.asax to web.config too. Will this 2-stage redirection logic work seamlessly?

Vote Up Vote Down

Login to post a comment!