: No. The RFC says: The format logically consists of a non-empty set or records, separated by blank lines. The records consist of a set of lines of the form: <Field> ":" <value>

No. The RFC says:

The format logically consists of a non-empty set or records,
separated by blank lines. The records consist of a set of lines of
the form:

<Field> ":" <value>

The formal syntax includes:

agentline = "User-agent:" *space agent [comment] CRLF
disallowline = "Disallow" ":" *space path [comment] CRLF
allowline = "Allow" ":" *space rpath [comment] CRLF
extension = token : *space value [comment] CRLF

If you have a Google account and use Google webmaster tools, you can use their robots.txt checker.

10% popularity Vote Up Vote Down

: Answer lies in your line If I am blocking a page called pay.php, google won't crawl this page. If search engines doesn't even crawl that page it won't then find any links etc on that particular

@Marchetta884

0 Comments

: If you want a certificate which is going to be trusted by most user's browsers (eg. chrome/firefox) then yes, you will have to pay for that. As for SSL with payments, yes, this is crucial.

@Marchetta884

0 Comments

: Will Google Analytics track URLs that just redirect? I have a link on my site. That links goes to another URL on my site. The code on the server sees that resource being requested and redirects

@Marchetta884

Posted in: #GoogleAnalytics #Redirects

4 Comments

: You don't have permission to access /index.php on this server I made a 'login with OpenID' page and I had a error when OpenID provider return to my page: You don't have permission to

: No. The RFC says: The format logically consists of a non-empty set or records, separated by blank lines. The records consist of a set of lines of the form: <Field> ":" <value>

More posts by @Marchetta884

: Answer lies in your line If I am blocking a page called pay.php, google won't crawl this page. If search engines doesn't even crawl that page it won't then find any links etc on that particular

: If you want a certificate which is going to be trusted by most user's browsers (eg. chrome/firefox) then yes, you will have to pay for that. As for SSL with payments, yes, this is crucial.

: Will Google Analytics track URLs that just redirect? I have a link on my site. That links goes to another URL on my site. The code on the server sees that resource being requested and redirects

: You don't have permission to access /index.php on this server I made a 'login with OpenID' page and I had a error when OpenID provider return to my page: You don't have permission to

Login to post a comment!

0 Comments

Back to top | Use Dark Theme