Mobile app version of vmapp.org
Login or Join
Nimeshi995

: Search Engines Failing to Index Subdirectories I have been searching for an answer to this for a while now, but haven't posted a topic since I am under the assumption that the answer will

@Nimeshi995

Posted in: #GoogleSearchConsole #Seo

I have been searching for an answer to this for a while now, but haven't posted a topic since I am under the assumption that the answer will be pretty straight forward and that I will slap myself in the face when I finally discover the answer.

I am attempting to index all of the webpages located on the sitemap here:
www.leagueofsmurfs.com/sitemap.xml
Unfortunately, when I run a filtered search on Google using the following directive, the results do not display all of the webpages referenced in my sitemap.

site:https://www.leagueofsmurfs.com/


My robots.txt file can be found here:
www.leagueofsmurfs.com/robots.txt
What am I doing wrong? It seems that Google does not process any of the webpages with a subdirectory.

10.01% popularity Vote Up Vote Down


Login to follow query

More posts by @Nimeshi995

1 Comments

Sorted by latest first Latest Oldest Best

 

@Annie201

You should define your sitemap contents as an XML file. and that's what the first line in this code does:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml">
<url>
<loc>
www.leagueofsmurfs.com/ </loc>
<changefreq>
monthly
</changefreq>
<lastmod>
2015-05-01
</lastmod>
<priority>
1.00
</priority>
</url>
<url>
<loc>
www.leagueofsmurfs.com/products/summoner-leveling.php </loc>
<changefreq>
monthly
</changefreq>
<lastmod>
2015-05-01
</lastmod>
<priority>
0.80
</priority>
</url>
<url>
<loc>
www.leagueofsmurfs.com/products/level-30s.php </loc>
<changefreq>
monthly
</changefreq>
<lastmod>
2015-05-01
</lastmod>
<priority>
0.80
</priority>
</url>
<url>
<loc>
www.leagueofsmurfs.com/products/influence.php </loc>
<changefreq>
monthly
</changefreq>
<lastmod>
2015-05-01
</lastmod>
<priority>
0.80
</priority>
</url>
<url>
<loc>
www.leagueofsmurfs.com/terms.php </loc>
<changefreq>
monthly
</changefreq>
<lastmod>
2015-05-01
</lastmod>
<priority>
0.70
</priority>
</url>
<!-- Privacy Policy -->
<url>
<loc>
www.leagueofsmurfs.com/privacy.php </loc>
<changefreq>
monthly
</changefreq>
<lastmod>
2015-05-01
</lastmod>
<priority>
0.70
</priority>
</url>
<url>
<loc>
www.leagueofsmurfs.com/refund.php </loc>
<changefreq>
monthly
</changefreq>
<lastmod>
2015-05-01
</lastmod>
<priority>
0.70
</priority>
</url>
</urlset>


Based on your sitemap, you are only interested in indexing certain pages, not entire subfolders, and its best not to index URLs pointing to only subfolders unless you plan to have an open directory of downloadable files to the world. It's also a security benefit to prevent directory listings from appearing on your site.

So far, your security is decent since I received no content from visiting:
www.leagueofsmurfs.com/products/
Once you fix your XML sitemap, go to google webmaster tools and access leagueofsmurfs.com domain (and verify it if you haven't already done so) then choose the option to submit a sitemap and google will tell you if there are issues. If not, then wait at least half a day to give google a chance to process your sitemap because google likes to spend time at things.

I do apologize for my lack of indentation and lack of commenting in my code but its better to make the code as compact as possible to save on bandwidth, as search engines have a limited crawling budget for every website.

10% popularity Vote Up Vote Down


Back to top | Use Dark Theme