Allow Browser Access From Robots.txt File For Blog

Allow Browser Access From Robots.txt File For Blog

Adsense publishers have recently complained about Adsense ads not appearing or ad slots appearing empty, especially for newly published publications.

Managing websites: Grant access to our crawler from your Robots.txt file

There are many speculations on this subject, one of which is that Google is probably updating Adsense.

In fact, Google keeps updating its products constantly. For example, Page Speed ​​Insight and Google Search Console have made updates due to some adjustments.
Similarly, with Adsense, Google has changed the way it serves ads on the website. If the web or blog or a page of the web or blog is not yet accessible, or if the Adsense bot cannot be accessed, Adsense does not automatically activate the web or blog monetization feature, this means that the ad revenue is not immediately active, just like Youtube videos. In this way, Adsense has made some changes to comply with AdSense policy and be able to track all posts more easily.

See: Adding Adsense Post Ads to Blog Posts

This change is clearly noticeable when you publish a new post. All Adsense ads will not appear on newly published posts or appear blank.

Of course, it makes sense, because Google bots may not have access to new posts. Adsense ads appear after the page is accessed by Google bots.

To ensure that Google and Adsense bots are crawling your blog, enable robots.txt and allow crawler robots access.

A warning is displayed in Google Search Console if the robots.txt file is not enabled on the blog. It appears that the Google robot could not crawl the blog because it could not find the robots.txt file for bot access rights.

Allow Browser Access From Robots.txt File For Blog


To fix these issues, make the following changes:

If you continue to use the robots.txt file on your blog as the file below, replace the robots.txt file on your blog.

Robots.TXT
User-agent: Mediapartners-Google
Disallow:

User-agent: *
Disallow: /search
Allow: /

Sitemap: http://blog_name/feeds/posts/default?orderby=UPDATED

Replace the robots.txt file above, the above robots.txt file will essentially block Adsense bots to crawl the blog, description here. So, pay attention to the SEO-friendly robots.txt setting suggestions that will block the Adsense bot. For this, use the robots.txt file below.

Robots.TXT
User-agent: *
Disallow: /search
Allow: /

Sitemap: http://blog_name/sitemap.xml

In order for Google bots to quickly crawl and update your new posts, enter your new post URL Search Submit it to Console. You can do this after each new release. As you can see in the image below, when you report the post URL to Google, you can see that the post is crawled and indexed within a few minutes.

Image
Fast Indexing Blog Posts

In this article, we have shared some information about the fact that Adsense ads are not displayed in new publications added to the blog, by citing the source. We welcome you to send us your comments and questions about this article by using the comment form.
Next Post Previous Post
No Comment
Add Comment
comment url