Google is adjusting its indexing system to crawl HTTPS pages in favor of their HTTP equivalents, the Internet giant announced on Thursday.
The move is meant to further promote the use of the HTTPS protocol, which should result in increased user security, the company notes in a blog post. Google also explained that it will start crawling HTTPS equivalents of HTTP web pages, even when the former are not linked to from any page.
The announcement follows last year’s adjustment in Google’s indexing system, which gave HTTPS pages a bump in rankings. By promoting HTTPS pages, the company shows its commitment towards making the web browsing a more private experience, and “not subject to eavesdropping, man-in-the-middle attacks, or data modification.”
The company has already implemented the security protocol in many of its services, including Gmail, Google, and YouTube, and the new change is expected to determine more website owners to follow suite. However, the change won’t affect domains that have only HTTP pages, it appears.
The company will favor HTTPS URLs instead of HTTP ones from the same domain only if they do not contain insecure dependencies, are not blocked from crawling by robots.txt, and do not redirect users to or through an insecure HTTP page. Additionally, HTTPS pages should not have a rel=”canonical” link to the HTTP page and should not contain a noindex robots meta tag, Google said.
Google also explains that HTTPS pages will be indexed if they do not have on-host outlinks to HTTP URLs and if the server has a valid TLS certificate. Another condition that these pages should meet is that sitemaps list the HTTPS URL, or doesn’t list the HTTP version of the URL.
Domain owners also looking to boost HTTPS pages can set their HTTP URLs to redirect to their HTTPS version and can implement the HSTS header on their server, which will determine other search engines to index the more secure page equivalents.
“We’re excited about taking another step forward in making the web more secure. By showing users HTTPS pages in our search results, we’re hoping to decrease the risk for users to browse a website over an insecure connection and making themselves vulnerable to content injection attacks,” Google’s Zineb Ait Bahajji wrote.
In the wake of fast-evolving online threats and attacks on user privacy, many tech companies have joined forces to make the Internet a safer place. One of the most recent examples is Let’s Encrypt, the free and open certificate authority (CA) focused on making it even easier for websites to adopt the HTTPS encryption, and which entered public beta earlier this month.
The initiative was proposed by the Electronic Frontier Foundation (EFF), but has received support from a variety of organizations and tech companies, including Mozilla, Cisco, Akamai, Automattic, the University of Michigan, IdenhTrust, and the Linux Foundation, which revealed in April of this year that it would host the project.