DuckDuckGo Will Automatically Encrypt More Sites You Visit
Credit to Author: Lily Hay Newman| Date: Tue, 19 Nov 2019 13:00:00 +0000
If a site offers HTTPS, DuckDuckGo's Smarter Encryption will take you there.
It's increasingly common for the data that passes between your browser and a website's server to be encrypted with HTTPS, which makes it impossible for outside snoops to read. But you don't get that protection if the URL drops that crucial "S" after HTTP. And while some mechanisms do redirect you to an encrypted version of a site, they often do so only after exposing that initial request. The makers of the privacy-focused search engine DuckDuckGo think there's a better way.
Today DuckDuckGo is releasing a feature called Smarter Encryption that combines its existing private search capabilities and tracker blocking service with a new tool to upgrade encryption for more of the sites you visit. It's available on DuckDuckGo's mobile browser for Android and iOS, and through the company's desktop browser extension for Firefox and Chrome. DuckDuckGo is also open sourcing the code behind the feature so other sites and platforms can adopt it as well. First up? Pinterest.
"I think people tend to think it’s a less of a problem because a lot of sites automatically redirect you to an encrypted version now, though a lot of sites also still don’t," says Gabriel Weinberg, DuckDuckGo's founder and CEO. "We wanted to give people a more comprehensive privacy solution no matter where the internet takes you."
DuckDuckGo isn't the first organization to tackle the HTTP upgrading problem. The Electronic Frontier Foundation's HTTPS Everywhere browser extension and Chromium's HSTS Preload List provide similar functionality. The latter is enabled by default across Chrome, Firefox, Safari, Opera, Edge, and Internet Explorer. These offerings all function basically the same way, working off a list of sites that offer HTTPS versions to upgrade connections before they're established. But DuckDuckGo's tool has one major difference: Rather than populating a list of upgradable sites manually, Smarter Encryption fills it out automatically using the same web crawling smarts built into DuckDuckGo's private search service. No one needs to add and remove entries from the list on their own; whenever the crawler sees that a site supports HTTPS, it records that as the default for all visitors using Smarter Encryption going forward, regardless of what URL they type or link they click.
This automatic element makes the list strikingly comprehensive. Compared to other tools, which have fewer than 150,000 sites on their preload lists, Smarter Encryption already works on 12 million sites, making it more likely that you'll reach for the encrypted version of a given site from the start.
"We wanted to give people a more comprehensive privacy solution no matter where the internet takes you."
Gabriel Weinberg, DuckDuckGo
Weinberg says DuckDuckGo's auto-populating strategy wasn't as easy to build as he first expected, because of the patchwork of encryption implementations on the web. For example, some sites are only set up to encrypt some of their pages. This means that if Smarter Encryption tries to upgrade your connection to that domain, some functionality and pages may break. It took a number of workarounds—including developing visual tests to automatically assess whether a page looked different after adding encryption upgrading—to make it all work without any browsing disruptions. DuckDuckGo launched a beta of the tool in 2018 to test for any issues. And now it's finally ready for prime time.
Search engines and social networks are prime platforms for adding encryption upgrades, because they both incorporate huge numbers of links that are crawler or user-generated and may not include "HTTPS." Pinterest itself is fully encrypted, but implemented Smarter Encryption to protect its users as they click links posted on the platform that lead to outside sites. Pinterest says that after incorporating DuckDuckGo's feature, about 80 percent of outbound traffic routes through HTTPS, up from 30 percent before.
"DuckDuckGo was the perfect fit for us because they maintain a comprehensive list of upgradable sites, generated by comparing the HTTP and HTTPS version of a site, and adding a site to the HTTPS upgrade list if the two versions are identical," Pinterest explains in a blog post shared with WIRED. "We can then regularly pull and ingest their list."
In an early trial deploying the changes to one percent of its users, the social network found that encryption upgrading didn't erode performance.
Smarter Encryption will upgrade more and more of your connections over time, to keep as much of your browsing data safe from prying eyes as possible. Theoretically DuckDuckGo might still be able to access those unencrypted requests, which is something to be aware of. But the company has a strong reputation, and Weinberg says that such behavior would violate its privacy policy.
There are a lot of privacy holes on the internet that need plugging and gaps in HTTPS is a prime example. Smarter Encryption is one extra protection, at least, that you can largely set and forget.