Back in 1996, the website TotalNews.com had a brilliantly evil idea. Why not make a single website that itself contained all the top American news websites, directly embedded within?
After all, why should people go to the trouble of visiting the Washington Post or New York Times websites directly? On TotalNews.com, the websites would be available in one place, with easy navigation.
Using a new technology called the HTML <frame> element, TotalNews embedded the top American news sites — content, design and everything! — in such a way that the news was always fresh. (Because, well, it was literally those other websites.)
Naturally, TotalNews added advertising around the news as well. It was only fair for them to be compensated with ad revenue for providing this incredible convenience.
Yeah, it was the early, wild west days of the web — but you have to admire their chutzpah. My own web browsing at the time was spent on Pearl Jam fan sites, so I never saw TotalNews firsthand, but this Archive.org snapshot from December 1996 provides a vague picture.
Somehow news sites didn’t appreciate TotalNews misappropriating and profiting from their content. So in February 1997, a group of them sued.
The case, The Washington Post, et als. v. TotalNews, Inc., et als., was settled out of court, and TotalNews was prohibited from embedding the sites going forward. This established a precedent, if not legally then at least culturally: Framing without permission was not OK.
But the shady practice continued. So much, in fact, that “framebuster” scripts became popular. This is the technique of putting some JavaScript in your site to check whether it’s currently being framed — and “breaking out of” the frame as needed. (See this excellent 2010 review of framebusting techniques and its accompanying slide deck for a technical overview.) This is essentially website self-defense.
Over time, web developers and security researchers realized there’s a more serious reason a website would want to protect against framing: clickjacking. That’s when, for example, a website frames your site, then hijacks user input such that users are fooled into thinking they’re interacting with your site while they’re actually providing data to the (evil) containing site. Imagine typing your bank credentials into (what you think is) your bank website, whereas it’s in fact an evil site logging everything you’ve typed.
These days, framebuster scripts are no longer necessary, because websites can use a special HTTP header, X-Frame-Options, to block framing in an elegant and effective way. And lovely web frameworks such as Django provide protection, via that header, out of the box. Framebusting is more or less a solved problem.
Except in one major case.
If you click a web link in the native Facebook, Instagram, Reddit or Twitter apps on your smartphone, you won’t be taken to your phone’s web browser. Instead, the app embeds the web page directly, so you don’t leave their environment.
For example, here’s what I got in the Twitter iOS app when I clicked the link in one of Simon Willison’s recent tweets:
To the untrained eye, this appears to be my phone’s web browser. It doesn’t identify itself as Twitter anywhere, and it looks, well, pretty plain.
But in fact, this is something entirely different — a more ephemeral thing called a “webview” or “in-app browser.” This is a way for a native app to embed a mini web browser, while asserting control over the user experience and attaching UI, functionality and other cruft. It looks like a separate browser, but in fact it’s still the Twitter app in disguise.
Seem familiar? This is framing, merely in app form. But this time, the framed website has no way to framebust.
It’s TotalNews — but for the 2020s, and much worse. These native apps aren’t (for the most part) putting advertising around websites — but they’re maintaining control over the user’s browsing experience, sometimes spying on users, and providing various problems for the framed websites, with zero recourse available for the users or website owners.
Somewhere along the way, despite a reasonably strong anti-framing culture, framing moved from being a huge no-no to a huge shrug. In a web context, it’s maligned; in a native app context, it’s totally ignored.
Why, precisely, is this bad? Here are four reasons. Each has a specific example, and in almost each case I have direct experience in my work running Soundslice.
Misappropriation
A native app can make misleading claims about the websites that it frames. And due to the seamless way webviews are implemented, a nontechnical user would have no way of knowing that they’re actually viewing a completely unaffiliated website in context of the native app.
For example, a few years ago we got word that an Android app was embedding Soundslice’s free MusicXML file viewer. The app was offering this as a “feature” to their users.
We didn’t find out about it until they had the nerve to contact us to report a bug regarding file upload within webviews.
We were doing everything right — our website already sends out the X-Frame-Options: Deny header, meaning we don’t allow framing. Yet native apps (and their mobile operating systems) ignore this header, which is a gaping loophole.
Poor user experience
If a native app opens a third-party website in a webview, that third-party site will begin a new session, without existing cookies. It’s effectively like using a web browser’s private (aka “incognito”) mode.
This means: If you’re logged into a website in your phone’s browser, but you click a link to that site from a native app’s webview, your logged-in state will not be honored. You’ll need to log in all over again.
At best, this is irritating. At worst, it gives people the false impression that the website is broken or logged them out.
A specific example is Soundslice’s Instagram account, where we highlight stuff people have created on our platform. If you see something on our Instagram and want to add it to a practice list in your Soundslice account, you quickly run into friction when opening the link within the Instagram app. Even if you’re logged into Soundslice in your phone’s browser, the Instagram app doesn’t honor your Soundslice login.
[Instagram is a particularly heinous example, because it doesn’t even let you add a link to a post. If you enter a URL in an Instagram post, it will not turn into a clickable link. Only one link, the one in your channel bio, is clickable. An entire cottage industry has formed around this “link in bio” madness.]
Yes, you can copy-and-paste the URL (if the webview displays it), or you can choose an “Open in web browser” option (if the webview provides it). But either case requires nontrivial technical sophistication. Most users would just say “Aw, man, I’ve somehow been logged out of Soundslice” and assign the blame to us.
Proponents of native apps would likely argue “But it’s a better user experience from the perspective of the native app! Because the user doesn’t have to context-switch into a different environment, aka the web browser.” There was indeed a time when this argument made sense: the years before 2015, which is when iOS 9 introduced a global Back button conveniently solving the problem. And of course Android has its global Back button. These days this argument holds no water.
Proponents of native apps would also likely say: “If you had your own native app, this problem would be solved, because you can register a link handler that will automatically open all soundslice.com URLs in your native app.” iOS calls this Universal Links; Android calls it App Links. This is true. It’s also an unreasonable, disproportionate demand. I shouldn’t have to develop a native app simply to wrangle control over how my website’s links are treated.
Weird, non-standard browsers
Popular apps such as Instagram and Facebook don’t just use vanilla webviews. They use customized ones, with their own quirks.
This means: If you click a link in the Facebook app and it opens in Facebook’s webview, the website might be slightly broken. In my experience, page dimensions/layout can be affected, and the website is provided incorrect information about its environment.
I highly recommend reading Alex Russell’s piece Hobson’s Browser, full of technical details on the deficiencies of in-app browsers.
A specific example is yet again Soundslice. Our main content type — the thing we usually link to — is a piece of interactive sheet music, which is dynamically sized to your screen. We’ve specifically had problems with the Instagram webview not properly communicating its screen size — leading our site to apply incorrect dimensions to the sheet music. Again, it’s a situation where we look bad due to an obscure technical detail outside our control.
Hilariously, a few years ago a Facebook employee announced they added a way to debug your website when it’s viewed in context of the Facebook in-app browser. After I replied it’d be easier if Facebook just opened links in the default browser in the first place, the employee rationalized the webview by saying it helps protect people. Which brings us to:
Security
This is the most important problem, though fortunately it’s the only one I haven’t directly experienced in my own work.
When a native app embeds a website via a webview, the native app has control over that page. Yes, even if it’s on a domain that the native app doesn’t control (!). This means the native app can inject whatever JavaScript it likes into any website that’s viewed in the webview.
Today I read an astounding exposé by Felix Krause, in which he discovered the Facebook and Instagram iOS apps inject JavaScript into all web pages that are viewed in their webviews. You should read and process this.
Facebook has a sterling reputation to uphold, so I’m sure they wouldn’t do anything horrible here. But more nefarious apps could steal passwords or perform other types of attacks.
The more I think about it, the more I cannot believe webviews with unfettered JavaScript access to third-party websites ever became a legitimate, accepted technology. It’s bad for users, and it’s bad for websites.
But fortunately, I think something can be done about all this.
A proposed solution
Fundamentally this is about power. It’s a struggle between four participants:
- The user wants to click a link to a website, retaining any useful state, with the ability to freely jump between apps/sites.
- The website wants the user to have a smooth experience.
- The native app wants to keep the user within its app (aka lock-in). In some cases, such as Facebook, the app wants to collect detailed information about the user’s browsing behavior.
- The mobile operating system (iOS and Android) wants developers to build native apps on its platform. The web is a bit of an afterthought, a lower priority.
At the moment, the power is squarely in the hands of the last two. I believe it should be more balanced, giving website owners a way to opt out of this behavior — in old-school terms, a way to framebust.
So my proposal is this: Apple and Google should honor the existing X-Frame-Options HTTP header in webviews. If a website is loaded into a webview, and the website includes the appropriate X-Frame-Options header, the mobile OS should immediately stop loading the webview and open the URL in the user’s preferred web browser.
This elegantly uses an existing technology and gives websites a much-needed opt out.
Unfortunately, the only way for this to happen would be to convince Apple and Google to do it. I can’t imagine a general opt-out solution that doesn’t involve iOS and Android providing hooks at the OS level. And, as Co-Monopolists Of The World Of Native Phone Apps, Apple and Google have zero incentive to make such a change.
It could in theory be implemented by individual apps, but I wouldn’t trust Facebook to do this because of conflict of interest. And the sad precedent of the Do Not Track header is instructive.
Our best bet is regulatory intervention, along the lines of what Open Web Advocacy is doing. In collecting my thoughts here, I hope to start this conversation. The modern version of TotalNews must be reined in.
Comments aren’t enabled for this page.