Papers tagged ‘Usable security’

Folk Models of Home Computer Security

It is well known that people who aren’t computer security experts tend to ignore expert advice on computer security, and (to some extent as a consequence) get exploited. This paper is not the first, or the last, to investigate why; see also the What Deters Jane papers [1] [2], So Long and No Thanks for the Externalities, and Users Are Not the Enemy. However, this paper provides a much more compelling explanation than anything earlier (that I have read), and a lens through which to view everything since. It’s plainly written and requires almost no specialized background knowledge; you should just go ahead and read the whole thing.

For those not in the mood to read the whole thing right now, I will summarize. Wash conducted semi-structured, qualitative interviews of 33 home computer users, who were selected to maximize sample diversity, and specifically to exclude security experts. From these, he extracted a number of what he calls folk models—qualitative, brief descriptions of how these people understand various threats to their computer security. The term folk is used to describe a model which accurately reflects users’ understanding of a system, and is broadly shared among a user population, but might not accurately reflect the true behavior of that system. [3] In this case, that means the models reflect what the interviewees think are the threats to their home computers, even if those aren’t accurate. Indeed, it is precisely where the model is inaccurate to the real threats that it provides an explanation of the phenomenon (i.e. users not bothering to follow well-publicized security advice).

A key aspect of all the models presented is a division of security threats into viruses and hackers. Virus is used by the interviewees as an umbrella term, corresponding most closely to what experts call malware—any piece of software which is unwanted and has harmful effects. (One model expands this category even further, to include programs which are unintentionally harmful, i.e. they have bugs.) The models differ primarily in users’ understanding of how viruses get into the computer, and what they are programmed to do once there. This can be very vague (e.g. viruses are bad software you don’t want on your computer) or quite specific (e.g. viruses are deliberately programmed by hackers as an act of vandalism; they cause annoying problems with the computer; you get them passively by visiting sketchy websites—an expert will acknowledge this as true-but-incomplete).

Hackers on the other hand are people who are actively seeking to exploit computers; most interviewees share the understanding that this involves taking control of a computer remotely, thus allowing it to be manipulated as if the hacker were physically present at its console. (Many of them hedge that they do not know how this is done, but they are sure that it is possible.) The models here differ primarily in the motives ascribed to the hackers, which are: vandalism, identity theft, or targeted identity theft and financial fraud. This last is one of the most telling observations in the entire paper: a significant number of people believe that they are safe from hackers because they have nothing worth stealing or exposing. (Again, an expert would recognize this as true-but-incomplete: there really is a subpopulation of black-hat actors who specialize in going after the big fish. The catch, of course, is that the data exfiltrated from a big fish might include millions of people’s personal credit card numbers…)

Having presented these models, Wash runs down a list of standard items of home computer security advice (drawn from Microsoft, CERT, and US-CERT’s guides on the topic) and points out how many of them are either useless or unimportant according to these models: for instance, if you think you can’t get viruses without actively downloading software, then antivirus software is pointless, and patching only valuable if it eliminates bugs you trip over yourself; if you think hackers rarely, if ever, vandalize a computer, then backups are not necessary to protect against that risk. He closes by comparing the novel-at-the-time threat of botnets to all the models, observing that none of them account for the possibility that an attacker might subvert computers indiscriminately and automatically, then use them only for their Internet connection. In particular, all of the hacker models assume that computers are attacked in order to do something to that computer, rather than as a means to an unrelated goal (sending spam, enlarging the botnet, executing DDoS attacks, …) and that the hacker must be doing something manually at the time of the attack.

The landscape of security threats has changed quite a bit since this paper was published. I would be curious to know whether ransomware, RATs, third-party data breaches, and so on have penetrated the public consciousness enough to change any of the models. I’d also like to know whether and how much people’s understanding of the threats to a mobile phone is different. And, although Wash did make an effort to cover a broad variety of non-expert home computer users, they are all from the general population near his Midwestern university, hence mostly WEIRDos. I’m not aware of any studies of this type conducted anywhere but North America and Europe, but I bet it’s not quite the same elsewhere…

Tor’s Usability for Censorship Circumvention

This is a report on a pilot usability study. The authors ran five journalists (there aren’t any more details than that) through the process of installing, activating, and using the Tor Browser for a small number of canned tasks, identifying a number of problems:

… people did have difficulty with installing Tor Browser (principally because of the Gatekeeper code-signing feature on OS X), did not understand what many of the many options meant, and were confused about why certain things were happening.

They are going to do a much larger study, and were soliciting feedback on experimental design. I have only two things to say. First, the proposal is to do a large test of 200 users and then, presumably, start making changes to the software to improve usability. The problem with this is, it is very likely that subtle (yet serious) UX issues are being masked out by the more blatant ones: no matter how many people you experiment on, you won’t detect the subtle problems until the blatant ones are fixed. Therefore, it would be far more valuable to do a series of smaller user studies, improving the software based on the results of each study before doing the next one. This strategy also ensures that the research results do get incorporated into the product, rather than being lost in the shuffle once the paper is published.

The other point is more of a hypothesis about what would be good to aim for. To use Tor in a way that genuinely improves your security outcomes, you need to understand what it is doing and why, and to do that you have to wrap your head around some concepts that may be unfamiliar—especially if you haven’t previously needed to understand the Internet itself in any kind of detail. (For instance, the fact that every IP packet is labeled with its source and destination is obvious once you think about it, but I never thought about it to a lot of people.) There probably needs to be a training manual, and this manual needs to take the attitude that yeah, this is a little tricky, and you have to think about it some, but don’t panic, you can understand it. Shoot for the we understand tone said to characterize Rust compiler errors (warning: Reddit). The place I’ve seen this done best, personally, was the tutorial and concepts guide for GnuCash, which took just this tone with regard to double-entry bookkeeping—also somewhat notorious for its inscrutability. (Note: I read this a long time ago, and I don’t know whether its current edition is still like that.)

Security Analysis of Consumer-Grade Anti-Theft Solutions Provided by Android Mobile Anti-Virus Apps

Today we have another analysis of Android device security against scenarios where the owner loses control of the phone, by the same researchers who wrote Security Analysis of Android Factory Resets. Here, instead of looking at what happens when the owner deliberately erases a phone in order to sell it, they study what happens when the phone is stolen and the owner tries to wipe or disable it remotely. A remote disable mechanism is commonly included in anti-virus programs for Android, and this study looks at ten such mechanisms.

As with factory reset, the core finding boils down to none of them work. The root causes are slightly different, though. The core of Android, naturally, is at pains to prevent normal applications from doing something as destructive as initiating a memory wipe, rendering the phone unresponsive to input, or changing passwords. To be properly effective the anti-theft program needs administrative privileges, which can be revoked at any time, so there’s an inherent race between the owner activating the anti-theft program and the thief disabling it. To make matters worse, UX bugs make it easy for these programs to appear to be installed correctly but not have the privileges they need; implementation bugs (possibly caused by unclear Android documentation) may leave loopholes even when the program was installed correctly; and several device vendors added backdoor capabilities (probably for in-store troubleshooting) that allow a thief to bypass the entire thing—for those familiar with Android development, we’re talking if the phone is turned off and then plugged into a computer it boots into recovery mode and activates ADB.

There is a curious omission in this paper: since 2013, Google itself has provided a remote lock/wipe feature as part of the Google Apps bundle that’s installed on most Android devices [1]. Since the Apps bundle is, nowadays, Google’s way of getting around vendors who can’t be bothered to patch the base operating system in a timely fashion, it has all the privileges it needs to do this job correctly, and the feature should be available regardless of Android base version. The UX is reasonable, and one would presume that the developers are at least somewhat less likely to make mistakes in the implementation. This paper, however, doesn’t mention that at all, despite (correctly) pointing out that this is a difficult thing for a third-party application to get right and the device vendors should step up.

Practical advice for phone-owners continues to be: encrypt the phone on first boot, before giving it any private information, and use a nontrivial unlock code. Practical advice for antivirus vendors, I think, is you’re not testing your software adversarially enough. The implementation bugs, in particular, suggest that the vendors’ test strategy confirmed that remote lock/wipe works, when set up correctly, but did not put enough effort into thinking of ways to defeat the lock.

Why Doesn’t Jane Protect Her Privacy?

Today’s paper is very similar to What Deters Jane from Preventing Identification and Tracking on the Web? and shares an author. The main difference is that it’s about email rather than the Web. The research question is the same: why don’t people use existing tools for enhancing the security and privacy of their online communications? (In this case, specifically tools for end-to-end encryption of email.) The answers are also closely related. As before, many people think no one would bother snooping on them because they aren’t important people. They may understand that their webmail provider reads their email in order to select ads to display next to it, but find this acceptable, and believe that the provider can be trusted not to do anything else with its knowledge. They may believe that the only people in a position to read their email are nation-state espionage agencies, and that trying to stop them is futile. All of these are broadly consistent with the results for the Web.

A key difference, though, is that users’ reported understanding of email-related security risks is often about a different category of threat that end-to-end encryption doesn’t help with: spam, viruses, and phishing. In fact, it may hurt: one of Gmail’s (former) engineers went on record with a detailed argument for why their ability to read all their users’ mail was essential to their ability to filter spam. [1] I’m not sure that isn’t just a case of not being able to see out of their local optimum, but it certainly does make the job simpler. Regardless, it seems to me that spam, viruses, and phishing are a much more visible and direct threat to the average email user’s personal security than any sort of surveillance. Choosing to use a service that’s very good at filtering, even at some cost in privacy, therefore strikes me as a savvy choice rather than an ignorant one. Put another way, I think a provider of end-to-end encrypted email needs to demonstrate that it can filter junk just as effectively if it wants to attract users away from existing services.

(In the current world, encryption is a signal of not being spam, but in a world where most messages were encrypted, spammers would start using encryption, and so would your PHB who keeps sending you virus-infected spreadsheets that you have to look at for your job.)

Another key difference is, you can unilaterally start using Tor, anti-tracking browser extensions, and so on, but you can’t unilaterally start encrypting your email. You can only send encrypted email to people who can receive encrypted email. Right now, that means there is a strong network effect against the use of encrypted email. There’s not a single word about this in the paper, and I find that a serious omission. It does specifically say that they avoided asking people about their experiences (if any) with PGP and similar software because they didn’t want to steer their thinking that way, but I think that was a mistake. It means they can’t distinguish what people think about email privacy in general, from what they think about end-to-end encryption tools that they may have tried, or at least heard of. There may be a substantial population of people who only looked into PGP just enough to discover that it’s only useful if the recipient also uses it, and don’t think of it anymore unless specifically prompted about it.

Security Analysis of Android Factory Resets

Unlike the last two papers, today’s isn’t theoretical at all. It’s a straightforward experimental study of whether or not any data can be recovered from an Android phone that’s undergone a factory reset. Factory reset is supposed to render it safe to sell a phone on the secondhand market, which means any personal data belonging to the previous owner should be erased at least thoroughly enough that the new owner cannot get at it without disassembling the phone. (This is NIST logical media sanitization, for those who are familiar with those rules—digital sanitization, where the new owner cannot extract any data short of taking an electron microscope to the memory chip, would of course be better, but in the situations where the difference matters, merely having a cell phone may mean you have already failed.)

The paper’s core finding is also straightforward: due to a combination of UI design errors, major oversights in the implementation of the factory-reset feature, driver bugs, and the market’s insistence that flash memory chips have to pretend to be traditional spinning rust hard disks even though they don’t work anything like that, none of the phones in the study erased everything that needed to get erased. The details are confusingly presented, but they’re not that important anyway. It ends with a concrete set of recommendations for future improvements, which is nice to see.

There are a few caveats. The study only covers Android 2.3 through 4.3; it is unclear to me whether the situation is improved in newer versions of Android. They don’t discuss device encryption at all—this is at least available in all versions studied, and should completely mitigate the problem provided that a factory reset successfully wipes out the old encryption keys, and that there’s no way for unencrypted data to survive the encryption process. (Unfortunately, the normal setup flow for a new phone, at least in the versions I’ve used, asks for a bunch of sensitive info before offering to encrypt the phone. You can bypass this but it’s not obvious how.) It would be great if someone tested whether encryption is effective in practice.

Beyond the obvious, this is also an example of Android’s notorious update problem. Many of the cell carriers cannot be bothered to push security updates—or any updates—to phones once they have been sold. [1] They are also notorious for making clumsy modifications to the software that harm security, usability, or both. [2] In this case, this means driver bugs that prevent the core OS from erasing everything it meant to erase, and failure to deploy patches that made the secure erase mechanism work better. Nobody really has a good solution to that. I personally am inclined to say that neither the telcos nor Google should be in the business of selling phones, and conversely, the consumer electronics companies who are in that business should not have the opportunity to make any modifications to the operating system. Whatever the hardware is, it’s got to work with a stock set of drivers, and the updates have to come direct from Google. That would at least simplify the problem.

(Arguably Google shouldn’t be in the OS business either, but that’s a more general antitrust headache. One thing at a time.)

Secrets, Lies, and Account Recovery

Today’s paper appeared in WWW 2015 and was also summarized on Google’s security blog. The subtitle is Lessons from the Use of Personal Knowledge Questions at Google, and the basic conclusion is: security questions (for recovering access to an account when you’ve forgotten your password) are terribly insecure. Well, duh, you are probably thinking, the answers are right there on my social network profile for anyone to read. That’s true, but it’s not the angle this paper takes. This paper is more interested in guessing by attackers who haven’t read your social network profile. At most, they know your preferred language and country of residence. It turns out that, if you only ask one security question for account recovery, three guesses by this adversary are sufficient to penetrate 10–20% of accounts; that is how many people typically share the most frequent three answers per language. (Depending on the question and language, it might be much worse; for instance, the three most common places of birth for Korean speakers cover 40% of that population.)

They also observe that questions where there should be no shared answers (what is your frequent flyer number?) do have shared answers, because people give fake answers, and the fake answers tend to be things like 123456790. This brings the security of such questions down to the level of all the others. Finally, there is a user survey, from which the most telling two items are: People think account recovery via security question is more secure and more reliable than account recovery via a secret code sent to them in a text message or email (precisely the opposite is true). People give fake answers because they want to avoid having the answers be visible on their social network profile (or otherwise easy to find).

If someone knows almost nothing about you, why are they bothering to try to take over your account? Probably the two most important reasons are they need throwaway addresses to send spam with and if they steal $100 from ten thousand people, that adds up to a million dollars. These are compelling enough that any major online service provider (such as Google) sees continuous, bulk, brute-force attacks by adversaries who don’t care which accounts they compromise. The authors don’t say, but I wouldn’t be surprised if those attacks constituted a supermajority of all attacks on Google accounts.

Turning it around, though, the probable negative consequences of having your account cracked by a spammer who doesn’t know or care about you as a person are really not that bad, compared to what someone who knows you and bears you a personal grudge could do. (Ransomware is the most prominent counterexample at present.) Google as a corporate entity is right to worry more about the most frequent type of attack they see—but you as an individual are probably also right to worry more about an attack that’s much less common but has way more downside for you. I think this contrast of priorities explains fake answers and the general reluctance to adopt two-factor authentication, recovery codes, etc. all of which might provide a security benefit in the future but definitely make life more inconvenient now. So Long, And No Thanks for the Externalities and How Do We Design For Learned Helplessness? go into considerably more detail on that point. (Facebook may well have improved their password reset sequence since the latter was written.) On a related note, if someone happens to have an email address or phone number that isn’t already looped back into the account that needs a recovery contact point, they might well consider it critically important that the panopticon running that account does not find out about it!

What Deters Jane from Preventing Identification and Tracking on the Web?

If you do a survey, large majorities of average people will say they don’t like the idea of other people snooping on what they do online. [1] [2] Yet, the existing bolt-on software that can prevent such snooping (at least somewhat) doesn’t get used by nearly as many people. The default explanation for this is that it’s because the software is hard to install and use correctly. [3] [4]

This paper presents a complementary answer: maybe people don’t realize just how ubiquitous or invasive online snooping is, so the benefit seems not worth the hassle. The authors interviewed a small group about their beliefs concerning identification and tracking. (They admit that the study group skews young and technical, and plan to broaden the study in the future.) Highlights include: People are primarily concerned about data they explicitly provide to some service—social network posts, bank account data, buying habits—and may not even be aware that ad networks and the like can build up comprehensive profiles of online activity even if all they do is browse. They often have heard a bunch of fragmentary information about cookies and supercookies and IP addresses and so on, and don’t know how this all fits together or which bits of it to worry about. Some people thought that tracking was only possible for services with which they have an account, while they are logged in (so they log out as soon as they’re done with the service). There is also general confusion about which security threats qualify as identification and tracking—to be fair, just about all of them can include some identification or tracking component. The consequences of being tracked online are unclear, leading people to underestimate the potential harm. And finally, many of the respondents assume they are not important people and therefore no one would bother tracking them. All of these observations are consistent with earlier studies in the same vein, e.g. Rick Wash’s Folk Models of Home Computer Security.

The authors argue that this means maybe the usability problems of the bolt-on privacy software are overstated, and user education about online security threats (and the mechanism of the Internet in general) should have higher priority. I think this goes too far. It seems more likely to me that because people underestimate the risk and don’t particularly understand how the privacy software would help, they are not motivated to overcome the usability problems. I am also skeptical of the effectiveness of user education. The mythical average users may well feel, and understandably so, that they should not need to know exactly what a cookie is, or exactly what data gets sent back and forth between their computers and the cloud, or the internal structure of that cloud. Why is it that the device that they own is not acting in their best interest in the first place?