The subtitle of today’s paper is On Measuring Censors’ Resources and Motivations. It’s a position paper, whose goal is to get other researchers to start considering how economic constraints might affect the much-hypothesized arms race or tit-for-tat behavior of censors and people reacting to censorship: as they say,

[…] the censor and censored have some level of motivation to accomplish various goals, some limited amount of resources to expend, and real-time deadlines that are due to the timeliness of the information that is being spread.

They back up their position by presenting a few pilot studies, of which the most compelling is the investigation of keyword censorship on Weibo (a Chinese microblogging service). They observe that searches are much more aggressively keyword-censored than posts—that is, for many examples of known-censored keywords, one is permitted to make a post on Weibo containing that keyword, but searches for that keyword will produce either no results or very few results. (They don’t say whether unrelated searches will turn up posts containing censored keywords.) They also observe that, for some keywords that are not permitted to be posted, the server only bothers checking for variations on the keyword if the user making the post has previously tried to post the literal keyword. (Again, the exact scope of the phenomenon is unclear—does an attempt to post any blocked keyword make the server check more aggressively for variations on all blocked keywords, or just that one? How long does this escalation last?) And finally, whoever is maintaining the keyword blacklists at Weibo seems to care most about controlling the news cycle: terms associated with breaking news that the government does not like are immediately added to the blacklist, and removed again just as quickly when the event falls out of the news cycle or is resolved positively. They give detailed information about this phenomenon for one news item, the Wukan incident, and cite several other keywords that seem to have been treated the same.

They compare Weibo’s behavior to similar keyword censorship by chat programs popular in China, where the same patterns appear, but whoever is maintaining the lists is sloppier and slower about it. This is clear evidence that the lists are not maintained centrally (by some government agency) and they suggest that many companies are not trying very hard:

At times, we often suspected that a keyword blacklist was being typed up by an over-worked college intern who was given vague instructions to filter out anything that might be against the law.

Sadly, I haven’t seen much in the way of people stepping up to the challenge presented, designing experiments to probe the economics of censorship. You can see similar data points in other studies of China [1] [2] [3] (it is still the case, as far as I know, that ignoring spurious TCP RST packets is sufficient to evade several aspects of the Great Firewall), and in reports from other countries. It is telling, for instance, that Pakistani censors did not bother to update their blacklist of porn sites to keep up with a shift in viewing habits. [4] George Danezis has been talking about the economics of anonymity and surveillance for quite some time now [5] [6] but that’s not quite the same thing. I mentioned above some obvious follow-on research just for Weibo, and I don’t think anyone’s done that. Please tell me if I’ve missed something.

It is well known that people who aren’t computer security experts tend to ignore expert advice on computer security, and (to some extent as a consequence) get exploited. This paper is not the first, or the last, to investigate why; see also the What Deters Jane papers [1] [2], So Long and No Thanks for the Externalities, and Users Are Not the Enemy. However, this paper provides a much more compelling explanation than anything earlier (that I have read), and a lens through which to view everything since. It’s plainly written and requires almost no specialized background knowledge; you should just go ahead and read the whole thing.

For those not in the mood to read the whole thing right now, I will summarize. Wash conducted semi-structured, qualitative interviews of 33 home computer users, who were selected to maximize sample diversity, and specifically to exclude security experts. From these, he extracted a number of what he calls folk models—qualitative, brief descriptions of how these people understand various threats to their computer security. The term folk is used to describe a model which accurately reflects users’ understanding of a system, and is broadly shared among a user population, but might not accurately reflect the true behavior of that system. [3] In this case, that means the models reflect what the interviewees think are the threats to their home computers, even if those aren’t accurate. Indeed, it is precisely where the model is inaccurate to the real threats that it provides an explanation of the phenomenon (i.e. users not bothering to follow well-publicized security advice).

A key aspect of all the models presented is a division of security threats into viruses and hackers. Virus is used by the interviewees as an umbrella term, corresponding most closely to what experts call malware—any piece of software which is unwanted and has harmful effects. (One model expands this category even further, to include programs which are unintentionally harmful, i.e. they have bugs.) The models differ primarily in users’ understanding of how viruses get into the computer, and what they are programmed to do once there. This can be very vague (e.g. viruses are bad software you don’t want on your computer) or quite specific (e.g. viruses are deliberately programmed by hackers as an act of vandalism; they cause annoying problems with the computer; you get them passively by visiting sketchy websites—an expert will acknowledge this as true-but-incomplete).

Hackers on the other hand are people who are actively seeking to exploit computers; most interviewees share the understanding that this involves taking control of a computer remotely, thus allowing it to be manipulated as if the hacker were physically present at its console. (Many of them hedge that they do not know how this is done, but they are sure that it is possible.) The models here differ primarily in the motives ascribed to the hackers, which are: vandalism, identity theft, or targeted identity theft and financial fraud. This last is one of the most telling observations in the entire paper: a significant number of people believe that they are safe from hackers because they have nothing worth stealing or exposing. (Again, an expert would recognize this as true-but-incomplete: there really is a subpopulation of black-hat actors who specialize in going after the big fish. The catch, of course, is that the data exfiltrated from a big fish might include millions of people’s personal credit card numbers…)

Having presented these models, Wash runs down a list of standard items of home computer security advice (drawn from Microsoft, CERT, and US-CERT’s guides on the topic) and points out how many of them are either useless or unimportant according to these models: for instance, if you think you can’t get viruses without actively downloading software, then antivirus software is pointless, and patching only valuable if it eliminates bugs you trip over yourself; if you think hackers rarely, if ever, vandalize a computer, then backups are not necessary to protect against that risk. He closes by comparing the novel-at-the-time threat of botnets to all the models, observing that none of them account for the possibility that an attacker might subvert computers indiscriminately and automatically, then use them only for their Internet connection. In particular, all of the hacker models assume that computers are attacked in order to do something to that computer, rather than as a means to an unrelated goal (sending spam, enlarging the botnet, executing DDoS attacks, …) and that the hacker must be doing something manually at the time of the attack.

The landscape of security threats has changed quite a bit since this paper was published. I would be curious to know whether ransomware, RATs, third-party data breaches, and so on have penetrated the public consciousness enough to change any of the models. I’d also like to know whether and how much people’s understanding of the threats to a mobile phone is different. And, although Wash did make an effort to cover a broad variety of non-expert home computer users, they are all from the general population near his Midwestern university, hence mostly WEIRDos. I’m not aware of any studies of this type conducted anywhere but North America and Europe, but I bet it’s not quite the same elsewhere…

Today’s paper takes another shot at modeling how the physical topology of the Internet affects the security of Tor against passive adversaries with the ability to snoop on a lot of traffic. It’s by some of the same people who wrote Defending Tor from Network Adversaries and is quite closely related.

Most of the work of this paper goes into building a flexible, formal threat model, which Tor client software could (in principle) use to inform its routing decisions. Acknowledging that there’s always going to be a good deal of uncertainty about what adversaries are out there and what they are capable of, they make two key design decisions. The model is probabilistic (based on a Bayesian belief network), and it takes user input. For instance, if you have reason to think the government of Transbelvia has it in for you, you can instruct Tor to avoid paths that Transbelvia might be able to snoop on, and the model will expand that out to all the ways they might do that. Conversely, if you trust a particular organization you might like to preferentially use its guards or exit nodes, and it can do that too.

The model is very thorough about different ways a government might be able to snoop on network traffic—not just relays physically hosted in the country, but ASes and IXPs (Transbelvia hosts a major IXP for Eastern Europe), submarine cable landing sites (not relevant for a landlocked country), mutual legal assistance treaties (MLATs) which might be used to have another country do some snooping on Transbelvia’s behalf, and even hacking into and subverting routers at interesting points in the connectivity graph. (The pun in the title refers to their analysis of how MLATs could allow several of the usual suspects to snoop on 90+% of all submarine cable traffic, even though they host hardly any cable landings themselves.) Equally important, it can be expanded at need when new techniques for spying are revealed.

I think something like this is going to be an essential building block if we want to add any spy-aware routing algorithm to Tor, but I have two serious reservations. First, simplest, but less important, right now all Tor clients make routing decisions more-or-less the same way (there have been small changes to the algorithm over time, but everyone is strongly encouraged to stay close to the latest client release anyway, just because of bugs). If clients don’t all make routing decisions the same way, then that by itself might be usable to fingerprint them, and thus cut down the number of people who might’ve taken some action, from all Tor users to all Tor users who make routing decisions like THIS. If highly personalized threat models are allowed, the latter group might be just one person.

Second, and rather more serious, the user-input aspect of this system is going to require major user experience research and design to have any hope of not being worse than the problem it’s trying to solve. It’s not just a matter of putting a friendly face on the belief language (although that does need to happen)—the system will need to educate its users in the meaning of what it is telling them, and it will need to walk them through the consequences of their choices. And it might need to provide nudges if there’s a good reason to think the user’s assessment of their threat model is flat-out wrong (even just making that judgement automatically is fraught with peril—but so is not making that judgement).