Skip to content

The End of Online Anonymity

June 10, 2019

Since 2015, I’ve been writing about the impact that machine learning will have on our society. One of the most concerning possibilities, in my mind, was always the potential abuse of these technologies by malicious actors to manipulate or scam people, either through subtle means or by impersonating those they trust.

Today, this concern is very much mainstream: “fake news” has become a buzzword and a kind of modern-day boogeyman. For the most part, I think most people aren’t overly worried. We know that there already are malicious actors creating sketchy content and putting it out there, but most of it seems obviously fake if you examine it more closely. We all assume that we will always be smart enough to tell real from fake, and carry on.

Media manipulation is nothing new. Attempts to control public discourse and influence the masses predate the internet, TV, newspapers and the printing press. What’s about to change is that now, with machine learning, it will become possible to turn electricity into millions of voices relentlessly spreading your gospel to every corner of the internet. At this point in time, it seems most of the fake content out there is not generated using machine learning, it’s created by human beings using puppet accounts. For the most part, someone still has to turn the crank. That limits how much content can be created and how many sources it can come from.

Personally, I’m not just worried about manipulative articles being passed as news. I’m also worried about the impact that networks of malicious bots will have on online communities. We’re still fairly far from being at the point where we can automatically generate news articles that appear convincing upon close inspection, but what about online comment threads? How difficult is it to build a bot that can write convincing one or two sentences comments?

Yesterday, I stumbled upon a link to a subreddit populated by bots based on OpenAIs GPT-2 text generation model. The result is certainly funny, but also leaves me feeling uncomfortable. Yes, much of the content is obviously fake, but many of the comments are actually believable. If you feel unimpressed, you should keep in mind that this is an individual’s side project that repurposed an existing neural network. As it is, the GPT-2 model simply generates text and completes a sentence. It’s an impressive and amusing tech demo, but not something you can easily control. In order to weaponize GPT-2, a malicious actor would need to add some kind of a guidance system: a way to condition text output of the model so as to spread a specific message.

The solution to the fake content problem may seem obvious: we can fight fire with fire, and build machine learning tools to detect machine-generated content. Tools like this are already in the works. Grover boasts 92% accuracy in detecting fake content. The sad reality, however, is that this is an arms race, and it’s not clear at all that this is something we can win. Facebook already employs thousands of human agents for the purpose of detecting and flagging malicious actors, and these people do flag a large volume of content, but they are struggling to keep up. As technology improves, fake content will become harder and harder to tell apart from real content. Manual content verification won’t be able to keep up with the volume, and automated filtering systems will fail.

In my opinion, there is only one effective way to stop fake content, and this is to verify that everyone who posts content is in fact human. You could ask people to upload pictures of themselves, but we’re already at the point where we can produce realistic images of imaginary people using GANs. Any counter-measure of this form will inevitably be defeated. Ultimately, one possibility is that online platforms will begin requiring a verified government ID in order to register. We could even end up living in a world where a kind of “e-passport”, crypto-signed government ID is attached to your every internet connection, and tracked everywhere online.

The rise of bots could render many online communities simply uninhabitable.  Large websites such as Facebook and reddit may have some hope of policing content, but smaller independent players likely won’t have the resources. We are moving towards a model where the internet is dominated by a few centralized content providers and their walled gardens, and generated content may unfortunately make it even harder for grassroots online communities to survive and grow.

I wish I had a more optimistic message for this post. I wish I could do more than point at a potential problem. Maybe there is a way to build a new web, a new kind of social media using a hash graph to implement a decentralized web of trust, something that can allow content verification without forcing everyone to sacrifice their right to remain anonymous online. I certainly think it’s a problem that’s worth thinking about, because unless we can come up with a technical solution, a regulatory solution may be imposed onto us, and it will inevitably favor the big players at the expense of the small.

Advertisements
9 Comments
  1. Mike S. permalink

    I think requiring a government-issued ID to participate opens its own can of worms. It erodes individual privacy even further. It also still permits the government itself to create propaganda AI bots by abusing the ID system – so a random business or media outlet might not be able to spam the network, but the government still can.

    Last month I set up my own Mastodon instance. You’re probably aware of it, it’s a free software / open source federated Twitter alternative. Many Mastodon servers have open registration, just like Facebook, Twitter, and so forth. But many have switched to invitation-only and that’s how I set my instance up. I think a shift to that model might be inevitable. I don’t think it’s a complete solution, but I’m at a loss to conceptualize anything better.

    • > I think requiring a government-issued ID to participate opens its own can of worms.

      It’s not a solution that I like or want, it’s the kind of solution that regulators might come up with in a “think of the children!” reactionary move.

      I hadn’t heard of Mastodon. I like the decentralized aspect, but it also sounds like way more of a free for all. I guess you can police your own small community yourself, but when it comes to the intersection of these small communities, I’m somewhat skeptical?

      • Mike S. permalink

        Oh, I see what you mean about having use of real identities imposed by authorities. I agree it’s a real possibility. I had misunderstood your post to mean you thought it was a good idea.

        With Mastodon the convention is that when you host an instance you post a Code of Conduct and enforce it. When a site comes up without one or with one that allows certain forms of discrimination a lot of the other sites do a whole-server block on that domain immediately. That won’t scale forever, if Mastodon gets big enough I’m sure the spam and bigot groups will create thousands of instances programmatically to spew advertisements and hatred and it will be impractical to manually block them all. At that point it would make sense to switch to domain whitelisting. But so far my signal to noise ratio is really good.

  2. Brian permalink

    Probably a little bit on the dark side, think reality will be a little better!

    Firstly there will always be sites that actually do journalism, albeit with bias views, BBC,Al Jazeera. even Fox News and RT often have some level of truth. If you follow enough of them you will get a reasonable feel for the truth.

    More importantly truth and ‘reality’ are often a matter of the writer and readers perspective bot or no bot

    Secondly future generations(hopefully) will be better schooled in evaluating information and its reliability, Generations x, y and z seem to be more gullible with each generation, as technology becomes more and more part of their upbringing, they need to be taught to be less trusting!

    Although probably all this might be irrelevant after the machines take over…….

    Of course we still have the option off the ‘OFF’ switch, till Apple/Google/Microsoft decide to remove it!

  3. Go to the source of information instead of stopping with just the article / forum post / etc. Once you get to the source, you can evaluate it and remember that the info came from THERE.

    If you’re so gullible as to believe some random snippet of info online (and we all are at some point because trust is a necessity for human communities), then you’re likely to pick up incorrect info anyways, bot or no bot.

    Here’s a thought: if the number of bots increases for one point of view, don’t you think there would be roughly the same number arguing for the opposite point of view?

    I disagree with Brian: truth are reality are NOT often a matter of perspective. But I think his opinion stems from considering that most “news” has turned into opinion: X is good, Y is bad. Says who? Depends on who you read.

    • brian permalink

      “truth are reality are NOT often a matter of perspective.”

      You don’t have to spend too long looking at how statistics are used for political ends to know truth and reality is a matter of perspective :)

  4. Anonymous permalink

    The only real solution that I know of are WoTs because they provide enumeration of goodness as opposed to enumeration of badness, which is what filters do. But the main thing is that the vast majority of human-generated content is junk and large communities typically suck one way or another. So you don’t even need a solution that would scale. Quality over quantity.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: