Climate Strike Observations

There is a global #ClimateStrike planned for September 20th through the 27th. This is a series of peaceful protests regarding government inaction on climate change. Some are announced in advance, there will certainly be popup activities as things get rolling, and there are multiple groups involved. Several of us have done work for various environmental and clean energy groups, and we are well aware of corporate PR campaigns and their automated backing.

We began tracking Extinction Rebellion’s efforts in early July of 2019. We only found 112 accounts then, but an update this week has revealed at least 397 that are active. We have an index of 214,000 tweets and mentions for those original 112.

Our systems have spent the last many hours grinding on follower profiles for several different constituencies:

  • Extinction Rebellion’s 397 accounts
  • 350 & Bill McKibben
  • DeSmogBlog
  • 17 Deniers profiled on DeSmogBlog

The denier accounts are from a study done about five years ago, so it is in no way a complete representation. We will probably have a look at the players in Lewandowsky et. al.’s Recursive Fury study, too.

The creation date curve for followers is a clue to legitimacy; bots became a rising force in 2015, and older accounts that have been “bleached” also participate in such activities. Bill McKibben’s followers show no sign of such activity.

And the @350 account seems to have a similar audience, except for this intriguing spike of accounts created in 2011. We’ll dig deeper into that later.

The followers of the Deniers are only 50% collected at this point, but the difference is stark. Given that there is one very large account in the mix this curve may balance out a bit once things are done, but the ramp up in 2015 and the sharp drop in mid-2018 are signs of automated accounts that we see for almost all legislators in English speaking countries.

The Canadian 350 contingent is tiny, just 8,000 members, but the creation date curve parallels the global account.

Our observation plan is still coming together, but for the moment we are collecting:

  • Climate Strike hashtags & keywords
  • Deniers & their mentions
  • Extinction Rebellion’s 397 accounts & mentions
  • Extinction Rebellion specific hashtags & keywords

This study is a natural progression from recent findings by @RVAwonk and @JessBots which appeared in How Maxime Bernier hijacked Canada’s #ClimateChange discussion.

Several Upgrade Efforts

It was pointed out that stating we had Six Hours Of Downtime, then not saying much, was creating the perception of an uncertain future. Not at all the case, here is what’s been happening.

The outage was triggered by a power transient and it was the first of a couple of problems we had, which triggered some introspection regarding design and implementation choices. Here we are a month later and we have a few bits of wisdom to share:

  • Elasticsearch 6.5.4 was OK, but 6.8.2. is fine.
  • Headless Debian 9.9 is much smaller than Lubuntu.
  • Do not make fixed size disks for your virtual machines.
  • Definitely do not try to run ZFS in a VM on top of ZFS. Just don’t.
  • Three VMs on a single spindle pair is a PILOT configuration.
  • Give data VMs one more core than they seem to need.
  • A brief visit to swap seems to be OK.

Taking care of all these equipment and software upgrades has left us with a system that will respond to queries against our collection of 53 million Twitter profiles in one to three seconds, instead of multiple thirty second waits before it finally does the job. While capturing fifty streams. And merging two large user profile indices.

That last point requires some explanation – we know the Lucene component of Elasticsearch is known to make heavy use of “off-heap” memory. Data VMs have been set with 8 gig of Java heap space and anywhere from 16 to 32 gig of ram. No matter what, they all “stick their toes” into their swap pools, but never more than a few megabytes worth. Normally swap access is the last gasp before things spiral out of control, but they’ve been doing this for days under intentionally demanding conditions and we haven’t been able to bowl them over.

What is driving this is a dramatic increase in volume is our rapidly evolving stream handling capability and increase in analysts using the system. We currently have the following areas of operation:

  • Legislatures – stream members and their interactions for 12 countries.
  • European MPs, U.S. Governors, and U.S. Presidential candidates.
  • Campaigns – non-election related advocacy.
  • Disinformation – things that do not appear to be legitimate.
  • Threat Monitoring – likely trouble areas get proactively captured.

Threat Monitoring is, somewhat sadly, the busiest area thanks to America’s penchant for mass shootings. Seven analysts inhabit a task oriented channel and they always have more to examine than they have time to do.

Disinformation has a strong overlap with Threat Monitoring, and whatever is on fire at the moment has been taking precedence over delving into the campaigns that create the preconditions for trouble. More hands and eyes will improve things in this area.

Campaigns & Legislatures have been purely the domain of Social Media Intelligence Unit. There are reports the go out, but they don’t see the light of day. We should probably periodically pick an area and publish the same sort of work here. In our copious spare time.

As we hinted above, we have added storage to our existing systems, but that is an interim measure. We are currently collecting hardware specifications and reviewing data center locations and rates. The cheapest is Hurricane Electric in Fremont, but the best, given what we do, might be Raging Wire in Sacramento. That’s the oldest and largest of Twitter’s four datacenters, for those not familiar with the name.

Legislatures and much of Campaigns are on their way to that datacenter. Disinformation has no public facing facet and will remain in house where we can keep an eye on it. Threat Monitoring is a mixed bag; some will stay in the office, some will involve tight integration with client systems.

Those who pay attention to the Netwar System Github will have noticed many changes over the last few days. We are approaching the time where we will declare the Elasticsearch 6.8.2 upgrade complete, and then maybe have another go at offering a Netwar System Community Edition VM. When this is ready there will be an announcement here.