Yottabytes: Storage and Disaster Recovery


February 15, 2018  11:53 PM

Lawyers Dealing With Bitcoin E-Discovery

Sharon Fisher Sharon Fisher Profile: Sharon Fisher
Bitcoin, Cryptocurrency, ediscovery

We talked a couple of months ago about the best places to store bitcoin and other cryptocurrencies and their advantages and disadvantages. It turns out there’s another nuance: Cryptocurrencies and e-discovery.

It makes sense. The whole point of cryptocurrency – at least, one of them – is to have anonymous currency that can’t be traced back to you. So, naturally, all sorts of nefarious people are finding that an advantage. “Because of the way cryptocurrency systems protect the anonymity of their participants, they have become ideal vehicles for money laundering, tax fraud and illicit purchases,” writes Eric Pesale for the Logikcull blog.

So what’s an attorney to do?

“Litigators could compel an opposing party to submit a hard drive image of the user’s cryptocurrency ‘wallet,’” write attorneys Nkosi Shields and Ryan A. Walton from the firm of BakerHostetler, in Mondaq.  “From this wallet, one could pinpoint files useful for identifying suspicious activity during the discovery process.” This is especially true if the wallet uses an open-source framework such as Bitcoin-QT, which stores the entire user’s transaction history within the app, Pesale writes. “If this is the case, a lawyer or investigator working from a hard drive image of the wallet could locate and extract files related to certain suspect transactions and cryptocurrency data activity,” he explains.

In fact, there can be traces in many devices, Pesale writes. “Any devices the user used to connect to Bitcoin’s network could contain important evidence in the devices’ volatile memory,” he writes. “Other useful evidence can be found by examining the user’s transactions in the cryptocurrency’s public blockchain ledger or by subpoenaing the user’s encrypted cryptocurrency credentials, though government attorneys trying this strategy during a criminal case could face Fifth Amendment issues similar to those involving encrypted hard drives and passwords.”

Other researchers are looking at ways to de-anonymize blockchain itself, Shields and Walton write. “By clustering different bitcoin addresses, one can assign common ownership to a user’s pseudonym(s),” they write. “International law enforcement agencies, including the Federal Bureau of Investigation, have made significant strides in piercing the anonymity of cryptocurrency transactions, leading to several notable prosecutions, including actions brought against BTC-E, a prominent virtual currency exchange believed to have been involved in international money laundering.”

Indeed, Noel Edlin writes in Law Technology Today that blockchain’s very design makes it easy for attorneys to use it in cases. “The very virtue that makes them attractive as a decentralized currency also makes attorneys leery: transactional transparency,” he writes. “There is no way to hide bitcoin transactions, because the bitcoin ledger is available to all. Every transaction conducted using bitcoins is tracked, meaning that through the internet, bitcoin transactions can be identified and monitored, although savvy users of cryptocurrencies will argue they are just as anonymous as cash.”

Pesale notes, though, that blockchain might not be admissible as evidence even if law enforcement organizations were able to compile it. “California attorney James Ching has explored the possibility that blockchain evidence could be inadmissible hearsay, falling outside of Federal Rule of Evidence 803’s exception for business records,” he explains.

Cryptocurrencies have also seen their anonymity challenged in the courts, Shields and Walton write. “In 2016, the Internal Revenue Service (IRS) issued a ‘John Doe’ summons in an effort to investigate potential investors who may have underreported or failed to report income from gains while trading cryptocurrencies.”

However, that use is controversial, Pesale writes. “These summonses, which are issued only upon receiving court approval, allow the IRS to investigate the tax liability of unidentified individual, or group of, taxpayers upon an initial finding of a tax compliance problem,” he writes. “They are often used to uncover anonymous tax shelter beneficiaries or owners of tax-exempt bonds. Although John Doe summonses are not supposed to be issued to conduct ‘fishing expeditions,’ the way the IRS has been using them in relation to cryptocurrency transactions is spurring controversy.” For example, one of its mass warrants has received a lawsuit, while the IRS also received a Sternly Worded Letter, he writes.

In any event, attorneys of all stripes need to be familiar with cryptocurrencies, writes Carolyn Elefant in Above the Law, as they’re even becoming common in divorces and wills.

February 11, 2018  5:55 PM

Too Much Bodycam Footage? Just Delete It

Sharon Fisher Sharon Fisher Profile: Sharon Fisher
government, privacy, Security

As you may recall, a number of police departments, upon implementing body cameras, have found that the cameras themselves are only the half of it. The real problem is storing all the data the cameras collect.

Some jurisdictions have found a solution to this problem: Just delete all that pesky data.

“Idaho Code requires counties to retain all records, including digital files of video and audio recordings from body worn cameras, for a minimum of two years,” notes the text of one bill under discussion in the Idaho Legislature. “Given the cost associated with management and storage of the digital media files associated with body worn cameras, many counties are currently unable to retain digital media recordings for the minimum requirement of two years. This legislation will add language to define digital recordings and a set minimum retention requirement for specified digital media files based upon the evidentiary value of the digital media file. These amendments will encourage counties to invest in digital recording devices by making the retention of county law enforcement records more achievable under county budgets.”

So, keep it if it’s evidence of a crime, but otherwise it can be discarded after 60 days, according to the proposed legislation. (That’s for bodycams. Cameras attached to a building can have their data deleted after 14 days.)

Idaho isn’t alone. In Michigan, data only needs to be kept 30 days, as of research dated last October by the National Council of State Legislatures (NCSL). Nebraska only requires 90 days, as does Miami-Dade. Washington also requires 90 days. Illinois also requires 90 days, which has led some towns to drop bodycams altogether. Missouri specifies 30 days (and Kansas City has done an exhaustive study on the storage costs of the program). Nevada can be as little as 15 days.

Short video retention periods can limit the number of people who come forward with complaints,” writes Mike Shoro in the Las Vegas Review-Journal. “A short retention period doesn’t account for people who might not be ready to file a complaint within the allotted time frame.” Short periods might also mean that footage gets deleted even though people have requested it, because of the slowness of the requesting process.

In contrast, Indiana requires 190 days for localities and 280 days for state agencies. Georgia and Oregon require it to be kept at least 180 days, as does Albany, New York.  Minnesota requires 90 days but has all sorts of exceptions. California has 60 days but again has all sorts of exceptions. New York City actually extended its retention period from six months to one year. (This is all according to the NCSL data (unless otherwise referenced), and of course things may have changed since October.) In comparison, Virginia law, for example, requires evidence to be held for at least 10 years, and sometimes 99 years.

And on the other hand, some organizations consider deletion of bodycam footage a good thing. “The Leadership Conference, together with a broad coalition of civil rights, privacy, and media rights groups, developed shared Civil Rights Principles on Body Worn Cameras,” writes the organization of its “Police Body Worn Camera Scorecard,” which on a civil rights basis prefers that law enforcement organizations delete data within six months and criticizes organizations that keep it longer than six months or don’t have a policy, out of concern that the devices will be used for community surveillance. For its part, the American Civil Liberties Union calls for recordings to be maintained for six months.


January 31, 2018  10:57 PM

Spinal Facility Loses ‘Back’-Up Hard Drive

Sharon Fisher Sharon Fisher Profile: Sharon Fisher
privacy, Security

It’s been a while since we had a good Companies Behaving Badly with people’s data story, but here we are: “Charles River Medical Associates says it lost a portable hard drive believed to contain personal information and x-ray images of everyone who received a bone density scan at its Framingham [Massachusetts] radiology lab within the past eight years,” writes Jonathan Dame in the Worcester Telegraph. “That is 9,387 people.”

What is it with medical facilities and losing data, anyway? Why are medical professionals always traipsing around with data and losing it? “Dammit, Jim, I’m a doctor, not a security professional!”

And this is in Framingham, the birthplace of IDG and Computerworld. You’d think they’d know better, through osmosis or something.

The interesting thing about this one is it isn’t someone who left a laptop in a cab or lost a thumb drive. The hard drive just turned up missing.

Oh, and it’s been missing since November – actually, maybe before that, because the data only got backed up once a month and the last time it was backed up was October — but it took them until early January to notify anybody because they were looking for it. “We determined a week and a half or so ago that … it was definitely lost,” the executive director of the clinic told Dame. “It’s hard to speculate on what could have happened to it.”

Don’t be silly. It’s easy to speculate on what could have happened to it.

  • Someone stole it for the data.
  • Someone stole it for the hardware.
  • Someone stole it for their kid.
  • Someone has a backbone fetish.
  • Someone stole it because there was data on it they didn’t want people to see, ranging from a potential case of medical malpractice to some medical condition they wanted to keep private. Didn’t you people ever watch House?
  • Someone thought it would make a good doorstop.
  • Someone accidentally damaged it and figured it would be better if it “disappeared.”

That’s just two minutes of speculation, and I was hardly trying.

Needless to say, the drive was not encrypted.

In case you’re wondering why someone needed the bone density scans of 9,387 people in the first place, apparently the disk drive was the backup, performed every month. So give them credit for that: They did backups.

(“Back” ups. Of spinal pictures. LOL.)

The good news is that, while the missing hard drive contained thousands of X-ray images of people’s spines, it did not have insurance information or Social Security numbers, Dame writes, quoting the letter that the facility was required to write to the U.S. Department of Health and Human Services, as well as to local media.

In the letter, Charles River Medical Associates warned patients to take precautionary steps “to guard against any potential negative impact from this unfortunate incident,” including monitoring credit reports.

How someone was going to get into someone’s credit account by waving an X-ray of a spine around, the facility didn’t say. Biometrics are big these days, but one usually hears about retinal scans or fingerprints rather than backbone pictures. Better safe than sorry, I suppose.

The company assures us that it will no longer use unencrypted portable storage devices to store medical records, and it’s “undertaking a broader review of its security protocols,” Dame writes. Perhaps they can find an IDG person to advise them. In the future, while it’s commendable that the organization does backups, it might want to think about backing the data up to the cloud, where it can’t go on walkabout. And, maybe, encrypt it?


January 30, 2018  12:04 PM

Strava Fitbit Map Shows Power of Metadata

Sharon Fisher Sharon Fisher Profile: Sharon Fisher
FItbit, map, Metadata

We’ve written before about fitness trackers such as Fitbits and the potential interesting challenges they create for electronic discovery. But here’s a new one.

Strava, the company that produces a variety of these devices, published a map aggregating the data of its users over a period of two years, showing just how widespread the devices are. Awesome. Maps are cool. The problem is, in certain areas, those maps also did a swell job of delineating soldier movements due to military personnel wearing those devices.

“In war zones and deserts in countries such as Iraq and Syria, the heat map becomes almost entirely dark — except for scattered pinpricks of activity,” writes Liz Sly in the Washington Post. “Zooming in on those areas brings into focus the locations and outlines of known U.S. military bases, as well as of other unknown and potentially sensitive sites — presumably because American soldiers and other personnel are using fitness trackers as they move around.”

Needless to say, the military is having kittens. “The U.S.-led coalition against the Islamic State said on Monday it is revising its guidelines on the use of all wireless and technological devices on military facilities as a result of the revelations,” Sly writes. “The existing rules on the privacy settings to be applied to devices such as fitness trackers are being ‘refined’ and commanders at bases are being urged to enforce existing rules governing their use, according to a statement from the Central Command press office in Kuwait.”

You think?

The company said it was reviewing its policies and reminding people how to set security and so on with their devices.

Incidentally, the map is still up. Not only that, but the letter from CEO James Quarles wringing his hands about how seriously Strava is taking the situation includes a link to the map.

This was all discovered by a 20-year-old Australian student on summer vacation who was playing with maps and who Tweeted out his discoveries. After he gave them the idea, a number of other people checked out other interesting locations around the globe. The kid has probably also ensured his future success. “His discovery would not hurt his career prospects,” the New York Times understated.

As Sly points out, it’s not like locals don’t know where military bases are. Even if the vast expanses of space surrounded by barbed wire and people with guns didn’t clue you in, after all, there’s always Google Earth. The problem with the Fitbit map is that it also shows the routes that the military personnel take within the base, as well as outside it. As they say, if you don’t want people to know where you are, then you need to change your routes periodically.

Security considerations aside, this is a beautiful example of the power of metadata. Remember when various government organizations have requested metadata about email and messages, and justify it by saying, oh, it’s okay because we’re not asking for personal information, and people say, no, actually, it’s still a problem? This is what those people meant. Do we know data about individual soldiers? No. Do we know where they are in real time? No. But we can still see patterns in the data, and just those patterns are enough to provide a great deal of information about troop movements as a whole, even if it’s not identifiable down to the individual soldier.

“Lines of activity extending out of bases and back may indicate patrol routes,” Sly writes. “The map of Afghanistan appears as a spider web of lines connecting bases, showing supply routes, as does northeast Syria, where the United States maintains a network of mostly unpublicized bases. Concentrations of light inside a base may indicate where troops live, eat or work, suggesting possible targets for enemies.”

And in the bravo-for-life’s-little-ironies department, in many cases the devices were actually apparently given to soldiers by the military. In the British military, overweight soldiers were given Fitbits in 2016, while on the U.S. side, the Pentagon has encouraged the use of Fitbits among military personnel and in 2013 distributed 2,500 of them as part of a pilot program to battle obesity, Sly writes. (And in a really ironic moment, check out this April Fool’s Day press release from the U.S. Army from just last year.)


January 24, 2018  9:05 PM

Microsoft-Ireland Data Case Brings Its Friends

Sharon Fisher Sharon Fisher Profile: Sharon Fisher
Microsoft, Security

The wheels of justice continue to grind in the Microsoft Ireland data servers case, with the simultaneous submission of 23 amicus briefs signed by almost 300 people worldwide to the Supreme Court, which is expected to hear the case next month.

As you may recall, the case, which started in 2014,  involved whether Microsoft must release data stored on one of its servers to a U.S. government agency, even though the data in question is outside the U.S. In January 2017, the Second Circuit Court of Appeals denied a rehearing of the case, which left the Supreme Court as the only option. At the very last minute – and after two extensions – the Department of Justice decided in June to go for it, and in October the Supreme Court agreed to hear the case. (Here’s a good description of it.)

The justices will hear oral arguments in late February, but in the meantime, a whole lot of people from a whole lot of countries, a number of Microsoft’s competitors, a slew of advocacy organizations, and a heap of computer science professors have lawyered up and sent in amicus briefs. Fortunately a lot of them worked together so the Supremes won’t have to read 300 separate briefs.

As Microsoft had suggested last June, several of the briefs from European Union (EU) countries referenced the General Data Protection Regulation (GPDR), a new law governing this issue that is scheduled to take effect in Europe in May. In fact, one of them was submitted on behalf of the guy who was responsible for the GPDR (who is, actually, on Twitter and is discussing the case there).

“In one of many amicus briefs filed Thursday on behalf of Microsoft, attorneys at White & Case wrote for European Parliament members, including Jan Philipp Albrecht, and former EU Justice Commissioner Viviane Reding,” writes Ben Hancock in The Recorder. “Albrecht helped shepherd the GDPR in the European Parliament to its ultimate adoption in 2016, and has been outspoken on digital privacy issues. He is the vice chairman of the European Parliament’s Committee on Civil Liberties, Justice and Home Affairs, and also sits on the legislature’s Special Committee on Terrorism.” A total of 10 EU members signed that brief.

Ultimately, the solution is not the Supreme Court or any other court, but rewriting the Electronic Communications Privacy Act (ECPA) to better reflect the realities of this century, say advocates. “The blame should be placed not either party to this case, but rather on the outdated ECPA in clear need of reform,” writes Casey Given in the Washington Examiner. “Congress must act to clarify the rules of the road with regard to consumer privacy and government powers in the age of the internet.”

In particular, Given calls for a rewrite of the ECPA called the International Communications Privacy Act, which would let law enforcement request a warrant for content on remote servers and give the foreign government at play the chance to object should the warrant violate their privacy laws, he writes.


January 10, 2018  12:33 AM

U.S. Tightens Laptop Border Search Restrictions

Sharon Fisher Sharon Fisher Profile: Sharon Fisher
government, laptop, privacy, Security

Taking a laptop across the border? There’s good news and bad news.

As you may recall, since 2009 there has been an ongoing struggle with how much right the U.S. government has to search laptops that people are carrying into the U.S., without a warrant, or even any particular reason other than that they feel like it. Plus, “border” has actually been defined to mean “within 100 miles of the border,” which literally covers a lot of territory.

Now, the agency has released an updated written directive that clarifies how passwords and cloud data should be handled, according to the Associated Press. “The new rules make clear that agents are only allowed to inspect information physically present on a device — and not information stored remotely, such as on the cloud,” the AP writes. “To prevent officers from accessing information they shouldn’t, they are now required to request that travelers turn off their devices’ network connectivity, or disable it themselves.” Passwords provided to Customs and Border Protection (CBP) must be deleted or destroyed immediately following a search.

The department also defined two levels of search.

  • Basic searches: officers can look through passengers’ contacts, photos and other material without reasonable suspicion of criminal behavior
  • Advanced searches: devices are connected to external equipment so their contents can be reviewed or stored, which requires the approval of a supervisor and “reasonable suspicion of activity in violation” of the law or “a national security concern,” such as a person’s presence on a terror watch list

On the other hand, “if someone refuses to unlock a device, the device can be detained by CBP,” writes Geneva Sands for ABC News. “U.S. citizens will always be allowed to enter the U.S., but their phones could be held back — generally for no more than five days,” though people have reported having their devices seized for up to seven months. “For non-citizens, refusal to open a device could lead to denied entry. If incriminating information is found, CBP officers could refer the case to an investigative agency, like the FBI, or for non-citizens, deny them entry into the U.S.”

“The New Directive does not specifically allege that travellers have a positive obligation to provide a passcode or other means of access to USCBP during a border search; it merely states that USCBP officers may request access and then detain the device for further examination if the traveller does not provide it,” writes Henry Chang of Blaney McMurtry LLP in Lexology. “This is likely because the law is still not clear regarding whether travellers actually have a legal obligation to provide passcodes or other means of access during a border search.” In addition, “nothing precludes USCBP from detaining an electronic device for a much longer period by alleging that ‘extenuating circumstances’ exist,” he adds. “The threat of having their electronic device seized, even temporarily, could compel some travellers to cooperate. The New Directive also does not address the issue of how long USCBP may delay the entry of a traveller in connection with the search of their electronic devices. The threat of an extended delay, which may cause the traveller to miss their flight, could also compel some travellers to cooperate.”

What brought this all on? The ACLU filed a lawsuit last year against the Department of Homeland Security on behalf of 11 travelers whose smartphones and laptops were searched at the border, the AP writes. Other organizations, such as the Electronic Frontier Foundation (EFF), have been pushing for a test case that would extend the Riley decision – which requires law enforcement officials to have a warrant to search someone’s cell phone — to laptops at the border. In December, the U.S. government filed a motion to dismiss the case. The ACLU is preparing its response for late January, according to Bart Jansen in USA Today.

“It is positive that CBP’s policy would at least require officers to have some level of suspicion before copying and using electronic methods to search a traveler’s electronic device,” said Neema Singh Guliani, legislative counsel at the ACLU. “However, this policy still falls far short of what the Constitution requires — a search warrant based on probable cause. The policy would still enable officers at the border to manually sift through a traveler’s photos, emails, documents, and other information stored on a device without individualized suspicion of any kind. Additionally, it fails to make clear that travelers should not be under any obligation to provide passcodes or other assistance to officers seeking to access their private information. Congress should continue to press CBP to improve its policy.”

A bill is actually under consideration  to require a warrant to search the devices of Americans at the border, write Charlie Savage and Ron Nixon in the New York Times, but it is unlikely to pass in this political climate, they add.

In other news, CBP announced some numbers. Its border agents inspected 30,200 phones and other electronic devices in fiscal year 2017, which ended in September — a nearly 60 percent spike from 2016, when 19,051 devices were searched,” according to the AP. It’s even a bigger spike from previous years: the New York Times reported that  inspection of electronic devices rose from 4764 in 2015 to 23,000 in 2016, while according to the Los Angeles Times, just 8,053 travelers had their devices searched in the 2015 fiscal year. That’s compared with 6,500 between 2008 and 2010, according to the American Civil Liberties Union. U.S. border agents also searched the electronic devices of 59 percent more international travelers in fiscal year 2017 than the previous year, including U.S. citizens and foreigners, adds the Los Angeles Times. Nonetheless, the agency said the searches represented just a tiny fraction — 0.007 percent of arriving international travelers — out of more than 397 million, the AP wrote.

(Yes, I’m aware some of the numbers don’t match up. Perhaps they’ve changed over time? Or some are fiscal year, which ends on September 30, and some are calendar year? Either way, lotta devices.)

In particular, this is a concern to attorneys, who worry about violations of attorney-client privilege. The New Directive does have some additional guidelines in this area, Chang writes.


December 31, 2017  11:29 PM

Rhode Island Email From 2003 Might be Missing

Sharon Fisher Sharon Fisher Profile: Sharon Fisher
Email, government, Retention

You could call it Schrodinger’s Email System: Email messages aren’t lost; there just isn’t any way to gain access to them.

That’s the situation the state of Rhode Island has been facing since 2015, which came to light earlier this month due to a public records request. Before then, state agencies had been using a combination of Novell Groupwise and Microsoft Exchange email servers, and decided to consolidate them into a single Office 365 email system.

A laudable goal, in general. The problem is that Microsoft and Novell never did play very well together, and by 2014 Novell was essentially not around anymore, with Micro Focus buying Groupwise. (In fact, you can still buy it today.) The upshot is that the vendor working with Rhode Island to help migrate the email systems – Microsoft Consulting Services – warned the state that it could lose up to 5 percent of the email messages in the process, according to an article in the Providence Journal, which broke the story.

Hence the Schrodinger’s email nature of the new system. “Department of Administration spokeswoman Brenna McCabe told The Journal, ‘We did not lose the emails,’” writes Katherine Gregg. “But she acknowledged, ‘They are [now] in a format that is not easily searchable … [And] it would take significant resources to put the data in an accessible, searchable format.’” So if you can’t search or gain access to the email messages, does it matter whether they’re “lost” or not? What would be different if they were considered “lost”?

The situation came to light when software entrepreneur and two-time candidate for governor Ken Block filed a public records request. He was reportedly performing a computer analysis of the potential for voter fraud in Rhode Island for a nonprofit organization co-founded by President Donald Trump’s former chief strategist Stephen Bannon, Gregg writes. “As part of his continuing inquiry, Block asked the elections board for communications — dating back to Jan. 1, 2003 — from the state to local boards of canvassers about ‘voter registration … voter identification policies and processes.’”

That was when Block learned about the inaccessible email messages, which he presumably passed on to the Journal. “There were significant problems with the syncing process, as the two systems are wholly incompatible with each other,” Richard Thornton, the campaign-finance director for the state Board of Elections, told Block. “There may, or may not be, additional email communications responsive to your request, but for which the State of RI has no capacity to retrieve presently.”

This is not to unduly pick on Rhode Island. Back in the day, people used proprietary email systems because that’s what there was, and different agencies used different email systems because that’s how they were typically acquired. While the state happened to bet on the wrong horse, technologically speaking, it hadn’t made a bad choice in picking Groupwise. If anything, one could criticize the state for not upgrading sooner, but there was a recession in there as well. Chances are, a number of other states are in the same pickle. One could also criticize the state for not having reduced the size of its email system before the migration, but again that’s hardly a problem confined to Rhode Island.

Plus Block appears to be going off the deep end about it a little bit. The state has not yet responded with how many email messages are missing – if it even can – but it isn’t at all clear that “most” email messages before 2015 are unrecoverable, as he told the Journal. He also complained about the cost of retrieving the email messages, but to be fair he’s asking for a large amount of email going back fifteen years on what could be little more than a fishing expedition in that the vast majority of studies have found that voter fraud is a minuscule problem.

Similarly, Block criticized Thornton for saying the state had moved to the new system “for more efficient backup of data, a more standardized approach for records retention and a secure disaster recovery solution” in light of the problem. “”Wow. How does that square with Thornton’s email?” he told the Journal. But obviously it was because of just this sort of situation that the state did make the move to a single email vendor.

Realistically, how many people can lay their hands on email that they sent in 2003? My Gmail account goes back to April 7, 2004, and I was one of the early adopters. While I may have .pst files around from Outlook systems I was using before then, what are the chances I could actually find a way to read them?

It’s an example of the sort of problem typically referred to as the “digital dark ages” brought on as organizations – particularly governments – went digital. But due to the proprietary nature of the hardware and software people used, plus technology’s inexorable march on, much of this old data may no longer be readable by future generations.

This is why organizations are encouraged to migrate their archives to up-to-date hardware and software every couple of years, to ensure the data can still be read going forward.


December 30, 2017  11:20 PM

Library of Congress Twitter Collection to Stop

Sharon Fisher Sharon Fisher Profile: Sharon Fisher
social media, Storage, twitter

Ever decided you were going to save all your data until you found out how much space it was going to take up, and then change your mind? The Library of Congress is in that position, which is why it recently announced it is no longer going to save every Tweet.

The Library had started saving all the Tweets in 2010, retroactive to the beginning of Twitter in 2006, but as of January, that will stop. While the Library is going to continue to keep the Tweets it already has, it will save Tweets only on a selective basis.

“The technical infrastructure for the Library’s Twitter archive follows the same general practices for monitoring and managing other digital collection data at the Library,” the Library wrote in January 2013. “Tape archives are the Library’s standard for preservation and long-term storage. Files are copied to two tape archives in geographically different locations as a preservation and security measure. The volume of tweets the Library receives each day has grown from 140 million beginning in February, 2011 to nearly half a billion tweets each day as of October, 2012.

In addition, “Since 2000, the library has been collecting pages from websites that document government information and activity,” writes Doug Criss for CNN. “Today, that archive is more than 300 terabytes in size and represents tens of thousands of different sites. The library’s entire collection of printed books has been estimated to total about 10 terabytes of data.”

Actually, there’s more to it than a simple matter of storage – it’s a matter of being able to find the information afterwards. The Library had said when it started saving the Tweets that it would develop a system by which people could search them, and that hasn’t happened yet, nor is it clear when such a search functionality will be available. In fact, the Library of Congress said public access to the archive would be blocked until it could  figure out “a cost-effective and sustainable” way to let people view and use it, Criss writes.

“Six years after the announcement, the Library of Congress still hasn’t launched the heralded tweet archive, and it doesn’t know when it will,” Andrew McGill wrote in the Atlantic in August, 2016 – indeed, predicting that the Library might eventually cut the project off.  “No engineers are permanently assigned to the project. So, for now, staff regularly dump unprocessed tweets into a server—the digital equivalent of throwing a bunch of paperclipped manuscripts into a chest and giving it a good shake. There’s certainly no way to search through all that they’ve collected.”

In fact, some are dubious that the Library’s decision has anything to do with storage at all, such as Kalev Leetaru in Forbes. “Given that the Library’s collection is, in its current form, essentially a dark archive saved to cold storage, enhanced compression and no query access mean the actual storage costs are minimal,” he writes. “By 2013 the Library’s Twitter archive totaled just 133 terabytes ‘for two compressed copies.’ Even assuming a massive growth rate, a full petabyte of data stored securely with “99.999999999% durability” (9 nine’s), costs just around $7,300 a month for immediate access or as low as $4,000 a month for batch access (5-12 hour access delay) in today’s modern commercial cloud. If the Library just needed to store the Twitter firehose securely and durably without any kind of user access (its current model), price and technical capability would not seem to be a limiting factor.”

Similarly, Leetaru continues, if the Library thinks text-only Tweets are no longer useful, why not just find a way to save the attached imagery? “It is unclear why the Library has concluded that it should simply abandon archiving Twitter, rather than making the argument that if a lack of multimedia content and links is the problem, then perhaps it should add multimedia and link archiving to its preservation pipeline or partner with organizations like the Internet Archive to preserve this content,” he writes. “After all, the Library has evolved countless times over the last two centuries as technology has changed.”

The timing on all this is actually interesting given how important Twitter is in international politics right now, such as the #MeToo movement being named the Time “Person of the Year.” “At a time when an increasing number of world leaders are taking to Twitter to discuss major policy issues, make formal statements and engage with their citizens at the same time we are talking about Russian influence operations, terrorist recruiting, trolls and harassment on social media, it might seem that this is absolutely the wrong time to suddenly announce with no warning that the Library of Congress will no longer be archiving the full Twitter firehose,” Leetaru writes.

“Generally, the tweets collected and archived will be thematic and event-based, including events such as elections, or themes of ongoing national interest, e.g. public policy,” the library said in a statement. But Leetaru criticized this sentiment. “One of the most basic problems with hand selecting specific topics over time is that it is often unclear at any given moment what will be important in the future,” he writes. “Not only does this mean that myriad society-influencing topics will not be archived at all, but even those topics that are archived will not be included until long after they have begun trending, meaning the early formative conversations around issues like future #metoo movements will be lost forever.”

“That’s a huge blow to the ability of Americans to hold politicians and companies accountable,” agrees Kara Alaimo in CNN. “In particular, social media has become more important to our politics than ever before, so we need more tools to hold politicians responsible for their behavior on social media — not less. But it’s precisely because Twitter has become such a big part of our national conversation that we need the tools to monitor it. If additional funding is necessary for the Library of Congress to be able to maintain a complete record, including visual and deleted tweets, Congress should provide it.”

Not everyone disagrees with the Library’s decision. “Its goal has never been to archive the web as a whole, only to preserve portions of it,” writes Jacob Brogan in Slate. “With that in mind, continuing to archive all of Twitter as such seems largely unnecessary, and possibly even counterproductive if future scholars really do want to look into the platform’s rise.”


December 20, 2017  10:59 PM

Text-Destroying App ‘Confide’ Used in Missouri by Politicians

Sharon Fisher Sharon Fisher Profile: Sharon Fisher
government, privacy, secrecy, TEXT

As you may recall, in 2014 I wrote about an app called Confide, billed as “Snapchat for business.”  Intended to send messages secretly, it didn’t allow people to read over your shoulder or let you take a screenshot, and deleted the messages after they’re read. Moreover, the company used end-to-end encryption, meaning it couldn’t read the messages, either, and the messages were never stored on the company’s servers. There was some handwringing at the time about what would happen if it got into the hands of politicians (though most of the attention appeared to be on what it could mean for infidelity).

Surprisingly, it took three years, but some politicians in Missouri recently got nailed for using it. Earlier this month, the Kansas City Star reported that Governor Eric Greitens and several staff members had the application on their personal cellphones. The paper wasn’t able to prove that the staff members were using it to conduct government business – kind of by definition — but it’s created a lot of attention in the appropriately named Show-Me State.

“In addition to Greitens — whose Confide account is under the name ‘Er Robert’ — the governor’s chief of staff, deputy chief of staff, legislative director, press secretary, policy adviser, director of cabinet affairs and several other senior staff members have Confide accounts connected to their personal cell phones,” the Star reports.  It did not say how it found this out.

“The fact that senior staff are using it as well hints that the public’s business is what’s at stake here, and if the governor and top officials are using this for public business, they are subverting the Missouri Sunshine Law,” writes the Joplin Globe in an editorial.

“For public servants, text messages constitute government communications,” writes the St. Louis Post-Dispatch in an editorial. “They’re a big deal.”

One state senator called for attorney general Josh Hawley – who is also running for the Senate — to investigate the situation, but at first he said he couldn’t because he is already defending the governor in several other legal cases, but he might appoint a special prosecutor, according to the Post-Dispatch. However, he has since said he will open an investigation.

Use of the application could violate the open records laws of Missouri, Hawley tells the Post-Dispatch.

Greitens spokesman Parker Briden had told The Star, “I don’t believe anyone has (Confide) downloaded on a state-issued device,” reports the AP. And that may well be true, but it evades the issue. The thing is, public officials have a long history of conducting public business on personal devices, either accidentally, because they thought it was more secure, or potentially to evade public records laws. So even if it’s not on a state-issued device, it could still be a problem.

Greitens blamed the liberal media for being “desperate for salacious headlines,” writes the Associated Press, and to judge by some of the comments on the articles, some Missouri voters agree with him.

Meanwhile, Hawley is suggesting that the state update its Sunshine laws to more explicitly address text messages.

Another nuance in the story is that both Hawley and Greitens are Republicans, while the state senator who called for investigation is a Democrat. Some also believe that Greitens intends to run for President in the future.

Ironically, one effect of the story has been to encourage many other Missouri politicians to sign up for the app, the Star reports. “Rep. Robert Cornejo, a St. Charles County Republican who already had a Confide account, posted a link to The Star’s story on Twitter, noting: ‘I get a notification every time one of my ‘contacts’ joins Confide. This story explains why my phone has been buzzing all morning as people (from both sides of the aisle) join.’”


December 13, 2017  3:50 PM

E-Discovery and the Alabama Recount

Sharon Fisher Sharon Fisher Profile: Sharon Fisher
ediscovery, Electronic discovery, government

If nothing else, Tuesday’s special election for Senator in Alabama is a fascinating case of electronic discovery.

The day before the election, “Montgomery County Circuit Judge Roman Ashley Shaul granted a preliminary injunction directing counties to set voting machines ‘to preserve all digital ballot images,’” writes Mary Papenfuss in the Huffington Post. “The order was requested in a lawsuit filed last week on behalf of Alabama citizens demanding that voting records be protected.” The judge ordered ballot images to be saved for six months.

Plaintiffs said they wanted images to be preserved because they felt that the ballot design was confusing (shades of Florida 2000) and they wanted to make it easier to do a recount if it were necessary, Papenfuss writes. Moreover, state and federal law requires the images to be saved, writes Connor Sheets in AL.com.

However, later that day, Alabama’s state Supreme Court stayed that order, saying that Alabama Secretary of State John Merrill and state administrator of elections Ed Packard, “do not have authority to maintain such records or to require local officials to do so,” Sheets writes in a different AL.com article.

“The court will hold a hearing on Dec. 21 about whether to dismiss the case outright,” Sheets writes. “By that point the state will have had ample time to destroy the digital ballot images legally under the stay.”

Interestingly, the state doesn’t count the actual ballots, but the images of the ballots, Sheets writes. And it’s actually less a matter of “destroying” the records than whether election officials would push the button on each machine to direct it to save the images in the first place, though certainly any images that did get saved could get destroyed.

Some of the machines include a switch to either save all, destroy, or save only write-in ballots, but not all the machines have the switch, according to Andrew Yawn in the Montgomery Advertiser. Alabama saved only images of the write-in ballots. Alabama Attorney General Steve Marshall wrote in a press release that “To change them, as the plaintiffs seek, would not mean simply flipping a switch, but would require the third-party vendor, Elections Systems and Software, to travel to 2,000 voting machines around Alabama to change them. This process could not be completed in a day. To attempt it the day before and day of the election would cause chaos, confusion, and delay,”

(One wonders, what is the default? And why isn’t the default simply set to be saving the images in the first place?)

This is all very interesting in light of Judge Roy Moore’s contention that he wants to have a recount of the ballots after his loss. What would there be to count? If nothing else, articles about the subject showed there’s a lot of confusion.

There are still the paper ballots, which are preserved for 22 months, according to Yawn. But because, by law, only the digital images are counted, they would presumably have to be rerun through the machines, which could end up with different results. Marshall said in his statement that that was the procedure. On the other hand, “Alabama law does not provide for such manual recounts, only a machine recount of the digital images that are taken at the time each ballot is cast,” writes Andrew Gumbel in the Guardian. “If those images are then destroyed, there is no easy way to verify that they were read and counted correctly.”

Incidentally, an automatic recount only happens if there is a .5 percent difference or less. The difference is actually about 20,000 votes, or around 1.5 percent, according to the Associated Press. “The state canvassing board will declare whether an automatic recount is needed, when it meets sometime between Dec. 26 and Jan. 3. The recount would begin within 72 hours of that decision.”

States that have given up paper ballots altogether in favor of electronic ones have varying requirements, writes Sean Steinberg in WhoWhatWhy. “Plenty of other states — including Florida, Michigan, and Wisconsin — already release their ballot images to the public upon request. Some jurisdictions, like Dane County, WI, go even further and post their ballot images online,” he writes. “Colorado has ruled in favor of keeping the ballot images, as has Arizona. While Colorado also designated ballot image files as public record, Arizona ultimately decided against making them publicly available.”

The case sets an important precedent for what could happen in the 2018 Congressional elections, writes Steven Rosenfeld in AlterNet, which are also expected to be closely contended.


Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: