Category Archives: News

Data Transfer Project: New models for interoperability

In spite of improved file standardization, interoperability of data is often a challenge. Say you’ve got a collection of pictures on Photobucket and you want to move them to a different site. You’ve got a lot of manual work ahead. It would be great if there were a tool to do it all for you. The Data Transfer Project aims at making that possible. Some big names are behind it: Facebook, Google, Microsoft, and Twitter. The basic approach is straightforward:

The DTP is powered by an ecosystem of adapters (Adapters) that convert a range of proprietary formats into a small number of canonical formats (Data Models) useful for transferring data. This allows data transfer between any two providers using the provider’s existing authorization mechanism, and allows each provider to maintain control over the security of their service.

Continue reading

DNA as data storage

What’s the oldest data format in the world? It’s not any of the ones that computer engineers developed in the 20th century, or even ones that telegraph engineers created in the 19th. Far older than those — by billions of years — is the DNA nucleotide sequence. We can think of it as a simple base-4 encoding of arbitrary length.

DNA double helix According to the usual, somewhat simplified, description, a DNA molecule is a double helix, with its backbone made of phosphates and sugars, and four types of nucleotides forming the sequence. They are adenine, guanine, thymine, and cytosine, or A, G, T, and C for short. They’re always found in pairs connecting the two strands of the helix. Adenine and thymine connect together, as do guanine and cytosine.

DNA for data encoding

That’s as deep as I care to go, since biochemistry is far away from my areas of expertise. What DNA does is fantastically complicated; change of few bits and you can get a human or a chimpanzee. But as a data model, it’s fantastically simple.
Continue reading

When sloppy redaction fails, resort to censorship

The Broward County School Board used an idiot’s form of “redaction” on a PDF before sending it to the media. The Sun Sentinel removed the blackout layer from the file and found newsworthy information on shooter Nikolas Cruz. They published it. Judge Elizabeth Scherer flew into a rage. She decreed, “From now on if I have to specifically write word for word exactly what you are and are not permitted to print… then I’ll do that.”

That’s called prior restraint, or censorship.
Continue reading

USD and USDZ format for 3D models

Pixar’s USD format allows representation of dynamic 3D scenes. It lets designers create large numbers of objects that fit together into a scene. People on a team can work independently of each other, each designing certain parts. The project is on GitHub.

USD’s design solves the problem of not having to work with one monolithic file (as Pixar did for Toy Story), but sometimes a monolithic file is useful. At WWDC 2018, Apple and Pixar announced a new wrapper for USD, called USDZ. It’s a Zip archive with some special rules. iOS 12 will support it.
Continue reading

The “Zip Slip” vulnerability

Sometimes my reaction to a story is “Wait, are they saying someone was that dumb? … No one could be that dumb! … Oh, gods, they were that dumb!” Naked Security’s account of the Zip Slip vulnerability is just such a story.

The article starts with a fair warning that the vulnerability is “so simple you’ll need to put a cushion on your desk before you read any further (in case of involuntary headdesk injury).” It explains that because of the coding mistake called “Zip Slip,” “attackers can create Zip archives that use path traversal to overwrite important files on affected systems, either destroying them or replacing them with malicious alternatives.” This is where I started to suspect.

The vulnerability isn’t in the Zip format as such, but in bad coding found in some of the zillion ad hoc pieces of software written to unpack Zip files. Have you figured it out yet? I’ll put the cut here to give you a chance to think…
Continue reading

Flash in the Library of Congress’s online archives

Everybody recognizes that Adobe Flash is on the way out. It takes effort to convert existing websites, though, and some sites aren’t maintained, so it won’t disappear from the Web in the next few decades.

When it’s minor or abandoned sites, it doesn’t matter so much, but even the Library of Congress has the issue. Its National Jukebox currently requires a browser with Flash enabled to be useful. Turning on Flash for reliable sites such as the Library of Congress should be safe, at least as long as those sites don’t include third-party ads from dubious sources. Not everyone has that option, though. If you’re using iOS, you’re stuck.

I came across the National Jukebox while doing research for my book project Yesterday’s Songs Transformed, and it’s frustrating that I can’t currently use it without taking steps which I’d rather avoid. The good news is that this is a temporary situation and work is already underway to eliminate the Flash dependency. David Sager of the National Jukebox Team replied to my email inquiry:
Continue reading

File corruption and political corruption

When people who don’t understand file formats manipulate files in order to cover their tracks, they generally fail miserably. Slate magazine gives an entertaining case in point from the Trump scandals. The article says:

There are two types of people in this world: those who know how to convert PDFs into Word documents and those who are indicted for money laundering. Former Trump campaign chairman Paul Manafort is the second kind of person.

The PDF Association chimes in with additional technical details.
Continue reading

Aside

The Libtiff source code repository is now on Gitlab. The old CVS repository on maptools.org will be maintained for historical purposes but won’t get any updates. One reason for choosing Gitlab rather than Github is that there’s already a libtiff … Continue reading

International Digital Preservation Day

Today is International Digital Preservation Day.

Files that Last coverIn honor of the day, I’m offering Files that Last: Digital Preservation for Everygeek on Smashwords at its lowest price ever. Today only, you can get it for $0.99 with the coupon code
AM26N. This is a one-day sale, so get it now if you don’t already have it!

There are new releases of VeraPDF and JHOVE today.

Libtiff 4.0.9 released

Libtiff 4.0.9 has been released. According to the email announcing it:

A great many security improvements have been implemented by Even Rouault.

Much thanks to OSS Fuzz, team OWL337, Roger Leigh, and of course Even Rouault.

Obligatory reminder: Don’t download from libtiff dot org. It’s many years out of date.