Secure Encryption & Decryption

What Business Problem Does It Solve?

With the arrival of the European Union's General Data Protection Regulation (GDPR), people have the right to ask any organisation for a copy of all the data held about them. Organisations are mostly not permitted to charge any fee for providing this information to anyone who asks. As a result, a lot of people are asking for this information; within your organisation this process needs to be streamlined and optimised to reduce costs.

The information supplied to these individuals is "personal information" and so it should be encrypted so that only the correct individual has access to it and no one else, not even your organisation's own IT staff.

Of course using encryption with ZendTo has always been possible, by using an external utility such as 7-Zip to encrypt the files before uploading them. However, doing that requires the recipients to be able to use 7-Zip, and install it if they haven't used it before. When sending encrypted information to members of the public, you do not want your Legal team sitting on the phone doing IT tech support for people who don't know how to open the 7-zipped file you just sent them!

As a direct result of this, there are suddenly a huge number of organisations who need to be able to easily and efficiently encrypt files before sending them. The recipient's experience must also be as simple and robust as possible.

The User Experience

I have worked to make this as simple and unobtrusive as possible.
Note: there are settings in preferences.php which allow organisations to adjust the default behaviour described below.

Sending Files
In the "new drop-off" form, there is 1 new checkbox labelled "Encrypt every file". Clicking that checkbox causes the user to be prompted to enter and confirm a passphrase.
Receiving Files
The recipient is presented with the page of download links as normal. If the drop-off has been encrypted, then their first attempt at downloading a file will cause a prompt to enter the passphrase. If the passphrase does not decrypt the file, an error message tells them and they try again. If the passphrase was correct, the decrypted file (and any other decrypted files they download from that drop-off page) is downloaded as normal without further prompts.
Sending a Request for some Files
In the "Request a Drop-off" form, there is 1 new checkbox labelled "Encrypt every file". Clicking that checkbox causes the user to be prompted to enter and confirm a passphrase. The person sending you the files has exactly the same user experience as they would if there was no encryption involved. They are unaware that any encryption is being applied to their uploaded files.

Implementation

The encryption and decryption functions are implemented using "libsodium", an easier-to-use fork of Dan Bernstein's "NaCl" (pronounced "salt") library. Dan's NaCl web site contains links to many reviews and comparisons with other such libraries. NaCl is designed to be highly secure, while also being significantly faster than other libraries.

Sodium is a standard part of PHP 7.2 onwards, and is also available as an installable "pecl" extension for PHP 7.0 and 7.1. The ZendTo Installer installs and configures this as necessary. PHP 7.2 is used wherever possible to ease installation and future updating.

Dan Bernstein is exactly the right person to produce work such as NaCl. I have worked on his designs and code before, most notably "qmail" when I was writing MailScanner (a previous project of mine). I mean this entirely as a compliment to Dan: he is the most fussy, perfectionist designer and developer you will ever find. He is incredibly careful in his design decisions and implementations. Who else would write an email engine which will happily process messages with "Subject:" lines longer than 4.3 billion characters?

Configuration Controls

There are a few configuration settings in preferences.php to adjust the behaviour of the encryption / decryption facility:

Disable encryption altogether
Set 'maxBytesForEncryption' => 0
Make encryption compulsory
Set 'enforceEncryption' => TRUE
Limit the maximum size of drop-offs that can be encrypted
Set 'maxBytesForEncryption' to that number of bytes
Enforce a minimum passphrase length
Set 'minPassphraseLength' to that number of characters
Enforce the passphrase must contain digits, punctuation, hieroglyphs and emoji
This setting does not exist as it results in passwords that are just as easy for computers to break, but far harder for humans to remember. Use a long passphrase, not a shorter but awkward password.

What Happens Internally

Creating an Encrypted Drop-off

After the user has ticked the checkbox in the "new drop-off" form and has entered their chosen passphrase, the passphrase is sent over https to the ZendTo server along with their files.
All encryption and decryption is done on the ZendTo server itself, none of it happens in the user's web browser.

Immediately the files have reached the server and have been virus-scanned and optionally checksummed,

  1. ZendTo creates a secure key derived from the user's passphrase. This has far more entropy than the passphrase contains, so is much better as an encryption key.
  2. ZendTo creates some extra (very) random bytes of data as the "initialisation vector" (IV) for the encryption algorithm.
  3. ZendTo then encrypts the file, 64 KBytes at a time so it does not need as much RAM as the size of the file, using the key and the IV.
  4. ZendTo replaces the unencrypted file with the encrypted one.
  5. ZendTo stores the IV in its database.
  6. ZendTo wipes the memory that is storing the passphrase, the key, the IV and any data derived from them.

At this point, the only copy of the user's files are encrypted. Absolutely no-one can decrypt the files without knowing the passphrase used to generate the key, and the IV. It is assumed that any hackers have thoroughly read all the source code of ZendTo, there is nothing hidden. Not even your own organisation's IT staff can decrypt the files, nor provide any hints as to how it might be done. The IV data is not considered to be secret.

Picking Up an Encrypted Drop-off

When a recipient comes to download the files, they are presented with exactly the same download page as normal. When they first click on a link (or the "Download All Files" button), they are now prompted to enter the passphrase if the drop-off was encrypted.

Immediately after they enter the passphrase,

  1. That passphrase is sent over https to the ZendTo server. This is done using "POST" instead of "GET" so it doesn't appear in the user's web browser history.
  2. ZendTo re-generates the encryption key from the passphrase, and retrieves the IV from its database.
  3. ZendTo then attempts to decrypt the file. If it is successful (all 3 of the encrypted file, the passphrase and the IV have to be correct) then ZendTo downloads it to the user as it decrypts it, so ZendTo doesn't need RAM or disk space for a temporary decrypted copy of the files.
  4. ZendTo wipes the memory that is storing the passphrase, the key, the IV and any data derived from them.
  5. If the decryption failed, the file download never starts, the user is told their attempt at the passphrase was probably wrong, and is passed back to the download page to try again.
  6. If the passphrase was correct and the decryption succeeded then, just as with an unencrypted drop-off containing several files, they can just continue to click the links to download files without any further prompting. The passphrase is sent separately to the ZendTo server for each file. The web browser remembers the passphrase until the page is re-loaded (which happens if they enter the wrong passphrase), or they leave that particular page in any way.

Creating a Request for an Encrypted Drop-off

After the user has ticked the checkbox in the "request a drop-off" form and has entered their chosen passphrase, it is sent over https to the ZendTo server along with the other information needed to issue the request. At this point, the passphrase is temporarily stored on the server, but only until the person who received the request actually uploads any files. It is stored encrypted, using a key that is unique to that server and is created by the ZendTo Installer.

The passphrase is never sent to the person sending the requested files. All they know is that their files are going to be encrypted before being stored. They have no means of discovering the passphrase.

As soon as the files have been uploaded, the encryption occurs just as described in the "Creating an Encrypted Drop-off" section above. The stored copy of the passphrase is carefully overwritten in memory with random data, then freed from memory and immediately deleted from the database.

Why Use the Extra Initialisation Vector (IV) Data?

The IV data is random and different for each encrypted drop-off. It is not secret. Using it provides the benefits of "authenticated encryption with associated data" (AEAD, for short). These are many and varied (and a fantastic cure for insomnia if you're not a keen cryptographer), but basically it means that a "bad actor" cannot simply replace one encrypted file with another that has been encrypted with the same key.

The IV data never leaves the ZendTo server; the user's passphrase (from which the key is derived) is never stored on the ZendTo server (except temporarily in RAM which is then wiped). So it should be much more difficult to gather both parts than just a single one of them. Without both parts, a "bad actor" cannot even start the process of faking an encrypted file well enough that it would decrypt successfully. Attempting to decrypt a file with the wrong IV data would fail immediately, just as it would with the wrong key/passphrase.

If the ZendTo server uses MySQL for its database, then the IV is not even stored on the ZendTo server either (except temporarily in RAM like the key/passphrase). So to get hold of the IV data, they would also need access to the database. Gaining access to a server to be able to replace a file is one thing, gaining access to a different server to be able to read database records is a different attack.

 
Copyright 2022 © Jules Field/ZendTo