Encryption

In today's world of research, researchers regularly handle data, send it over the internet and store it in the cloud. At any point, especially when the internet is involved, the data is exposed to some risk. Keeping data safe and encrypted is hence a key component of IRB requirements and research ethics. Encryption should take place whenever dealing with sensitive data in any stage of research: from sampling and data collection to cleaning and analysis. This page discusses encryption in transit and at rest; key pairs; password management; and encryption with SurveyCTO data.

Read First

• Store encrypted data iefolder’s EncryptedData folder. Note that while iefolder makes the folder, it does not encrypt it.
• World Bank SurveyCTO server data must be encrypted via SurveyCTO.
• Almost all encryption depends on a public/private key pair, which should be securely stored in a password manager.
• Encryption in transit, or encryption while data is sent over the internet, is extremely important: there is never a case when not using encryption in transit is at all ok. Encryption at rest, or encryption while data is stored on a server or computer, is also important.

Encryption in Transit and at Rest

Encryption in Transit

Encryption in transit is by far the most important type of encryption. Service providers almost always take care of (i.e. Survey Solutions, SurveyCTO, OneDrive). However, when using less established services, confirm that they use encryption in transit by looking at the internet address. If the service provider uses secure transfer methods, you will see https:// in the internet address instead of simply http://. Data sent from an API using http:// can easily be spied on.

Never send anything of importance over the internet unless the URL starts with https://. Data transferred over an http:// connection can often be openly read by every server through which the data passes. Those servers are controlled by governments and private companies; hackers can easily tap into their traffic and read data, copy files, read passwords etc. https:// is not the only secure way to transfer data over the internet, but it is the one that researchers should know of as we use it frequently. If setting up advanced protocols to send files, make sure that they are set up to be secure. For example, instead of using FTP, use FTPS.

Note that encryption in transit has nothing to do with a service requiring a username or password. While a password-protected resource can only be requested by someone with the correct password, once it is in transit to the authorized user, the servers handling it can still see it if it is not encrypted.

Encryption at Rest

Encryption at rest means that the data stored on the server is scrambled in such a way that it is unreadable by anyone – even if they can access the file directly. If the data is not encrypted at rest, then anyone with access to the server can read that data (including, for example, the host company and your team’s administrators). If the data is encrypted at rest, however, the data is impossible to read even if someone gained unauthorized access to the database or the files where the data is stored. Encryption at rest uses an authorization tool called a public/private key pair.

Encryption at rest is not as easy to implement as encryption in transit. Both types of encryption use a private/public key pair to ensure that no unauthorized person gets access to the data. Since the time during which the data must be encrypted in transit is so short, the web servers never need to give users the key pair. Once the transfer is complete, the key pair is discarded and never needed or used again. However, in encryption at rest, the research team encrypts the data at one point in time and will access the data at a later point in time. The computer therefore must give the key pair to a human. As is so often the case, the weakest link is always the human factor. No private/public key pair is secure if the computer that generated the key saves it or is able to re-generate it. Thus, humans must safekeep the key pair. If we lose it, the data is lost forever and there is no way whatsoever to regain access to it.

The exact way in which encryption at rest is implemented depends on the service. For more information, read the instruction specific to your service.

Public/Private Key Pair

The keys used in the private/public key pair are either strings or small files. Exactly how the private/public key pair is created differs depending on the service. Complex mathematical relationships connect the two keys (we never need to understand these), allowing anyone with the private key to decrypt anything encrypted by someone with the public key. In essence, the public/private key pair system is like a vault with two doors. One door has a tiny opening where you can only put things in the vault but not take anything out. To open this door, you only need the public key. Since the door cannot be used to take anything out of the vault, it is safe for multiple people to have this key. The second door is a big door that can be used to take out all the content of the vault. To open this door, you need the private key. It is therefore very important to restrict access to the private key.

The following principles always hold for public/private key pairs:

• If you lose your private key, there is no way of decrypting your data ever again. Your data is lost forever. To safeguard the private key, use a password manager.
• The key pair can only be created once. Any services that claim to be able to re-generate the private key for you are not safe: this is equivalent to them having your password. This either means that the data is not properly encrypted or that they saved a copy of your private key that gives them full access to your encrypted data.
• It is perfectly fine if the service keeps a copy of your public key. This allows them to encrypt new data as it is coming in. Some services only give you the private key and you do not have to worry about the public key. It depends on the context.
• Most services provide a convenient way of decrypting your data, but if you are not asked for the key every time the data is viewed or decrypted, then your data is not encrypted properly.

Password and Key Management

When using public/private key pairs, you need to keep a lot of keys safe and well-organized. You can never store, organize, or transfer them using unsecure methods like email or Dropbox. DIME Analytics recommends using password managers as a convenient and safe solution for storing private keys and passwords. While there are many password managers out there, DIME Analytics recommends LastPass, whose free, basic tier satisfies all requirements a research team would ever need.

Password managers like LastPass allow you to store small text files or string fields, as private/public key pairs sometimes come in file format. LastPass can save keys as small text files or as strings for you as secure notes, which you can also use for other important information, like noting to which encryption container the private key pertains. Password managers like LastPass can also help you randomize passwords so that your passwords are impossible-to-guess, long strings. Since the password manager store and remembers this password for you, you will not need to memorize it and you can easily have a different long random string for each account you have.

Some password managers allow you to share passwords across accounts. This is a great feature: if one person updates the password, then everyone else has the updated password in their account immediately. Password sharing is sometimes a paid feature. One way around paying is for the research team to create one account and share access to that account. However, make sure to not share this account to more people than those who really needs it.

Encryption with SurveyCTO Data

When programming questionnaires in SurveyCTO, use the key generator to create a public-private key pair for your data with the name of your intended survey. These will download two files to your laptop: these are the keys. Create an encrypted survey with the public key. Then put the keys in the vault via LastPass or VeraCrypt. Note that you will need the private key for the form data download. Store encrypted data iefolder’s EncryptedData folder. Note, however, that while iefolder makes the folder, it does not encrypt it.

In your first cleaning do-file, de-identify the PII data to create a de-identified dataset. If you’re using Veracrypt, the veracrypt command allows Stata to call for VeraCrypt to mount the drive. Then you have to manually enter the password anytime you run the do-file. The first cleaning file should move the non-PII version of your data to the regular data folder.

Back to Parent

This article is part of the topic Data Security

Additional Research

• DIME Analytics' slides on Encryption
• Poverty Action Lab’s Data Security Procedures for Researchers