File-oriented encryption tools like GnuPG are great for encrypting individual files, which can then be sent across untrusted networks as well as stored encrypted on disk. But sometimes they can be inconvenient, because the file must be decrypted each time it is to be used; this is especially cumbersome when you have a large collection of files to protect. Any time a security tool is cumbersome to use, there's a chance you'll forget to use it properly, leaving the files unprotected for the sake of convenience.

Worse, readable copies of the encrypted contents might still exist on the hard disk. Even if you overwrite these files (using rm -P) before unlinking them, your application software might make temporary copies you don't know about, or have been paged to swapspace—and even your hard disk might have silently remapped failing sectors with data still in them.

The solution is to simply never write the information unencrypted to the hard disk. Rather than taking a file-oriented approach to encryption, consider a block-oriented approach—a virtual hard disk, that looks just like a normal hard disk with normal filesystems, but which encrypts and decrypts each block on the way to and from the real disk.

The cgd device looks and behaves to the rest of the operating system like any other disk driver. Rather than driving real hardware directly, it provides a logical function layered on top of another block device. It has a special configuration program, cgdconfig, to create and configure a cgd device and point it at the underlying disk device that will hold the encrypted data.

NetBSD includes several other similar logical block devices, each of which provides some other function where cgd provides encryption. You can stack several of these logical block devices together: cgd on top of vnd is handy to make an encrypted volume in a regular file without repartitioning, or you can make an encrypted raid to protect your encrypted data against hard disk failure as well.

Once you have created a cgd disk, you can use disklabel to divide it up into partitions, swapctl to enable swapping to those partitions or newfs to make filesystems, then mount and use those filesystems, just like any other new disk.

The cgd driver was written by Roland C. Dowdeswell, and introduced in the NetBSD 2.0 release.

To use cgd you need a kernel with support for the cgd pseudo-device. Make sure the sure the module is loaded:

# modload cgd

If the cgd driver was not already present/loaded (it is loaded by default in some ports), add cgd to /etc/modules.conf.

The following ciphers are supported:

The following obsolete ciphers are supported for compatibility with old disks.

WARNING: These obsolete ciphers are implemented without timing side channel protection, so, for example, JavaScript code in a web browser that can measure the timing of disk activity may be able to recover the secret key. These are also based on 64-bit block ciphers and are therefore unsafe for disks much larger than a gigabyte. You should not use these except where compatibility with old disks is necessary.

cgdconfig can examine the disk to verify that it was decrypted using the correct key. The following verification methods are available:

First, decide which filesystems you want to move to an encrypted device. You're going to need to leave at least the small root (/) filesystem unencrypted, in order to load the kernel and run init, cgdconfig and the rc.d scripts that configure your cgd. In this example, we'll encrypt everything except the root (/) filesystem.

We are going to delete and re-make partitions and filesystems, and will require a backup to restore the data. So make sure you have a current, reliable backup stored on a different disk or machine. Do your backup in single-user mode, with the filesystems unmounted, to ensure you get a clean dump. Make sure you back up the disklabel of your hard disk as well, so you have a record of the partition layout before you started.

With the system at single user, / mounted read-write and everything else unmounted, use disklabel to delete all the data partitions you want to move into cgd.

Then make a single new partition in all the space you just freed up, say, wd0e. Set the partition type for this partition to cgd Though it doesn't really matter what it is, it will help remind you that it's not a normal filesystem later. When finished, label the disk to save the new partition table.

We have removed the partition table information, but the existing filesystems and data are still on disk. Even after we make a cgd device, create filesystems, and restore our data, some of these disk blocks might not yet be overwritten and still contain our data in plaintext. This is especially likely if the filesystems are mostly empty. We want to scrub the disk before we go further.

We could use dd to copy /dev/zero over the new wd0e partition, but this will leave our disk full of zeros, except where we've written encrypted data later. We might not want to give an attacker any clues about which blocks contain real data, and which are free space, so we want to write "noise" into all the disk blocks. So we'll create a temporary cgd, configured with a random, unknown key.

First, we configure a cgd to use a random key:

# cgdconfig -s cgd0 /dev/wd0e aes-xts 256 < /dev/urandom

Now we can write zeros into the raw partition of our cgd (/dev/rcgd0d on NetBSD/i386 and amd64, /dev/rcgd0c on most other platforms):

# dd if=/dev/zero of=/dev/rcgd0d bs=64k

The encrypted zeros will look like random data on disk. This might take a while if you have a large disk. Once finished, unconfigure the random-key cgd:

# cgdconfig -u cgd0

The cgdconfig program, which manipulates cgd devices, uses parameters files to store such information as the encryption type, key length, and a random password salt for each cgd. These files are very important, and need to be kept safe—without them, you will not be able to decrypt the data!

We'll generate a parameters file and write it into the default location (make sure the directory /etc/cgd exists and is mode 700):

# cgdconfig -g -V disklabel -o /etc/cgd/wd0e aes-cbc 256

This creates a parameters file /etc/cgd/wd0e describing a cgd using the aes-cbc cipher method, a key verification method of disklabel, and a key length of 256 bits. It will look something like this:

algorithm aes-cbc;
iv-method encblkno;
keylength 256;
verify_method disklabel;
keygen pkcs5_pbkdf2/sha1 {
        iterations 6275;
        salt AAAAgHTg/jKCd2ZJiOSGrgnadGw=;
};

Now it's time to create our cgd, for which we'll need a passphrase. This passphrase needs to be entered every time the cgd is opened, which is usually at each reboot. The encryption key is derived from this passphrase and the salt. Make sure you choose something you won't forget, and others won't guess.

The first time we configure the cgd, there is no valid disklabel on the logical device, so the validation mechanism we want to use later won't work. We override it this one time:

# cgdconfig -V re-enter cgd0 /dev/wd0e

This will prompt twice for a matching passphrase, just in case you make a typo, which would otherwise leave you with a cgd encrypted with a passphrase that's different to what you expected.

Now that we have a new cgd, we need to partition it and create filesystems. Recreate your previous partitions with all the same sizes, with the same letter names.

Then, use newfs to create filesystems on all the relevant partitions. This time your partitions will reflect the cgd disk names, for example:

# newfs /dev/rcgd0h

We've moved several filesystems to another (logical) disk, and we need to update /etc/fstab accordingly. Each partition will have the same letter (in this example), but will be on cgd0 rather than wd0. So you'll have /etc/fstab entries something like this:

/dev/wd0a   /     ffs     rw    1 1
/dev/cgd0b  none  swap    sw            0 0
/dev/cgd0b  /tmp  mfs     rw,-s=132m    0 0
/dev/cgd0e  /var  ffs     rw            1 2
/dev/cgd0f  /usr  ffs     rw            1 2
/dev/cgd0h  /home ffs     rw            1 2

Each time you reboot, you're going to need your cgd configured early, before fsck runs and filesystems are mounted.

Put the following line in /etc/cgd/cgd.conf:

cgd0    /dev/wd0e

This will use /etc/cgd/wd0e as config file for cgd0.

To finally enable cgd on each boot, put the following line into /etc/rc.conf:

cgd=YES

You should now be prompted for /dev/cgd0's passphrase whenever /etc/rc starts.

Next, mount your new filesystems, and restore your data into them. It often helps to have /tmp mounted properly first, as restore can use a fair amount of temporary space when extracting a large dumpfile.

To test your changes to the boot configuration, umount the filesystems and unconfigure the cgd, so when you exit the single-user shell, rc will run like on a clean boot, prompting you for the passphrase and mounting your filesystems correctly. Now you can bring the system up to multi-user, and make sure everything works as before.

cgd(4) provides highly secure encryption of whole partitions or disks. Unfortunately, creating "normal" CDs is not disklabeling something and running newfs on it. Neither can you just put a CDR into the drive, configure cgd and assume it to write encrypted data when syncing. Standard CDs contain at least an ISO-9660 filesystem created with mkisofs(8) from the sysutils/cdrtools package. ISO images may not contain disklabels or cgd partitions.

But of course CD reader/writer hardware doesn't care about filesystems at all. You can write raw data to the CD if you like—or an encrypted FFS filesystem, which is what we'll do here. But be warned, there is NO way to read this CD with any OS except NetBSD—not even other BSDs due to the lack of cgd.

The basic steps when creating an encrypted CD are:

The first step when creating an encrypted CD is to create a single image file with dd. The image may not grow, so make it large enough to allow all CD content to fit into. Note that the whole image gets written to the CD later, so creating a 700 MB image for 100 MB content will still require a 700 MB write operation to the CD. Some info on DVDs here: DVDs are only 4.7 GB in marketing language. 4.7GB = 4.7 x 1024 x 1024 x 1024 = 5046586573 bytes. In fact, a DVD can only approximately hold 4.7 x 1000 x 1000 x 1000 = 4700000000 bytes, which is about 4482 MB or about 4.37 GB. Keep this in mind when creating DVD images. Don't worry for CDs, they hold "real" 700 MB (734003200 Bytes).

Invoke all following commands as root!

For a CD:

# dd if=/dev/zero of=image.img bs=1m count=700

or, for a DVD:

# dd if=/dev/zero of=image.img bs=1m count=4482

Now configure a vnd(4)-pseudo disk with the image:

# vnconfig vnd0 image.img

In order to use cgd, a so-called parameter file, describing encryption parameters and a containing "password salt" must be generated. We'll call it /etc/cgd/image here. You can use one parameter file for several encrypted partitions (I use one different file for each host and a shared file image for all removable media, but that's up to you).

AES-CBC with a keylength of 256 bits will be used in this example. Refer to cgd(4) and cgdconfig(8) for further details and alternative ciphers.

The following command will create the parameter file as /etc/cgd/image. YOU DO NOT WANT TO INVOKE THE FOLLOWING COMMAND AGAIN after you burnt any CD, since a recreated parameter file is a lost parameter file and you'll never access your encrypted CD again (the "salt" this file contains will differ among each call). Consider this file being SACRED, BACK IT UP and BACK IT UP AGAIN! Use switch -V to specify verification method "disklabel" for the CD (cgd cannot detect whether you entered a valid password for the CD later when mounting it otherwise).

# cgdconfig -g -V disklabel aes-cbc 256 > /etc/cgd/image

Now it's time to configure a cgd for our vnd drive. (Replace slice "d" with "c" for all platforms that use "c" as the whole disk (where "sysctl kern.rawpartition" prints "2", not "3"); if you're on i386 or amd64, "d" is OK for you):

# cgdconfig -V re-enter cgd1 /dev/vnd0d /etc/cgd/image

The "-V re-enter" option is necessary as long as the cgd doesn't have a disklabel yet so we can access and configure it. This switch asks for a password twice and uses it for encryption.

Now it's time to create a disklabel inside the cgd. The defaults of the label are ok, so invoking disklabel with

# disklabel -e -I cgd1

and leaving vi with ":wq" immediately will do.

Let's create a filesystem on the cgd, and finally mount it somewhere:

# newfs /dev/rcgd1a
# mount /dev/cgd1a /mnt

The cgd is alive! Now fill /mnt with content. When finished, reverse the configuration process. The steps are:

The following commands are examples to burn the images on CD or DVD. Please adjust the dev= for cdrecord or the /dev/rcd0d for growisofs. Note the "rcd0d" is necessary with NetBSD. Growisofs is available in the sysutils/dvd+rw-tools package. Again, use "c" instead of "d" if this is the raw partition on your platform.

Finally, write the image file to a CD:

# cdrecord dev=/dev/rcd0d -v image.img

...or to a DVD:

# growisofs -dvd-compat -Z /dev/rcd0d=image.img

Congratulations! You've just created a really secure CD!

After creating an encrypted CD as described above, we're not done yet—what about mounting it again? One might guess, configuring the cgd on /dev/cd0d is enough—no, it is not.

NetBSD cannot access FFS file systems on media that is not 512 bytes/sector format. It doesn't matter that the cgd on the CD is, since the CD's disklabel the cgd resides in has 2048 bytes/sector.

But the CD driver cd(4) is smart enough to grant "write" access to the (emulated) disklabel on the CD. So before configuring the cgd, let's have a look at the disklabel and modify it a bit:

# disklabel -e cd0
# /dev/rcd0d:
type: ATAPI
disk: mydisc
label: fictitious
flags: removable
bytes/sector: 2048    # -- Change to 512 (= orig / 4)
sectors/track: 100    # -- Change to 400 (= orig * 4)
tracks/cylinder: 1
sectors/cylinder: 100 # -- Change to 400 (= orig * 4)
cylinders: 164
total sectors: 16386  # -- Change to value of slice "d" (=65544)
rpm: 300
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0

4 partitions:
#     size  offset  fstype [fsize bsize cpg/sgs]
 a:   65544   0     4.2BSD  0     0     0  # (Cyl. 0 - 655+)
 d:   65544   0     ISO9660 0     0        # (Cyl. 0 - 655+)

If you don't want to do these changes every time by hand, you can use Florian Stoehr's tool neb-cd512 which is (at time of writing this) in pkgsrc-wip and will move to sysutils/neb-cd512 soon. You can also download the neb-cd512 source from http://sourceforge.net/projects/neb-stoehr/ (be sure to use neb-cd512, not neb-wipe!).

It is invoked with the disk name as parameter, by root:

# neb-cd512 cd0

Now as the disklabel is in 512 b/s format, accessing the CD is as easy as:

# cgdconfig cgd1 /dev/cd0d /etc/cgd/image
# mount -o ro /dev/cgd1a /mnt

Note that the cgd MUST be mounted read-only or you'll get illegal command errors from the cd(4) driver which can in some cases make even mounting a CD-based cgd impossible!

Now we're done! Enjoy your secure CD!

# ls /mnt

Remember you have to reverse all steps to remove the CD:

# umount /mnt
# cgdconfig -u cgd1
# eject cd0

To encrypt the iSCSI device, we use the NetBSD iSCSI initiator, available in NetBSD-6 and newer, and the standard cgd device. In all, setting up an encrypted device in this manner should take less than 15 minutes, even for someone unfamiliar with iSCSI or cgd.

The approach is to layer a vnd on top of the "storage" file presented by the iSCSI target. This is exactly the same as normal. On top of that vnd, we layer a cgd device, which ensures that all data is encrypted on the iSCSI device.

WARNING: cgd only keeps the content of the volume secret—it doesn't keep the access patterns secret, and it doesn't prevent or even detect a malicious network or iSCSI target tampering with the volume.

Firstly, the initiator is started, pointing at the machine which is presenting the iSCSI storage (i.e. the machine on which the iSCSI target is running). In this example, the target is running on the same machine as the initiator (a laptop called, in a moment of inspiration, inspiron1300). A 50 MB iSCSI target is being presented as target1.

# iscsi-initiator -u agc -h inspiron1300.wherever.co.uk /mnt &
[1] 11196

# df
Filesystem   1K-blocks       Used      Avail %Cap Mounted on
/dev/dk0      28101396   20862004    5834324  78% /
kernfs               1          1          0 100% /kern
procfs               4          4          0 100% /proc
ptyfs                1          1          0 100% /dev/pts
/dev/puffs           0          0          0 100% /mnt

Looking at the last line, we can see that the initiator is running via the puffs device.

A vnd device is created on top of the storage which the target is presenting:

# vnconfig vnd0 /mnt/inspiron1300.wherever.co.uk/target1/storage

A disklabel which is offset 63 blocks into the iSCSI device needs to be added. This is so that the encrypted device which we shall put on top of the vnd does not clash with the vnd's label. The cgd's type should be set to "cgd".

# disklabel -e vnd0
# /dev/rvnd0d:
type: vnd
disk: vnd
label: fictitious
flags:
bytes/sector: 512
sectors/track: 32
tracks/cylinder: 64
sectors/cylinder: 2048
cylinders: 50
total sectors: 102400
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0

4 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a:    102336        63        cgd   2048 16384 28360  # (Cyl.      0 -     49)
 d:    102400         0     unused      0     0        # (Cyl.      0 -     49)

The cgd device can now be created on the vnd device

# cgdconfig -s cgd0 /dev/vnd0a aes-xts 256 < /dev/urandom

and the cgd device's storage zeroed

# dd if=/dev/zero of=/dev/rcgd0d bs=32k
dd: /dev/rcgd0d: Invalid argument
1601+0 records in
1600+0 records out
52428800 bytes transferred in 16.633 secs (3152095 bytes/sec)

Unconfigure the cgd device and write a disklabel using the verification method onto the cgd. Note: sometimes, this process does not always complete properly, and so it has to be repeated.

# cgdconfig -g -V disklabel -o /etc/cgd/vnd0a aes-cbc 256
cgdconfig: could not calibrate pkcs5_pbkdf2
cgdconfig: Failed to generate defaults for keygen
# cgdconfig -g -V disklabel -o /etc/cgd/vnd0a aes-cbc 256

A password can then be added to the cgd device

# cgdconfig -V re-enter cgd0 /dev/vnd0a
/dev/vnd0a's passphrase:
re-enter device's passphrase:

Then create a disklabel inside the cgd itself

# disklabel -I -e cgd0

# /dev/rcgd0d:
type: cgd
disk: cgd
label: fictitious
flags:
bytes/sector: 512
sectors/track: 2048
tracks/cylinder: 1
sectors/cylinder: 2048
cylinders: 49
total sectors: 102336
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0

4 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a:    102336         0     4.2BSD   2048 16384 28360  # (Cyl.      0 -     49*)
 d:    102336         0     unused      0     0        # (Cyl.      0 -     49*)

Having placed a disklabel inside the cgd, we can now make a filesystem on there:

# newfs /dev/rcgd0a
/dev/rcgd0a: 50.0MB (102336 sectors) block size 8192, fragment size 1024
using 4 cylinder groups of 12.49MB, 1599 blks, 3136 inodes.
super-block backups (for fsck_ffs -b #) at:
32, 25616, 51200, 76784,

the new file system in the cgd can now be mounted

# df
Filesystem   1K-blocks       Used      Avail %Cap Mounted on
/dev/dk0      28101396   20910216    5786112  78% /
kernfs               1          1          0 100% /kern
procfs               4          4          0 100% /proc
ptyfs                1          1          0 100% /dev/pts
/dev/puffs           0          0          0 100% /mnt
# mount /dev/cgd0a /iscsi
# df
Filesystem   1K-blocks       Used      Avail %Cap Mounted on
/dev/dk0      28101396   20910216    5786112  78% /
kernfs               1          1          0 100% /kern
procfs               4          4          0 100% /proc
ptyfs                1          1          0 100% /dev/pts
/dev/puffs           0          0          0 100% /mnt
/dev/cgd0a       49519          1      47043   0% /iscsi

The new file system, mounted on /iscsi, can now be used as normal.

The device can be freed up using the following commands

# umount /iscsi
# cgdconfig -u cgd0
# vnconfig -u vnd0

In normal usage, the device can be mounted. Firstly, the initiator must be configured to connect to the device:

# vnconfig vnd0 /mnt/inspiron1300.wherever.co.uk/target1/storage
# cgdconfig cgd0 /dev/vnd0a
/dev/vnd0a's passphrase:
# mount /dev/cgd0a /iscsi
# ls -al /iscsi
total 3
drwxr-xr-x   2 root  wheel   512 Jan  1  1970 .
drwxr-xr-x  35 root  wheel  1536 Jan  5 08:59 ..
# df
Filesystem   1K-blocks       Used      Avail %Cap Mounted on
/dev/dk0      28101396   20910100    5786228  78% /
kernfs               1          1          0 100% /kern
procfs               4          4          0 100% /proc
ptyfs                1          1          0 100% /dev/pts
/dev/puffs           0          0          0 100% /mnt
/dev/cgd0a       49519          1      47043   0% /iscsi

The following section will be replaced in NetBSD 10 by a sysctl knob “vm.swap_encrypt=1”, which provides better security and simpler setup.

You may want to use a dedicated random-key cgd for swap space, regenerating the key each reboot. The advantage of this is that once your machine is rebooted, any sensitive program memory contents that may have been paged out are permanently unrecoverable, because the decryption key is never known to you.

We created a temporary cgd with a random key when scrubbing the disk in the example above, using a shorthand cgdconfig -s invocation to avoid creating a parameters file.

The cgdconfig params file includes a “randomkey” keygen method. This is more appropriate for "permanent" random-key configurations, and facilitates the easy automatic configuration of these volumes at boot time.

For example, if you wanted to convert your existing /dev/wd0b partition to a dedicated random-key cgd1, use the following command to generate /etc/cgd/wd0b:

# cgdconfig -g -o /etc/cgd/wd0b -V none -k randomkey blowfish-cbc

When using the randomkey keygen method, only verification method "none" can be used, because the contents of the new cgd are effectively random each time (the previous data decrypted with a random key). Likewise, the new disk will not have a valid label or partitions, and swapctl will complain about configuring swap devices not marked as such in a disklabel.

In order to automate the process of labeling the disk, prepare an appropriate disklabel and save it to a file, for example /etc/cgd/wd0b.disklabel. Please refer to disklabel(8) for information about how to use disklabel to set up a swap partition.

On each reboot, to restore this saved label to the new cgd, create the /etc/rc.conf.d/cgd file as below:

swap_device="cgd1"
swap_disklabel="/etc/cgd/wd0b.disklabel"
start_postcmd="cgd_swap"

cgd_swap()
{
	if [ -f $swap_disklabel ]; then
		disklabel -R -r $swap_device $swap_disklabel
	fi
}

The same technique could be extended to encompass using newfs to re-create an ffs filesystem for /tmp if you didn't want to use mfs.

Avoid data loss by making sure you can always recover your passphrase and parameters file. Protect the parameters file from disclosure, perhaps by storing it on removable media as above, because the salt it contains helps protect against dictionary attacks on the passphrase.

Keeping the data encrypted on your disk is all very well, but what about other copies? You already have at least one other such copy (the backup we used during this setup), and it's not encrypted. Piping dump through file-based encryption tools like gpg can be one way of addressing this issue, but make sure you have all the keys and tools you need to decrypt it to restore after a disaster.

Like any form of software encryption, the cgd key stays in kernel memory while the device is configured, and may be accessible to privileged programs and users, such as /dev/kmem grovellers. Taking other system security steps, such as running with elevated securelevel, is highly recommended.

Once the cgd volumes are mounted as normal filesystems, their contents are accessible like any other file. Take care of file permissions and ensure your running system is protected against application and network security attack.

Avoid using suspend/resume, especially for laptops with a BIOS suspend-to-disk function. If an attacker can resume your laptop with the key still in memory, or read it from the suspend-to-disk memory image on the hard disk later, the whole point of using cgd is lost.