I highly recommend the Cybersecurity course at Open.edu lead by Cory Doctorow. In it will also clarify the law, who is responsible and accountable and when it is so. For example, if you leave your doors and windows unlocked when not home, you can be sued by clients for information stolen by a thief or hacker. 1) Unimportant documents that might be shared publicly I back up on a commercial cloud service.2) Important documents I encrypt and keep on an SD card in my keeping. 3) I also create Restore Points of my Operating System regularly.
If you keep client information, you are responsible for their safe-guarding. You can keep them on a removable hard-drive that you might store in a safe.
I use https://www.7-zip.org/ which is a free Zip utility with a password feature that I unlock each time I need certain files.
COMMGiving 