rclone details

In a prior post, I'd found that using rclone to upload RESTful (rclone uses REST, not SOAP) data had become more complex -- by at least three steps -- than two foundational videos from 2017:

1. Rclone basics   (8:30) Tyler, 2017.
2. Rclone encrypted   (10:21) Tyler, 2017.

These videos are still worthy for concepts, but additional steps --choices actually -- must be navigated for both encrypted and unencrypted storage, whichever one desires. Thus, a second post. Unlike signing in and out of one's various Google and OneDrive accounts, all are accessed from a single rclone client. Rclone is written in Go (500Mb), so that immense dependency must be installed.

across devices

To install rclone on multiple devices, including one's Android phone (RCX), save one's ~/.config/rclone/rclone.config. For each installed client, simply duplicate this file and one can duplicate the features of the original installation. If one has encryption, losing this file would be very bad.

deleted configurations

1. ~/.config/rclone/rclone.config (client). If this file is lost, duplicate it from another device. If lost entirely, access must be re-established entirely from scratch, and the encrypted files will be lost permanently.
2. scope (google) Google requires authentication for access details for which Google keeps. Documentation is difficult to find, other than the OAuth info in the prior sentence. It appears that users cannot directly edit any of the 11 access scopes (files) defined, but rather only through a Google dialog screen. When installing rclone, 5 of the 11 scopes are available, for which I typically like "drive.file".
   
   
   

command usage

For simplest use, to the root directory...

$ rclone copy freedom.txt mygoogleserv:/

Not all commands work on all servers, so use...

$ rclone help

instead of...

$ rclone --help

The former will display only those commands on the installed version of rclone. The latter shows all commands, but not every compilation has these.

$ rclone about mygoogleserv:
Total: 15G
Used: 10.855k
Free: 14.961G
Trashed: 0
Other: 40.264M

Of course, there's also the GUI, rclone-browser.

encryption notes

Rclone documentation notes strong encryption, especially if salt is used. Minimally, we're talking 256-bit. Of course governments can read it, but what can't they read?

• unencrypted accounts must be established first. Encryption is an additional feature superimposed onto unencrypted accounts.
• remember the names of uploaded encrypted files; even the names of files are encrypted on the server and the original filename is necessary for download.
• keep the same encryption password on all devices on which rclone is installed.

glossary

• application data folder (Google) a hidden folder in one's Drive (not in one's PC). The folder cannot be accessed directly via a web browser, but can be accessed from authorized (eg OAth) apps, eg rsync. The folder holds "scope" information for file permissions.
• authorization (OAuth, JWT, OpenID) protocols for using a third party REST app (rclone) to move files in and out of a cloud server (Google, AWS, Azure, Oracle), there's an authorization process between them, even though you are authenticated in both.
  What is OAuth (10:56) Java Brains, 2019.
  What is JWT (10:34) Bitfumes, 2018.
• scope (Google). the permissions granted inside Drive to RESTful data uploaded by users using, eg, rclone.
• REST Representational State Transfer API for server to client data transfer. Wikipedia notes this as an industry term and not a copyrighted concept by Oracle or Google. It refers to data exchanged by user-authorized third party apps between applications or databases and applications. This is as opposed to data directly entered by users, or data that is not authorized by users between servers.

  REST API concepts and examples (8:52) WebConcepts, 2014. Conceptually sound on this HTTP API, even though dated with respect to applications. Around 7:00 covers OAuth comprehensibly.

• SOAP Simple Object Access Protocol. This is the older API for server to client data transfer.

  SOAP v. REST API (2:34) SmartBear, 2017. Very quick comparison.

---

Google 15GB

Users can personally upload and save files in Google Drive through their browser as we all know. However, Google treats rclone as a third party app doing a RESTful transfer and uses OAth to authorize it. Additional hidden files are created by Google and placed into one's Drive account to limit or control the process.
Within that process, there are two ways to rclone with Google Drive, slower or faster. The faster method requires Google Cloud services (with credit card) and a ClientID (personal API key). The slower way uses rclone's generic API connection.

1. Slower uploads

Faster to set-up, but slower uploads. Users regularly backing-up only a few MB of files can use this to avoid set-up hassles. It bypasses the Cloud Services API, and uses the built in rclone ID to upload as directed

1. $ rclone config

   ... and just accept all defaults. For scope access, I chose option "3", which gives control over whatever's uploaded.
2. verify function by uploading a sample file and by looking in ~/.config/rclone/rclone-config to see that the entry looks sane

2. Faster uploads

This method requires a lengthier set-up but, once configured, rclone transfers files more quickly than the generic method above. Users need a credit card for a Google Cloud Services account, which in turn supplies them with a ClientID or API key for rclone or other 3rd party access into Drive.

1. get a Google email
2. sign-up for Google Cloud services
3. register one's project "app". In this case it's just rclone) with the Google API development team
4. waiting for their approval -- up to 2 weeks
5. receiving a Client ID and Client Secret which allow faster uploading and downloading through one's Drive account

These two videos move very quickly however they have the preferred Client ID and Client Secret method that supposedly speeds the process over the built-in ID's.

Rclone with Google API (6:38) Seedit4me, 2020. The first four minutes cover creating a remote and the 5 steps in creating the Client ID and Secret.
Get Client ID and Secret (7:29) DashSpan.me, 2020. Download and watch at 40% speed.

OneDrive 2GB

This primer is probably the best for OneDrive, however it also applies to many of the other providers

metadata and scope

These are hidden files within one's Google Drive. is is part of the Google Drive API v.3, which is what rclone uses to connect and transfer files. In particular, you will want to know about the Application Data Folder

Google API v3 usage (5:28) EVERYDAY BE CODING, 2017.
Get Client ID and Secret (7:29) DashSpan.me, 2020. Download and watch at 40% speed.
RESTFUL resources and OAuth (55:49) Oracle Developers, 2017.

Slackware 12.0 - Safety Part 1: expectations

high expectations, low knowledge, under-equipped

We typical home pc users combine our computer security system, end-user applications, and our critical data into a single box. Further, we design this arrangement without IT or programming experience. How effective do you believe such a security arrangement is likely to be?

Accordingly, seen in security terms, our home computer systems are an "opportunity". In large organizations, well-paid IT professionals design layers of dedicated security systems to protect workstation computer networks and data is backed up and retained elsewhere. Since we don't have that option, let's at least think about our configuration.

situational awareness - threats

Given that our home systems present opportunities, are there threats which seek to take advantage of them? First, it appears one may assume governments, US or otherwise, can easily penetrate our leaky home systems, and perhaps they do so regularly. It's also likely a large number of civilian entities penetrate our home computer systems. Given the opportunity and the threat, and our typical lack of resources to build a $3-5K layered and ported system, about all we can do is make ourselves a less inviting target than other home machines out there. Let's suppose here we're going to make the effort, where do we begin?

step one - no security

Linux security features sometimes function like shells around processes, sometimes around files, sometimes around ports, sometimes around data streams, sometimes around other things. The security system is complex. The dedicated security servers of the corporate world do this more easily since they operate with firmware or static-linked libraries and have no other applications or data to interfere. But we want applications and data on our home systems. This means we're bound to have permission or authentication problems when we configure complex security functions around desktop applications and web interfaces. Accordingly, I have one rule for a home system prior to activating iptables.chains,tripwire,ssh, really any security function - the system has to work exactly the way it's supposed to work before the security is implemented, and hopefully before it's connected to any network. If we don't have desktop functionality first, is the permission problem we see when activating security related to the security configuration or is it due to the user configuration? We have no way to cut the problem in half. Alternatively, we could bring up our security half before the desktop operations half, but this means we would have to estimate user applications in advance. So the first step of (home) security is to configure the desktop system to run reliably, including all it's peripherals of printers, scanners, cameras, audio inputs, and so on. Subsequently, we will activate our security and the net packages together, and adjust the mix when a desktop application ceases to work due to security layers. Ideally, we should test system applications after adding each security layer.