S3¶
Our s3 Bucket is kbsoftware
Our tree structure is kbsoftware/contacts/<client site name e.g. hatherleigh_info>/<client folders>
Add a new Client to s3¶
Create a new folder in s3:
kbsoftware/contacts/hatherleigh_info
Add a new User in s3 Security Credentials:
https://console.aws.amazon.com/iam/home?region=eu-west-1#security_credential
User Name format as per client site name - select User from RH menu
Un check the “Generate an access key for each user” box and click Create
Open the newly created User:
Add them to the contacts group
Under Manage Password give them a defined password and record it along with their User name
Click Create Access Key and download the credentials.csv file to the relevant folder in the Private/Source folder for later use
Cyberduck CLI¶
We are using Cyberduck CLI to upload files to the s3 server.
Install Cyberduck CLI from:
https://dist.duck.sh/
For WINDOWS use: duck-4.8.0.18009.exe
Note its installation folder, probabaly:
c:\Program Files (i386)\Cyberduck CLI\duck.exe
Place the files to be uploaded to s3 in a known folder, note the path
Construct a script using Cyberducks duck.exe location, the Access Keys, the Source and Target paths e.g.:
"c:\Program Files (x86)\Cyberduck CLI\duck.exe" --username <Access Key ID from credentials.csv> --password <Secret Access Key from credentials.csv> --upload s3://kbsoftware/contacts/hatherleigh_info/<client folder>/ c:/Users/<user>/repo/wip/hatherleigh_info/ --existing overwrite
- NOTE:
If the path has spaces enclose it in “” marks
Use
--verbose
to see what is happening in the command windowIf using cut & paste ensure white space is not unwittingly included - Cyberduck will create a new folder on s3 and not update the foldere you created
Run the script from the command line to test the files are uploaded and then update in the correct folder
Open Task Scheduler in Windows and create a new basic task,name it and set a Trigger
Under Action select Start a Program and paste the script into the Program/script box - click next
Task Scheduler will ask to place the
--arguments
into the Arguments box - select YesThe Task is ready to run - you can run it manually from the Scheduler from The the Task Scheduler Library
To test that the Policy prevents this User writing to another Users folder run the following tests after creating a test user and keys in s3:
test to test - should work: "c:\Program Files (x86)\Cyberduck CLI\duck.exe" --username <Access Key ID from test_credentials.csv> --password <Secret Access Key from test_credentials.csv> --upload s3://kbsoftware/contacts/test/ c:/Users/<user>/repo/wip/hatherleigh_info/ --existing overwrite test to hatherleigh_info - should fail: "c:\Program Files (x86)\Cyberduck CLI\duck.exe" --username <Access Key ID from test_credentials.csv> --password <Secret Access Key from test_credentials.csv> --upload s3://kbsoftware/contacts/hatherleigh_info/<client folder>/ c:/Users/<user>/repo/wip/hatherleigh_info/ --existing overwrite hatherleighcommunitycentre_couk to test - should fail: "c:\Program Files (x86)\Cyberduck CLI\duck.exe" --username <Access Key ID from credentials.csv> --password <Secret Access Key from credentials.csv> --upload s3://kbsoftware/contacts/test/ c:/Users/<user>/repo/wip/hatherleigh_info/ --existing overwrite
Apply a Bucket Policy¶
These policies appear to apply to all users.
Click Add or Edit Bucket Policy from:
https://console.aws.amazon.com/s3/home?region=eu-west-1
To allow Public Read Only access to all folders:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowPublicRead", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": "s3:GetObject", "Resource": "arn:aws:s3:::kbsoftware/*" } ] }
Apply a User, Group or Role Policy¶
Click Create Policy in:
https://console.aws.amazon.com/iam/home?region=eu-west-1#policies
Select “Create Your Own Policy” and enter a Policy Name and the code
To allow access to User folders by only specific User:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowGroupToSeeBucketListInTheConsole", "Action": [ "s3:ListAllMyBuckets", "s3:GetBucketLocation" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::*" ] }, { "Sid": "AllowRootAndHomeListingOfCompanyBucket", "Action": [ "s3:ListBucket" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::kbsoftware" ], "Condition": { "StringEquals": { "s3:prefix": [ "", "contacts/" ], "s3:delimiter": [ "/" ] } } }, { "Sid": "AllowListingOfUserFolder", "Action": [ "s3:ListBucket" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::kbsoftware" ], "Condition": { "StringLike": { "s3:prefix": [ "contacts/${aws:username}/*" ] } } }, { "Sid": "AllowAllS3ActionsInUserFolder", "Action": [ "s3:*" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::kbsoftware/contacts/${aws:username}/*" ] } ] }