Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to content

Latest commit

 

History

History
 
 

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 

README.md

Transfer Service sample using Java

This app creates two types of transfers using the Transfer Service tool.

These samples are used on the following documentation pages:

Prerequisites

  1. Set up a project on Google Cloud Console.

  2. Go to the Google Cloud Console and create or select your project. You will need the project ID later.

  3. Enable the Google Storage Transfer API in the Google Cloud Console.

  4. Within Developers Console, select APIs & auth > Credentials.

  5. Select Add credentials > Service account > JSON key.

  6. Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to point to your JSON key.

  7. Add the Storage Transfer service account as an editor of your project.

  8. To get the email address used for the service account, execute the googleServiceAccounts.get REST method. There should be a "Try It" section on that page, otherwise execute it in the APIs Explorer.

    It should output an email address like:

    storage-transfer-1234567890@partnercontent.gserviceaccount.com
    
  9. Add this as a member and select the Project -> Editor permission on the Google Cloud Console IAM and Admin page.

  10. Set up gcloud for application default credentials.

  11. gcloud components update

  12. gcloud init

Transfer from Amazon S3 to Google Cloud Storage

Creating a one-time transfer from Amazon S3 to Google Cloud Storage.

  1. Set up data sink.
  2. Go to the Developers Console and create a bucket under Cloud Storage > Storage Browser.
  3. Set up data source.
  4. Go to AWS Management Console and create a bucket.
  5. Under Security Credentials, create an IAM User with access to the bucket.
  6. Create an Access Key for the user. Note the Access Key ID and Secret Access Key.
  7. Set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.
  8. Compile the package with
    mvn compile
  9. Run the transfer job with
    mvn exec:java \
        -Dexec.mainClass="com.google.cloud.storage.storagetransfer.samples.AwsRequester" \
        -DprojectId=your-google-cloud-project-id \
        -DjobDescription="Sample transfer job from S3 to GCS." \
        -DawsSourceBucket=your-s3-bucket-name \
        -DgcsSinkBucket=your-gcs-bucket-name
    1. Note the job ID in the returned Transfer Job.

Transfer data from a standard Cloud Storage bucket to a Cloud Storage Nearline bucket

Creating a daily transfer from a standard Cloud Storage bucket to a Cloud Storage Nearline bucket for files untouched for 30 days.

  1. Set up data sink.
  2. Go to the Developers Console and create a bucket under Cloud Storage > Storage Browser.
  3. Select Nearline for Storage Class.
  4. Set up data source.
  5. Go to the Developers Console and create a bucket under Cloud Storage > Storage Browser.
  6. In NearlineRequester.java, fill in the user-provided constants.
  7. Run with mvn compile and mvn exec:java -Dexec.mainClass="com.google.cloud.storage.storagetransfer.samples.NearlineRequester"
  8. Note the job ID in the returned Transfer Job.

Checking the status of a transfer

  1. In RequestChecker.java, fill in the user-provided constants. Use the Job Name you recorded earlier.
  2. Run with mvn compile and mvn exec:java -Dexec.mainClass="com.google.cloud.storage.storagetransfer.samples.RequestChecker"

References