Skip to content

Commit 3466297

Browse files
frankynfrankyn
authored
storage: S3 SDK sample to list buckets (GoogleCloudPlatform#1343)
* Humble beginnings * Add new s3 sdk sample to module list * Fix lint issues * Fix hanging resources and change client var name * Update README.md * Rename files to match other languages and update README * Fix incorrect class and update environment variable * Addressing review comments * Remove unnecessary variable * Update var name * Remove main and exec instructions from README
1 parent 118cf6e commit 3466297

File tree

7 files changed

+215
-0
lines changed

7 files changed

+215
-0
lines changed

‎.kokoro/tests/diff_tests.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ gradle -v
3131
export GOOGLE_APPLICATION_CREDENTIALS=${KOKORO_GFILE_DIR}/service-acct.json
3232
export GOOGLE_CLOUD_PROJECT=java-docs-samples-testing
3333
source ${KOKORO_GFILE_DIR}/aws-secrets.sh
34+
source ${KOKORO_GFILE_DIR}/storage-hmac-credentials.sh
3435
source ${KOKORO_GFILE_DIR}/dlp_secrets.txt
3536
# Activate service account
3637
gcloud auth activate-service-account\

‎.kokoro/tests/run_tests.sh

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ gradle -v
3131
export GOOGLE_APPLICATION_CREDENTIALS=${KOKORO_GFILE_DIR}/service-acct.json
3232
export GOOGLE_CLOUD_PROJECT=java-docs-samples-testing
3333
source ${KOKORO_GFILE_DIR}/aws-secrets.sh
34+
source ${KOKORO_GFILE_DIR}/storage-hmac-credentials.sh
3435
source ${KOKORO_GFILE_DIR}/dlp_secrets.txt
3536
# Activate service account
3637
gcloud auth activate-service-account\

‎pom.xml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,7 @@
9595

9696
<module>storage/cloud-client</module>
9797
<module>storage/json-api</module>
98+
<module>storage/s3-sdk</module>
9899
<module>storage/storage-transfer</module>
99100
<module>storage/xml-api/cmdline-sample</module>
100101
<module>storage/xml-api/serviceaccount-appengine-sample</module>

‎storage/s3-sdk/README.md

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Using Google Cloud Storage (GCS) with the S3 SDK
2+
3+
[Google Cloud Storage][1] features APIs that allows developers to store and access arbitrarily-large
4+
objects. The [GCS XML API][5] provides support for AWS S3 API users that use S3 SDKs.
5+
Learn more about [Migrating to GCS][6].
6+
7+
## Prerequisites
8+
9+
Install [Maven](http://maven.apache.org/).
10+
11+
## Setup
12+
13+
1. Clone this repo.
14+
15+
```
16+
git clone https://github.com/GoogleCloudPlatform/java-docs-samples.git
17+
```
18+
19+
1. Change into this directory:
20+
21+
```
22+
cd java-docs-samples/storage/s3-sdk
23+
```
24+
25+
1. Build this project from this directory:
26+
27+
```
28+
mvn package
29+
```
30+
31+
1. Get your [Interoperable Storage Access Keys][3] and set the following environment variables:
32+
33+
## Test Sample
34+
35+
1. Set the following environment variable with the default project for Interoperable Storage Access Keys.
36+
37+
* GOOGLE_CLOUD_PROJECT_S3_SDK=[GOOGLE_PROJECT_ID]
38+
* STORAGE_HMAC_ACCESS_KEY_ID=[ACCESS_KEY_ID]
39+
* STORAGE_HMAC_ACCESS_SECRET_KEY=[ACCESS_SECRET_KEY]
40+
41+
1. Run test using the following Maven command:
42+
43+
```
44+
mvn verify
45+
```
46+
47+
## Products
48+
- [Google Cloud Storage][2]
49+
50+
## Language
51+
- [Java][2]
52+
53+
## Dependencies
54+
- [AWS S3 Java SDK][4]
55+
56+
[1]: https://cloud.google.com/storage
57+
[2]: https://java.com
58+
[3]: https://cloud.google.com/storage/docs/migrating#keys
59+
[4]: https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3
60+
[5]: https://cloud.google.com/storage/docs/xml-api/overview
61+
[6]: https://cloud.google.com/storage/docs/migrating

‎storage/s3-sdk/pom.xml

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
<!--
2+
Copyright 2019 Google LLC
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
-->
16+
<project>
17+
<modelVersion>4.0.0</modelVersion>
18+
<groupId>com.google.apis-samples</groupId>
19+
<artifactId>storage-s3-interop-api</artifactId>
20+
<version>1</version>
21+
22+
<!--
23+
The parent pom defines common style checks and testing strategies for our samples.
24+
Removing or replacing it should not affect the execution of the samples in anyway.
25+
-->
26+
<parent>
27+
<groupId>com.google.cloud.samples</groupId>
28+
<artifactId>shared-configuration</artifactId>
29+
<version>1.0.10</version>
30+
</parent>
31+
32+
<properties>
33+
<maven.compiler.target>1.8</maven.compiler.target>
34+
<maven.compiler.source>1.8</maven.compiler.source>
35+
</properties>
36+
37+
<dependencies>
38+
<dependency>
39+
<groupId>com.amazonaws</groupId>
40+
<artifactId>aws-java-sdk-s3</artifactId>
41+
<version>1.11.445</version>
42+
</dependency>
43+
<!-- Test Dependencies -->
44+
<dependency>
45+
<groupId>junit</groupId>
46+
<artifactId>junit</artifactId>
47+
<scope>test</scope>
48+
<version>4.12</version>
49+
</dependency>
50+
<dependency>
51+
<groupId>com.google.truth</groupId>
52+
<artifactId>truth</artifactId>
53+
<version>0.36</version>
54+
<scope>test</scope>
55+
</dependency>
56+
</dependencies>
57+
</project>

‎storage/s3-sdk/src/main/java/ListGcsBuckets.java

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
/*
2+
* Copyright 2019 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
// [START storage_s3_sdk_list_buckets]
18+
import com.amazonaws.auth.AWSStaticCredentialsProvider;
19+
import com.amazonaws.auth.BasicAWSCredentials;
20+
import com.amazonaws.client.builder.AwsClientBuilder;
21+
import com.amazonaws.services.s3.AmazonS3;
22+
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
23+
import com.amazonaws.services.s3.model.Bucket;
24+
25+
import java.util.List;
26+
27+
public class ListGcsBuckets {
28+
public static List<Bucket> listGcsBuckets(String googleAccessKeyId,
29+
String googleAccessKeySecret) {
30+
// Create a BasicAWSCredentials using Cloud Storage HMAC credentials.
31+
BasicAWSCredentials googleCreds = new BasicAWSCredentials(googleAccessKeyId,
32+
googleAccessKeySecret);
33+
34+
// Create a new client and do the following:
35+
// 1. Change the endpoint URL to use the Google Cloud Storage XML API endpoint.
36+
// 2. Use Cloud Storage HMAC Credentials.
37+
AmazonS3 interopClient =
38+
AmazonS3ClientBuilder.standard()
39+
.withEndpointConfiguration(
40+
new AwsClientBuilder.EndpointConfiguration(
41+
"https://storage.googleapis.com", "auto"))
42+
.withCredentials(new AWSStaticCredentialsProvider(googleCreds))
43+
.build();
44+
45+
// Call GCS to list current buckets
46+
List<Bucket> buckets = interopClient.listBuckets();
47+
48+
// Print bucket names
49+
System.out.println("Buckets:");
50+
for (Bucket bucket : buckets) {
51+
System.out.println(bucket.getName());
52+
}
53+
54+
// Explicitly clean up client resources.
55+
interopClient.shutdown();
56+
57+
return buckets;
58+
}
59+
// [END storage_s3_sdk_list_buckets]
60+
}
61+

‎storage/s3-sdk/src/test/java/ListGcsBucketsTest.java

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
/*
2+
* Copyright 2019 Google LLC
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
import static com.google.common.truth.Truth.assertThat;
18+
19+
import com.amazonaws.services.s3.model.Bucket;
20+
import java.util.List;
21+
import org.junit.Test;
22+
23+
public class ListGcsBucketsTest {
24+
private static final String BUCKET = System.getenv("GOOGLE_CLOUD_PROJECT_S3_SDK");
25+
private static final String KEY_ID = System.getenv("STORAGE_HMAC_ACCESS_KEY_ID");
26+
private static final String SECRET_KEY = System.getenv("STORAGE_HMAC_ACCESS_SECRET_KEY");
27+
28+
@Test
29+
public void testListBucket() throws Exception {
30+
List<Bucket> buckets = ListGcsBuckets.listGcsBuckets(KEY_ID, SECRET_KEY);
31+
assertThat(buckets.toString()).contains(BUCKET);
32+
}
33+
}

0 commit comments

Comments
 (0)