Amazon S3 driver for VFS (Apache Commons Virtual File System)

Roadmap for 17.0

• [ ] Migrate to Java 17, Gradle, and Palantir code style
• [ ] Update all dependencies and set up local testing with LocalStack and MinIO
• [ ] Split into multi-module project: native filesystem, Spring integration, Commons VFS adapter
• [ ] Make the project AI-native — bots and tooling for fast contribution acceptance
• [ ] Shade AWS SDK to avoid version conflicts with consumer projects

Latest branch 4.x.x

How to add as dependency into your Maven build

For an artifact with embedded AWS SDK (the easiest way to bootstrap without jar hell)

<dependency>
    <groupId>com.github.abashev</groupId>
    <artifactId>vfs-s3-with-awssdk-v1</artifactId>
    <version>4.4.0</version>
</dependency>

For an artifact without dependencies

<dependency>
    <groupId>com.github.abashev</groupId>
    <artifactId>vfs-s3</artifactId>
    <version>4.4.0</version>
</dependency>

How to add as dependency into your Gradle build

implementation 'com.github.abashev:vfs-s3:4.4.0'

URL format

s3://[[access-key:secret-key]:sign-region]@endpoint-url/folder-or-bucket/

• Access key and Secret key could come from default AWS SDK chain (environment, container etc)
• Sign-region is useful for custom implementations
• If endpoint URL from known providers then we will try to get bucket name from host, if not able to do it then bucket is first path segment

Supported providers

Sample groovy scripts - https://github.com/abashev/vfs-s3/tree/branch-4.x.x/samples

s3-copy able to copy between clouds, via http url or between different accounts

s3-copy s3://access1:secret1@s3-tests.storage.yandexcloud.net/javadoc.jar s3://access2:secret2@s3.eu-central-1.amazonaws.com/s3-tests-2/javadoc.jar

s3-copy https://oss.sonatype.org/some-name/120133-1-javadoc.jar s3://s3.eu-central-1.amazonaws.com/s3-tests-2/javadoc.jar

Sample Java Code for AWS S3

// Create a folder
FileSystemManager fsManager = VFS.getManager();
FileObject dir = fsManager.resolveFile("s3://simple-bucket.s3-eu-west-1.amazonaws.com/test-folder/");
dir.createFolder();

// Upload file to S3
FileObject dest = fsManager.resolveFile("s3://s3-eu-west-1.amazonaws.com/test-bucket/backup.zip");
FileObject src = fsManager.resolveFile(new File("/path/to/local/file.zip").getAbsolutePath());
dest.copyFrom(src, Selectors.SELECT_SELF);

Sample Java Code for Yandex Cloud https://cloud.yandex.ru/

// Upload file
FileObject dest = fsManager.resolveFile("s3://s3-tests.storage.yandexcloud.net/backup.zip");
FileObject src = fsManager.resolveFile(new File("/path/to/local/file.zip").getAbsolutePath());
dest.copyFrom(src, Selectors.SELECT_SELF);

Local Development

This project uses mise to manage Java and Maven versions locally.

mise trust
mise exec -- mvn compile
mise exec -- mvn test

On CI (GitHub Actions), Java and Maven are installed via actions/setup-java, so mvn is used directly without mise.

Running tests

For running tests you need active credentials for AWS. You can specify them as

1. Shell environment properties

   export AWS_ACCESS_KEY=AAAAAAA
   export AWS_SECRET_KEY=SSSSSSS
   export BASE_URL=s3://<full url like simple-bucket.s3-eu-west-1.amazonaws.com or s3-eu-west-1.amazonaws.com/test-bucket>
   

2. Or any standard ways how to do it in AWS SDK (iam role and so on)

Make sure that you never commit your credentials!

TODO

• [x] Shadow all dependencies inside vfs-s3 artifact

Old releases

Branch	Build Status	Code coverage
branch-3.0.x
branch-2.4.x
branch-2.3.x
branch-2.2.x