Mount S3 on linux
Important considerations when using s3fs, namely related to the inherent limitations of S3:
– no file can be over 5GB
– you can’t partially update a file so changing a single byte will re-upload the entire file.
– operation on many small files are very efficient (each is a separate S3 object after all) but large files are very inefficient
– Though S3 supports partial/chunked downloads, s3fs doesn’t take advantage of this so if you want to read just one byte of a 1GB file, you’ll have to download the entire GB.