Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I have wanted this forever from S3 (cherry-picking from stored object archives) without having to implement reading/caching the zip index and doing a range request. This is awesome, solid feature.


(dev here) Thanks!

The initial idea was a customer request was to specify ranges as key=x->y;key2=z->w when uploading a file via a header and then specifying the key as a substitute for Range on get.

We chewed a bit on that and found that the zip method was much more flexible and easy to use, where the file name effectively is the key. Either would require client-side changes, so we might as well make it as easy to use as possible.


While not zip S3 does now support SELECT into a number of datatypes. https://docs.aws.amazon.com/AmazonS3/latest/userguide/select...


Correct, but it pretty much forces a scan through the file. For AWS, you pay for scanning the file, even if only a few results are returned. Currently this is $0.002 per scanned GB, plus request and transfer fee.

For MinIO you "pay" in CPU time and disk iops to scan the file.

So for retrieving single values it is a very inefficient/expensive method.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: