arangodb s3 bucket name compatibility (#3588)

* Update arangodb_store.go

* update readme, properly escape queries, add name patching

* use underscore

* use underscore

* better comment

* fix readme

Co-authored-by: a <a@a.a>
This commit is contained in:
gfx
2022-09-09 11:43:42 -05:00
committed by GitHub
parent 10d545060f
commit 48db56ddad
3 changed files with 58 additions and 7 deletions

View File

@@ -22,6 +22,39 @@ i test using this dev database:
`docker run -p 8529:8529 -e ARANGO_ROOT_PASSWORD=test arangodb/arangodb:3.9.0`
## database structure
arangodb has a few restrictions which require the use of a few tricks in order to losslessly store the data.
### filer store
arangodb does not support []byte, and will store such as a uint64 array. this would be a waste of space. to counteract this, we store the data as a length prefixed uint64 byteset.
### filer kv
same as above
### filer buckets
s3 buckets are implemented through arangodb collection. this allows us to do very fast bucket deletion by simply deleting the collection
arangodb collection name rules is character set `azAZ09_-` with a 256 character max. however the first character must be a letter.
s3 bucket name rule is the set `azAZ09.-` with a 63 characters max.
the rules for collection names is then the following:
1. if the bucket name is a valid arangodb collection name, then nothing is done.
2. if the bucket name contains a ".", the "." is replaced with "_"
3. if the bucket name now begins with a number or "_", the prefix "xN--" is prepended to the collection name
this allows for these collection names to be used.
## features i don't personally need but are missing
[ ] provide tls cert to arango
[ ] authentication that is not basic auth