@ -92,20 +96,28 @@ If stored as a string, in theory, you would need 8+1+16+8=33 bytes. A char(33) w
If space is really a concern, you can store the file id in your own format. You would need one 4-byte integer for volume id, 8-byte long number for file key, 4-byte integer for file cookie. So 16 bytes are enough (more than enough).
If space is really a concern, you can store the file id in your own format. You would need one 4-byte integer for volume id, 8-byte long number for file key, 4-byte integer for file cookie. So 16 bytes are enough (more than enough).
### Read File ###
### Read File ###
Here is the example on how to render the URL.
Here is the example on how to render the URL.
First lookup the volume server's URLs by the file's volumeId:
First lookup the volume server's URLs by the file's volumeId. However, since usually there are not too many volume servers, and volumes does not move often, you can cache the results most of the time. Depends on the replication type, one volume can have multiple replica locations. Just randomly pick one location to read.
(However, since usually there are not too many volume servers, and volumes does not move often, you can cache the results most of the time. Depends on the replication type, one volume can have multiple replica locations. Just randomly pick one location to read.)
Now you can take the public url, render the url or directly read from the volume server via url:
Now you can take the public url, render the url or directly read from the volume server via url:
```
```
http://localhost:8080/3,01637037d6.jpg
http://localhost:8080/3,01637037d6.jpg
```
```
Notice we add an file extension ".jpg" here. It's optional and just one way for the client to specify the file content type.
Notice we add an file extension ".jpg" here. It's optional and just one way for the client to specify the file content type.
If you want a nicer URL, you can use one of these alternative URL formats:
If you want a nicer URL, you can use one of these alternative URL formats:
Or the master server can determine the data center via volume server's IP address and settings in weed.conf file.
Or the master server can determine the data center via volume server's IP address and settings in weed.conf file.
Now when requesting a file key, an optional "dataCenter" parameter can limit the assigned volume to the specific data center. For example, this specify
Now when requesting a file key, an optional "dataCenter" parameter can limit the assigned volume to the specific data center. For example, this specifies that the assigned volume should be limited to 'dc1':
```
```
http://localhost:9333/dir/assign?dataCenter=dc1
http://localhost:9333/dir/assign?dataCenter=dc1
```
```
@ -182,9 +200,9 @@ When a client needs to read a file based on <volume id, file key, file cookie>,
Please see the example for details on write-read process.
Please see the example for details on write-read process.
### Storage Size ###
### Storage Size ###
In current implementation, each volume can be size of 8x2^32^=32G bytes. This is because of aligning contents to 8 bytes. We can be easily increased to 64G, or 128G, or more, by changing 2 lines of code, at the cost of some wasted padding space due to alignment.
In current implementation, each volume can be size of 8x2^32 bytes (32GiB). This is because of aligning contents to 8 bytes. We can be easily increased to 64G, or 128G, or more, by changing 2 lines of code, at the cost of some wasted padding space due to alignment.
There can be 2^32^ volumes. So total system size is 8 x 2^32^ x 2^32^ = 8 x 4G x 4G = 128GG bytes. (Sorry, I don't know the word for giga of giga bytes.)
There can be 2^32 volumes. So total system size is 8 x 2^32 bytes x 2^32 = 8 x 4GiB x 4Gi = 128EiB (2^67 bytes, or 128 exbibytes).
Each individual file size is limited to the volume size.
Each individual file size is limited to the volume size.
yourchanges
10 years ago