ernado
11 years ago
2 changed files with 304 additions and 0 deletions
-
303docs/api.rst
-
1docs/index.rst
@ -0,0 +1,303 @@ |
|||||
|
API |
||||
|
=================================== |
||||
|
|
||||
|
Master |
||||
|
################################### |
||||
|
You can append to any HTTP API with &pretty=y to see a formatted json output. |
||||
|
|
||||
|
Assign a file key |
||||
|
*********************************** |
||||
|
|
||||
|
.. code-block:: bash |
||||
|
|
||||
|
# Basic Usage: |
||||
|
curl http://localhost:9333/dir/assign |
||||
|
{"count":1,"fid":"3,01637037d6","url":"127.0.0.1:8080", |
||||
|
"publicUrl":"localhost:8080"} |
||||
|
# To assign with a specific replication type: |
||||
|
curl "http://localhost:9333/dir/assign?replication=001" |
||||
|
# To specify how many file ids to reserve |
||||
|
curl "http://localhost:9333/dir/assign?count=5" |
||||
|
# To assign a specific data center |
||||
|
curl "http://localhost:9333/dir/assign?dataCenter=dc1" |
||||
|
|
||||
|
Lookup volume |
||||
|
*********************************** |
||||
|
We would need to find out whether the volumes have moved. |
||||
|
|
||||
|
.. code-block:: bash |
||||
|
|
||||
|
curl "http://localhost:9333/dir/lookup?volumeId=3&pretty=y" |
||||
|
{ |
||||
|
"locations": [ |
||||
|
{ |
||||
|
"publicUrl": "localhost:8080", |
||||
|
"url": "localhost:8080" |
||||
|
} |
||||
|
] |
||||
|
} |
||||
|
# Other usages: |
||||
|
# You can actually use the file id to lookup |
||||
|
curl "http://localhost:9333/dir/lookup?volumeId=3,01637037d6" |
||||
|
# If you know the collection, specify it since it will be a little faster |
||||
|
curl "http://localhost:9333/dir/lookup?volumeId=3&collection=turbo" |
||||
|
|
||||
|
Force garbage collection |
||||
|
*********************************** |
||||
|
If your system has many deletions, the deleted file's disk space will not be synchronously re-claimed. There is a background job to check volume disk usage. If empty space is more than the threshold, default to 0.3, the vacuum job will make the volume readonly, create a new volume with only existing files, and switch on the new volume. If you are impatient or doing some testing, vacuum the unused spaces this way. |
||||
|
|
||||
|
.. code-block:: bash |
||||
|
|
||||
|
curl "http://localhost:9333/vol/vacuum" |
||||
|
curl "http://localhost:9333/vol/vacuum?garbageThreshold=0.4" |
||||
|
|
||||
|
The garbageThreshold=0.4 is optional, and will not change the default threshold. You can start volume master with a different default garbageThreshold. |
||||
|
|
||||
|
Pre-Allocate Volumes |
||||
|
*********************************** |
||||
|
|
||||
|
One volume servers one write a time. If you need to increase concurrency, you can pre-allocate lots of volumes. |
||||
|
|
||||
|
.. code-block:: bash |
||||
|
|
||||
|
curl "http://localhost:9333/vol/grow?replication=000&count=4" |
||||
|
{"count":4} |
||||
|
# specify a collection |
||||
|
curl "http://localhost:9333/vol/grow?collection=turbo&count=4" |
||||
|
# specify data center |
||||
|
curl "http://localhost:9333/vol/grow?dataCenter=dc1&count=4" |
||||
|
|
||||
|
This generates 4 empty volumes. |
||||
|
|
||||
|
Upload File Directly |
||||
|
*********************************** |
||||
|
|
||||
|
.. code-block:: bash |
||||
|
|
||||
|
curl -F file=@/home/chris/myphoto.jpg http://localhost:9333/submit |
||||
|
{"fid":"3,01fbe0dc6f1f38","fileName":"myphoto.jpg", |
||||
|
"fileUrl":"localhost:8080/3,01fbe0dc6f1f38","size":68231} |
||||
|
|
||||
|
This API is a little convenient. The master server would contact itself via HTTP to get an file id and store it to the right volume server. It is a convenient API and does not support different parameters when assigning file id. |
||||
|
|
||||
|
Check System Status |
||||
|
*********************************** |
||||
|
|
||||
|
.. code-block:: bash |
||||
|
|
||||
|
curl "http://10.0.2.15:9333/cluster/status?pretty=y" |
||||
|
{ |
||||
|
"IsLeader": true, |
||||
|
"Leader": "10.0.2.15:9333", |
||||
|
"Peers": [ |
||||
|
"10.0.2.15:9334", |
||||
|
"10.0.2.15:9335" |
||||
|
] |
||||
|
} |
||||
|
curl "http://localhost:9333/dir/status?pretty=y" |
||||
|
{ |
||||
|
"Topology": { |
||||
|
"DataCenters": [ |
||||
|
{ |
||||
|
"Free": 3, |
||||
|
"Id": "dc1", |
||||
|
"Max": 7, |
||||
|
"Racks": [ |
||||
|
{ |
||||
|
"DataNodes": [ |
||||
|
{ |
||||
|
"Free": 3, |
||||
|
"Max": 7, |
||||
|
"PublicUrl": "localhost:8080", |
||||
|
"Url": "localhost:8080", |
||||
|
"Volumes": 4 |
||||
|
} |
||||
|
], |
||||
|
"Free": 3, |
||||
|
"Id": "DefaultRack", |
||||
|
"Max": 7 |
||||
|
} |
||||
|
] |
||||
|
}, |
||||
|
{ |
||||
|
"Free": 21, |
||||
|
"Id": "dc3", |
||||
|
"Max": 21, |
||||
|
"Racks": [ |
||||
|
{ |
||||
|
"DataNodes": [ |
||||
|
{ |
||||
|
"Free": 7, |
||||
|
"Max": 7, |
||||
|
"PublicUrl": "localhost:8081", |
||||
|
"Url": "localhost:8081", |
||||
|
"Volumes": 0 |
||||
|
} |
||||
|
], |
||||
|
"Free": 7, |
||||
|
"Id": "rack1", |
||||
|
"Max": 7 |
||||
|
}, |
||||
|
{ |
||||
|
"DataNodes": [ |
||||
|
{ |
||||
|
"Free": 7, |
||||
|
"Max": 7, |
||||
|
"PublicUrl": "localhost:8082", |
||||
|
"Url": "localhost:8082", |
||||
|
"Volumes": 0 |
||||
|
}, |
||||
|
{ |
||||
|
"Free": 7, |
||||
|
"Max": 7, |
||||
|
"PublicUrl": "localhost:8083", |
||||
|
"Url": "localhost:8083", |
||||
|
"Volumes": 0 |
||||
|
} |
||||
|
], |
||||
|
"Free": 14, |
||||
|
"Id": "DefaultRack", |
||||
|
"Max": 14 |
||||
|
} |
||||
|
] |
||||
|
} |
||||
|
], |
||||
|
"Free": 24, |
||||
|
"Max": 28, |
||||
|
"layouts": [ |
||||
|
{ |
||||
|
"collection": "", |
||||
|
"replication": "000", |
||||
|
"writables": [ |
||||
|
1, |
||||
|
2, |
||||
|
3, |
||||
|
4 |
||||
|
] |
||||
|
} |
||||
|
] |
||||
|
}, |
||||
|
"Version": "0.47" |
||||
|
} |
||||
|
|
||||
|
Volume Server |
||||
|
################################### |
||||
|
|
||||
|
Upload File |
||||
|
*********************************** |
||||
|
|
||||
|
.. code-block:: bash |
||||
|
|
||||
|
curl -F file=@/home/chris/myphoto.jpg http://127.0.0.1:8080/3,01637037d6 |
||||
|
{"size": 43234} |
||||
|
|
||||
|
The size returned is the size stored on WeedFS, sometimes the file is automatically gzipped based on the mime type. |
||||
|
|
||||
|
Upload File Directly |
||||
|
*********************************** |
||||
|
|
||||
|
.. code-block:: bash |
||||
|
|
||||
|
curl -F file=@/home/chris/myphoto.jpg http://localhost:8080/submit |
||||
|
{"fid":"3,01fbe0dc6f1f38","fileName":"myphoto.jpg","fileUrl":"localhost:8080/3,01fbe0dc6f1f38","size":68231} |
||||
|
|
||||
|
This API is a little convenient. The volume server would contact the master to get an file id and store it to the right volume server(not necessarily itself). |
||||
|
|
||||
|
Delete File |
||||
|
*********************************** |
||||
|
|
||||
|
.. code-block:: bash |
||||
|
|
||||
|
curl -X DELETE http://127.0.0.1:8080/3,01637037d6 |
||||
|
|
||||
|
Create а specific volume on a specific volume server |
||||
|
***************************************************** |
||||
|
.. code-block:: bash |
||||
|
|
||||
|
curl "http://localhost:8080/admin/assign_volume?replication=000&volume=3" |
||||
|
|
||||
|
This generates volume 3 on this volume server. |
||||
|
|
||||
|
If you use other replicationType, e.g. 001, you would need to do the same on other volume servers to create the mirroring volumes. |
||||
|
|
||||
|
Check Volume Server Status |
||||
|
*********************************** |
||||
|
|
||||
|
.. code-block:: bash |
||||
|
|
||||
|
curl "http://localhost:8080/status?pretty=y" |
||||
|
{ |
||||
|
"Version": "0.34", |
||||
|
"Volumes": [ |
||||
|
{ |
||||
|
"Id": 1, |
||||
|
"Size": 1319688, |
||||
|
"RepType": "000", |
||||
|
"Version": 2, |
||||
|
"FileCount": 276, |
||||
|
"DeleteCount": 0, |
||||
|
"DeletedByteCount": 0, |
||||
|
"ReadOnly": false |
||||
|
}, |
||||
|
{ |
||||
|
"Id": 2, |
||||
|
"Size": 1040962, |
||||
|
"RepType": "000", |
||||
|
"Version": 2, |
||||
|
"FileCount": 291, |
||||
|
"DeleteCount": 0, |
||||
|
"DeletedByteCount": 0, |
||||
|
"ReadOnly": false |
||||
|
}, |
||||
|
{ |
||||
|
"Id": 3, |
||||
|
"Size": 1486334, |
||||
|
"RepType": "000", |
||||
|
"Version": 2, |
||||
|
"FileCount": 301, |
||||
|
"DeleteCount": 2, |
||||
|
"DeletedByteCount": 0, |
||||
|
"ReadOnly": false |
||||
|
}, |
||||
|
{ |
||||
|
"Id": 4, |
||||
|
"Size": 8953592, |
||||
|
"RepType": "000", |
||||
|
"Version": 2, |
||||
|
"FileCount": 320, |
||||
|
"DeleteCount": 2, |
||||
|
"DeletedByteCount": 0, |
||||
|
"ReadOnly": false |
||||
|
}, |
||||
|
{ |
||||
|
"Id": 5, |
||||
|
"Size": 70815851, |
||||
|
"RepType": "000", |
||||
|
"Version": 2, |
||||
|
"FileCount": 309, |
||||
|
"DeleteCount": 1, |
||||
|
"DeletedByteCount": 0, |
||||
|
"ReadOnly": false |
||||
|
}, |
||||
|
{ |
||||
|
"Id": 6, |
||||
|
"Size": 1483131, |
||||
|
"RepType": "000", |
||||
|
"Version": 2, |
||||
|
"FileCount": 301, |
||||
|
"DeleteCount": 1, |
||||
|
"DeletedByteCount": 0, |
||||
|
"ReadOnly": false |
||||
|
}, |
||||
|
{ |
||||
|
"Id": 7, |
||||
|
"Size": 46797832, |
||||
|
"RepType": "000", |
||||
|
"Version": 2, |
||||
|
"FileCount": 292, |
||||
|
"DeleteCount": 0, |
||||
|
"DeletedByteCount": 0, |
||||
|
"ReadOnly": false |
||||
|
} |
||||
|
] |
||||
|
} |
||||
Write
Preview
Loading…
Cancel
Save
Reference in new issue