You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
303 lines
7.8 KiB
303 lines
7.8 KiB
API
|
|
===================================
|
|
|
|
Master server
|
|
###################################
|
|
You can append to any HTTP API with &pretty=y to see a formatted json output.
|
|
|
|
Assign a file key
|
|
***********************************
|
|
|
|
.. code-block:: bash
|
|
|
|
# Basic Usage:
|
|
curl http://localhost:9333/dir/assign
|
|
{"count":1,"fid":"3,01637037d6","url":"127.0.0.1:8080",
|
|
"publicUrl":"localhost:8080"}
|
|
# To assign with a specific replication type:
|
|
curl "http://localhost:9333/dir/assign?replication=001"
|
|
# To specify how many file ids to reserve
|
|
curl "http://localhost:9333/dir/assign?count=5"
|
|
# To assign a specific data center
|
|
curl "http://localhost:9333/dir/assign?dataCenter=dc1"
|
|
|
|
Lookup volume
|
|
***********************************
|
|
We would need to find out whether the volumes have moved.
|
|
|
|
.. code-block:: bash
|
|
|
|
curl "http://localhost:9333/dir/lookup?volumeId=3&pretty=y"
|
|
{
|
|
"locations": [
|
|
{
|
|
"publicUrl": "localhost:8080",
|
|
"url": "localhost:8080"
|
|
}
|
|
]
|
|
}
|
|
# Other usages:
|
|
# You can actually use the file id to lookup
|
|
curl "http://localhost:9333/dir/lookup?volumeId=3,01637037d6"
|
|
# If you know the collection, specify it since it will be a little faster
|
|
curl "http://localhost:9333/dir/lookup?volumeId=3&collection=turbo"
|
|
|
|
Force garbage collection
|
|
***********************************
|
|
If your system has many deletions, the deleted file's disk space will not be synchronously re-claimed. There is a background job to check volume disk usage. If empty space is more than the threshold, default to 0.3, the vacuum job will make the volume readonly, create a new volume with only existing files, and switch on the new volume. If you are impatient or doing some testing, vacuum the unused spaces this way.
|
|
|
|
.. code-block:: bash
|
|
|
|
curl "http://localhost:9333/vol/vacuum"
|
|
curl "http://localhost:9333/vol/vacuum?garbageThreshold=0.4"
|
|
|
|
The garbageThreshold=0.4 is optional, and will not change the default threshold. You can start volume master with a different default garbageThreshold.
|
|
|
|
Pre-Allocate Volumes
|
|
***********************************
|
|
|
|
One volume servers one write a time. If you need to increase concurrency, you can pre-allocate lots of volumes.
|
|
|
|
.. code-block:: bash
|
|
|
|
curl "http://localhost:9333/vol/grow?replication=000&count=4"
|
|
{"count":4}
|
|
# specify a collection
|
|
curl "http://localhost:9333/vol/grow?collection=turbo&count=4"
|
|
# specify data center
|
|
curl "http://localhost:9333/vol/grow?dataCenter=dc1&count=4"
|
|
|
|
This generates 4 empty volumes.
|
|
|
|
Upload File Directly
|
|
***********************************
|
|
|
|
.. code-block:: bash
|
|
|
|
curl -F file=@/home/chris/myphoto.jpg http://localhost:9333/submit
|
|
{"fid":"3,01fbe0dc6f1f38","fileName":"myphoto.jpg",
|
|
"fileUrl":"localhost:8080/3,01fbe0dc6f1f38","size":68231}
|
|
|
|
This API is a little convenient. The master server would contact itself via HTTP to get an file id and store it to the right volume server. It is a convenient API and does not support different parameters when assigning file id.
|
|
|
|
Check System Status
|
|
***********************************
|
|
|
|
.. code-block:: bash
|
|
|
|
curl "http://10.0.2.15:9333/cluster/status?pretty=y"
|
|
{
|
|
"IsLeader": true,
|
|
"Leader": "10.0.2.15:9333",
|
|
"Peers": [
|
|
"10.0.2.15:9334",
|
|
"10.0.2.15:9335"
|
|
]
|
|
}
|
|
curl "http://localhost:9333/dir/status?pretty=y"
|
|
{
|
|
"Topology": {
|
|
"DataCenters": [
|
|
{
|
|
"Free": 3,
|
|
"Id": "dc1",
|
|
"Max": 7,
|
|
"Racks": [
|
|
{
|
|
"DataNodes": [
|
|
{
|
|
"Free": 3,
|
|
"Max": 7,
|
|
"PublicUrl": "localhost:8080",
|
|
"Url": "localhost:8080",
|
|
"Volumes": 4
|
|
}
|
|
],
|
|
"Free": 3,
|
|
"Id": "DefaultRack",
|
|
"Max": 7
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"Free": 21,
|
|
"Id": "dc3",
|
|
"Max": 21,
|
|
"Racks": [
|
|
{
|
|
"DataNodes": [
|
|
{
|
|
"Free": 7,
|
|
"Max": 7,
|
|
"PublicUrl": "localhost:8081",
|
|
"Url": "localhost:8081",
|
|
"Volumes": 0
|
|
}
|
|
],
|
|
"Free": 7,
|
|
"Id": "rack1",
|
|
"Max": 7
|
|
},
|
|
{
|
|
"DataNodes": [
|
|
{
|
|
"Free": 7,
|
|
"Max": 7,
|
|
"PublicUrl": "localhost:8082",
|
|
"Url": "localhost:8082",
|
|
"Volumes": 0
|
|
},
|
|
{
|
|
"Free": 7,
|
|
"Max": 7,
|
|
"PublicUrl": "localhost:8083",
|
|
"Url": "localhost:8083",
|
|
"Volumes": 0
|
|
}
|
|
],
|
|
"Free": 14,
|
|
"Id": "DefaultRack",
|
|
"Max": 14
|
|
}
|
|
]
|
|
}
|
|
],
|
|
"Free": 24,
|
|
"Max": 28,
|
|
"layouts": [
|
|
{
|
|
"collection": "",
|
|
"replication": "000",
|
|
"writables": [
|
|
1,
|
|
2,
|
|
3,
|
|
4
|
|
]
|
|
}
|
|
]
|
|
},
|
|
"Version": "0.47"
|
|
}
|
|
|
|
Volume Server
|
|
###################################
|
|
|
|
Upload File
|
|
***********************************
|
|
|
|
.. code-block:: bash
|
|
|
|
curl -F file=@/home/chris/myphoto.jpg http://127.0.0.1:8080/3,01637037d6
|
|
{"size": 43234}
|
|
|
|
The size returned is the size stored on WeedFS, sometimes the file is automatically gzipped based on the mime type.
|
|
|
|
Upload File Directly
|
|
***********************************
|
|
|
|
.. code-block:: bash
|
|
|
|
curl -F file=@/home/chris/myphoto.jpg http://localhost:8080/submit
|
|
{"fid":"3,01fbe0dc6f1f38","fileName":"myphoto.jpg","fileUrl":"localhost:8080/3,01fbe0dc6f1f38","size":68231}
|
|
|
|
This API is a little convenient. The volume server would contact the master to get an file id and store it to the right volume server(not necessarily itself).
|
|
|
|
Delete File
|
|
***********************************
|
|
|
|
.. code-block:: bash
|
|
|
|
curl -X DELETE http://127.0.0.1:8080/3,01637037d6
|
|
|
|
Create а specific volume on a specific volume server
|
|
*****************************************************
|
|
.. code-block:: bash
|
|
|
|
curl "http://localhost:8080/admin/assign_volume?replication=000&volume=3"
|
|
|
|
This generates volume 3 on this volume server.
|
|
|
|
If you use other replicationType, e.g. 001, you would need to do the same on other volume servers to create the mirroring volumes.
|
|
|
|
Check Volume Server Status
|
|
***********************************
|
|
|
|
.. code-block:: bash
|
|
|
|
curl "http://localhost:8080/status?pretty=y"
|
|
{
|
|
"Version": "0.34",
|
|
"Volumes": [
|
|
{
|
|
"Id": 1,
|
|
"Size": 1319688,
|
|
"RepType": "000",
|
|
"Version": 2,
|
|
"FileCount": 276,
|
|
"DeleteCount": 0,
|
|
"DeletedByteCount": 0,
|
|
"ReadOnly": false
|
|
},
|
|
{
|
|
"Id": 2,
|
|
"Size": 1040962,
|
|
"RepType": "000",
|
|
"Version": 2,
|
|
"FileCount": 291,
|
|
"DeleteCount": 0,
|
|
"DeletedByteCount": 0,
|
|
"ReadOnly": false
|
|
},
|
|
{
|
|
"Id": 3,
|
|
"Size": 1486334,
|
|
"RepType": "000",
|
|
"Version": 2,
|
|
"FileCount": 301,
|
|
"DeleteCount": 2,
|
|
"DeletedByteCount": 0,
|
|
"ReadOnly": false
|
|
},
|
|
{
|
|
"Id": 4,
|
|
"Size": 8953592,
|
|
"RepType": "000",
|
|
"Version": 2,
|
|
"FileCount": 320,
|
|
"DeleteCount": 2,
|
|
"DeletedByteCount": 0,
|
|
"ReadOnly": false
|
|
},
|
|
{
|
|
"Id": 5,
|
|
"Size": 70815851,
|
|
"RepType": "000",
|
|
"Version": 2,
|
|
"FileCount": 309,
|
|
"DeleteCount": 1,
|
|
"DeletedByteCount": 0,
|
|
"ReadOnly": false
|
|
},
|
|
{
|
|
"Id": 6,
|
|
"Size": 1483131,
|
|
"RepType": "000",
|
|
"Version": 2,
|
|
"FileCount": 301,
|
|
"DeleteCount": 1,
|
|
"DeletedByteCount": 0,
|
|
"ReadOnly": false
|
|
},
|
|
{
|
|
"Id": 7,
|
|
"Size": 46797832,
|
|
"RepType": "000",
|
|
"Version": 2,
|
|
"FileCount": 292,
|
|
"DeleteCount": 0,
|
|
"DeletedByteCount": 0,
|
|
"ReadOnly": false
|
|
}
|
|
]
|
|
}
|