Asset-Compression-Manager/README.md

Asset Compression Manager (ACM) 
========================

## About

This tool exists to help compress, store, retrieve and validate media for games. The goal is to provide a simple workflow for checking assets for changes, generating new compressed versions based on those changes, storing those changed versions, and retrieving compressed files.

This tool is designed to work with an S3 compatible bucket storage provider.

## Requirements

- An S3 Compatible Storage Service
- Python 3.7+
- MozJpeg for jpeg compression
- OptiPNG for png optimization
- FFMpeg for video compression and audio decompression
- Opusenc for audio compression

## Using

### Configuring

ACM expects a configuration file specified with `--config` or as `acm-config.json` in the current directory.

The S3 compatible endpoint needs to be configured in the `s3` object of the configuration file.

```json
{
  "s3": {
    "secure": false,
    "host": "127.0.0.1:9000"
  }
}
```

- `secure` - specifies if __*https*__ protocol is used.
- `host` - is the \<hostname>\[:\<port>] for the S3 compatible endpoint.

Additionally the following environment variables need to be populated to interact with the S3 endpoint.

- `ACM_S3_ACCESS` - A valid S3 Access Key for the endpoint.
- `ACM_S3_SECRET` - A valid S3 Secret Key for the Access Key on the endpoint.

### Common Options

- `-c, --config`: The config file to read. Default file is `acm-config.json` in the current directory.
- `-x, --context`: The remote bucket to use. For `store` and `retrieve` operations it is `<value>-data`.
- `-s, --stdin`: Read the file list to process from stdin.
- `-p, --prefix`: The prefix to strip from the input. i.e. `acm.py -x test -p /tmp/data/storage/ check /tmp/data/storage/images/img1.jpg` => `images/img1.jpg`

### Listing Files

List all files in a bucket
```bash
$ ./acm.py -x <bucket> list
```

List all files while adding a prefix and stripping a suffix
```bash
$ ./acm.py -x <bucket> -p <prefix> list --suffix <suffix>
```

List all files with sha256sum compatible output
```bash
$ ./acm.py --context testing --prefix "/tmp/" --stdin list --suffix .json --sha256sum
```

Print out a sha256sum compatible check list

### Checking For Matches

Do a comparison of the remote bucket for files with a matching sha256sum value.

Process a list of files
```bash
$ ./acm.py -x <bucket> -p <prefix to strip> match FILES... 
```

Process a list from stdin
```bash
$ find /tmp -name '*.jpg' | ./acm.py -x <bucket> -p <prefix to strip> match
```

### Checking For Changes

Do a comparison of the remote bucket for missing files or files with a mismatch in their sha256sum values.

Process a list of files
```bash
$ ./acm.py -x <bucket> -p <prefix to strip> check FILES... 
```

Process a list from stdin
```bash
$ find /tmp -name '*.jpg' | ./acm.py -x <bucket> -p <prefix to strip> check
```

### Updating Metadata For Changed Files

Update the remote bucket with new metadata for the listed files. Calculates new sha256sum values.

Process a list of files
```bash
$ ./acm.py -x <bucket> -p <prefix to strip> update FILES... 
```

Process a list from stdin
```bash
$ find /tmp -name '*.jpg' | ./acm.py -x <bucket> -p <prefix to strip> update
```

### Storing Files

Store the listed files in `<bucket>-data`.

Process a list of files
```bash
$ ./acm.py -x <bucket> -p <prefix to strip> store FILES... 
```

Process a list from stdin
```bash
$ find /tmp -name '*.jpg' | ./acm.py -x <bucket> -p <prefix to strip> store
```

### Retrieving Files

Retrieve remote files matching listed files. Optionally place the downloaded files in a different destination.

Process a list of files
```bash
$ ./acm.py -x <bucket> -p <prefix to strip> retrieve [-d <destination>] FILES... 
```

Process a list from stdin
```bash
$ find /tmp -name '*.jpg' | ./acm.py -x <bucket> -p <prefix to strip> retrieve [-d <destination>]
```

### Configuring Profiles

### Compressing Changed Assets

## Contributing

## License
See [LICENSE.md](LICENSE.md)