This script can be scheduled as a cronjob, and will dump the database,
compress and GPG-encrypt it, and upload to an FTP. Afterwards, it will
also delete any backups older than the specified retention periods, both
locally as well as on the FTP (with individual retention periods).
No functional difference, but probably a little better to use default
values where possible instead of specifying particular ways of setting
it back to default (especially for things like topic_type).
This shouldn't make much difference in practice since deleted_time
should be updated automatically, but it's probably safest to make sure
that the posts are still deleted before cleaning them up, in case the
deleted_time ends up staying set somehow.
Some new fields have been added to comments and topics recently
(excerpt, original_url), but they weren't added to the cleanup script
yet. There was also no reason to keep information about whether the
posts had been edited or not.
As of version 3.0, the redis-py package no longer has a distinction
between its Redis and StrictRedis classes, and both behave the same
(StrictRedis is just an alias for Redis).
This would have continued working as-is, but we might as well switch it
back to the normal name now that StrictRedis doesn't have any benefit.
mypy 0.640 has made it so that it's no longer necessary to annotate the
return type for __init__ methods, since it's always None. The only time
it's necessary now is if the method doesn't have any arguments, since
this shows that the method should still be type-checked.
Both the initialize_db script and Alembic need to import all
DatabaseModel subclasses so that DatabaseModel.metadata has all the
models attached. Previously this wasn't being done properly, and the
intialize_db script wasn't creating the scraper_result table since it
hadn't been imported.
This commit creates a dedicated module to import all of those classes,
so that both those locations can simply import * from it instead of
needing to import all the models individually. This still isn't great
overall, but it should be less prone to mistakes at least.
This follows the REUSE practices to add license and copyright info to
all source files: https://reuse.software/practices/2.0/
In addition, LICENSE.md was switched to a plaintext LICENSE file, to
support the tag-value header as recommended.
Note that files that are closer to configuration than code did not have
headers added. This includes all Salt files, Alembic files, and Python
files such as most __init__.py files that only import other files, since
those are similar to header files which are not considered
copyrightable.
Pylama is no longer maintained, and has been gradually getting slower
and slower, as well as being incompatible with Python 3.7 and newer
versions of astroid and pylint. This replaces it with Prospector, which
is being maintained by the same group as pylint and some other code
quality tools.
Black won't re-wrap comments because it has no way to determine when
that's a "safe" thing to do (that might break deliberate line-breaks).
So this is a quick pass through to re-wrap most multi-line comments and
docstrings to the new max line-length of 88.
This commit contains only changes that were made automatically by Black
(except for some minor fixes to string un-wrapping and two
format-disabling blocks in the user and group schemas). Some manual
cleanup/adjustments will probably need to be made in a follow-up commit,
but this one contains the result of running Black on the codebase
without significant further manual tweaking.
The cleanup of old deleted comments and topics was repeatedly hitting
the same items even though they had already been cleaned up on previous
runs. This just explicitly excludes all the ones that have already had
their user_id info removed, so it will only hit ones that actually need
it.
Deimos
Chad Birch