As of Python 3.9, it's no longer necessary to import things like List
and Dict from the typing module, and we can just use the built-in types
like this.
This also involved installing some new packages for the type stubs for a
few of the major third-party libraries.
I also had to change some of the imports in some model files in strange
ways, I'm not sure why some of these were necessary. I suspect this
might be a bug in mypy, but I'm not sure if I'll be able to build a
reproduction of it to be able to report it.
This is kind of dirty, but the prospector tool was broken after updating
Python to 3.9, and it seems to no longer be maintained. I forked it to
my personal GitHub account, un-pinned its dependencies, fixed a bug that
came up after updating pylint, and deleted a few dependencies that I
don't use (pylint plugins for Django, Flask, and Celery).
This commit also fixes all the new complaints from the updated pylint,
which were mostly explicitly re-raising exceptions, and some places
where I could use a generator instead of an unnecessary list
comprehension.
This will work for now, but I probably don't want to leave it in this
state. I should probably just stick to using the tools like pylint
directly, since this is now the second time I've needed to replace my
"tool runner" when it stopped being maintained (the first one was
pylama).
There is one special exception in here: the unread_user_ids column in
the message_conversations table had to be left as an integer array,
since the PostgreSQL intarray extension doesn't work with bigints. The
trigger that updates that column also needed a minor tweak.
This isn't good, but I don't really like how that was done anyway (it
was for the purpose of group messages that don't even exist), so it
could probably just be eliminated.
The minimal updates here were to update pygit2 and pip-tools.
However, prospector is currently broken as well, so the full code style
checks currently will not pass. This is not trivial to fix:
- Currently, pylint returns errors from some of the mypy annotations
- Upgrading pylint/astroid to the newest version fixes those errors,
but breaks prospector
- There is no newer release of prospector
I'm not totally sure how I want to fix this, I may need to fork
prospector.
This changes the site to run on Debian 10 instead of Ubuntu 16.04. It
also fully converts the previous Salt setup to use Ansible instead.
Most of this was a relatively straightforward conversion, and it should
be very close to equivalent. One notable difference is that I removed
the setup for the "monitoring" server, since I wasn't confident that the
way of setting up self-hosted Sentry and Grafana was working any more.
I'll look to re-add that at some point, but it's not urgent.
If user types "tag1 tag2" then adds a comma between,
it should respect the comma to give "tag1" and "tag2".
We use keydown and setTimeout because keyup
works on a keyboard but not reliably on mobile.
Nonzero timeout is needed or else the comma is sometimes
inserted too late and not seen by addChip(),
tested on desktop Firefox.
People are still continuing to try to abuse the donate page to check
stolen credit card numbers, and last night there was a massive burst of
attempts coming from many IPs, so the current rate-limiting wasn't able
to block most of it. Luckily Stripe blocked all of the charges this
time, but I can't keep risking another incident where Tildes is the
source of a bunch of fraudulent charges.
This adds a global rate-limit to the donate page that should never get
hit during normal usage. Hopefully this will be enough to keep the abuse
away from the page when it stops working for them relatively quickly.
Previously, rate limits had to apply to a particular user or a
particular IP address, or both. This adds support for global
rate-limits, where the limit will apply to everyone trying to perform
the action. This probably won't be used much overall, but might be
necessary for certain cases where something abusive is happening and it
can't be easily blocked by user or IP.
This is a bit ugly and would probably be better implemented by having a
separate class that inherits from RateLimitedAction or something
similar, but it will do the job.
This adds the backend pieces (no interface yet) to configure Lua scripts
that will be applied to topics and comments due to different events.
Initially, it only supports running a script when a new topic or comment
is posted. For example, here is a Lua script that would prepend a new
topic's title with "[Text] " or "[Link] " depending on its type, as well
as replace its tags with either "text" or "link":
function on_topic_post (topic)
if (topic.is_text_type) then
topic.title = "[Text] " .. topic.title
topic.tags = {"text"}
elseif (topic.is_link_type) then
topic.title = "[Link] " .. topic.title
topic.tags = {"link"}
end
end
There can be a global script as well as group-specific scripts, and the
scripts are sandboxed, with limited access to data as well as being
restricted to a subset of Lua's built-in functions. The Lua sandboxing
code comes from Splash (https://github.com/scrapinghub/splash). It will
need to be modified, but this commit keeps it unmodified so that future
changes can be more easily tracked by comparing to the original state of
the file.
The sandboxing also includes some restrictions on number of instructions
and memory usage, but this might be more effectively managed on the OS
level. More research will still need to be done on security and resource
restrictions before this feature can be safely opened to users.
This adds an "Edit title" choice in the actions dropdown for topics on
listing pages, instead of needing to go to the comments page.
Some pieces of this feel a little hack-ish (like needing to reduce the
bottom padding because of the usually-empty div that the title-editing
input gets put into), so I'll probably want to try and find a better
overall approach to this eventually, but it should do the job for now.
Just rearranges the module so the functions are in alphabetical order
(except web_server_reload, which has to be earlier so it can be called
as a post-task).
I think this is going to be a better way to name invoke tasks. The
previous naming where a verb was often first made it much harder for
anyone to figure out the name of a task that affects a certain thing
without always looking through the entire list.
For example, if someone is looking for a task that affects the web
server, it's much easier to find web-server-reload than
reload-web-server.
The changes were:
- check-code-style -> code-style-check
- reload-web-server -> web-server-reload
- renew-tls-certificate -> tls-certificate-renew
- type-checking -> type-check
- update-pip-requirements -> pip-requirements-update
I should have just done this all along, these have been way more trouble
than they're worth.
If the information is needed, it's always possible to just do a temp run
of pip-compile without --no-annotate or use a dedicated tool like
pipdeptree.
This is simpler than needing to know that --html-validation is the flag
to use to make sure that all tests are run, and can stay constant even
if we add other types of excluded-by-default tests in the future.
These were set up to redirect the original locations of the development
pages to their new locations inside the instructions folder, but can't
be used any more now that we're creating a development folder.
If a topic title has multiple sentences in it, it looks strange to strip
the trailing period off it, so we only want to do that automatically
when it's a single sentence.
Updates the Black code-formatter for Python to the latest version, and
applies it to some files that had formatting that the new version does
differently (splitting collections with trailing commas across lines).
This enables tab-completion for the new invoke tasks in the dev version.
So for example, you can type "invoke ty<Tab>" and it will complete to
"type-checking".
This way, instead of needing to know that you run "pytest" and knowing
tricks like "pytest -m ''" to run webtests and HTML validation, you can
now just run "invoke test", with more intuitive flags. This also reduces
the output in quiet mode even more.
After adding invoke tasks for some of the other tools/checks, I'll be
able to switch the git hooks to use these instead.
First invoke task: uses pip-compile to update the versions of all the
pip packages in requirements.txt and requirements-dev.txt. It also
post-processes the output file and removes any comments that have a "-r"
reference in them, since those currently cause Salt to break (and are
kind of redundant anyway).
Unfortunately, as part of writing this I discovered that invoke can't
handle type annotations in the definitions of its task functions, so I
had to exclude tasks.py from being checked by mypy. That makes me a
little nervous about whether invoke is still being maintained. Relevant
issue (over 4 years old): https://github.com/pyinvoke/invoke/issues/357
Deimos
Ras Fincher
Gyrfalcon05
Flashynuff
Andrew Shu
Carlos E. Garcia
Cassidy Dingenskirchen
Bauke