I'm going to start using pip-tools to manage dependencies:
https://github.com/jazzband/pip-tools
This makes updating the dependencies and virtualenv easier in a few
ways, and makes it simple to keep dev dependencies split out (so I can
stop installing them in production).
Now, to do a check and update all packages to their newest versions, the
main command is:
pip-compile --no-header --upgrade requirements.in
and again with requirements-dev.in to update that one as well. This will
update all the package versions in requirements.txt and
requirements-dev.txt. The virtualenv can then be updated to match those
versions by running:
pip-sync requirements.txt
(or requirements-dev.txt for dev environment). This currently needs to
be run with sudo, but I'm going to try to fix that shortly.
Now that people can use quotes inside searches, it's awkward to be
displaying them with quotes already around them. This should remove
everywhere that could have been happening.
I don't think this has ever affected anything, but this function
shouldn't have been defined as STABLE. That enables the database to
potentially optimize away some calls to it, and since this has side
effects, we never want that to happen. Removing this defaults it back to
classifying as VOLATILE, which is what we want.
This installs PL/Python (specifically plpython3u), enables it in the
database, and creates a function id36_to_id that calls the Python
function with the same name inside the tildes.lib.id module. This will
enable doing queries similar to this, when I have a topic's ID36 from
the site:
SELECT * FROM topics WHERE topic_id = id36_to_id('asdf');
The fact that this was possible to set up without having to port the
id36_to_id logic to a different language is blowing my mind a little.
There are some really interesting possibilities from being able to
import all of the Python code into the database itself.
Now that PostgreSQL has been upgraded to version 12 (from 10), the
websearch_to_tsquery() function is available. This is much better for
general search purposes, and will add multiple capabilities.
I'm not sure if there's a full specification of what it supports
anywhere, but this is the main relevant section in the docs:
https://www.postgresql.org/docs/12/textsearch-controls.html
Changing these pillar values are the only actual changes to Tildes
code/config needed, but if you're upgrading an existing version from 10
to 12 you will need to do some manual steps. The below should cover it -
lines starting with a * are descriptions of things you need to do, while
the rest are actual commands to run:
sudo apt-get install postgresql-12
sudo systemctl stop postgresql@10-main.service
sudo systemctl stop postgresql@12-main.service
cd /var/lib/postgresql
sudo -u postgres /usr/lib/postgresql/12/bin/pg_upgrade -b /usr/lib/postgresql/10/bin/ -B /usr/lib/postgresql/12/bin/ -d /var/lib/postgresql/10/main/ -D /var/lib/postgresql/12/main/ -o '-c config_file=/etc/postgresql/10/main/postgresql.conf' -O '-c config_file=/etc/postgresql/12/main/postgresql.conf'
* Change pillar value to 12, and run salt
sudo systemctl stop postgresql@10-main.service
* Edit /etc/postgresql/12/main/postgresql.conf and change port to 5432
sudo systemctl restart postgresql@12-main.service
sudo -u postgres ./analyze_new_cluster.sh
* After verifying the new version seems to be working, clean up the old version:
sudo apt-get remove postgresql-10
sudo rm -rf /usr/lib/postgresql/10/
sudo rm -rf /var/lib/postgresql/10/
sudo rm -rf /etc/postgresql/10/
The creation of the Theme Previews page showed that the way I was doing
metrics for database model objects being created wasn't very good.
Whenever someone loaded the Theme Previews page, the "total topics"
metric would increase by 2, "total comments" by 4, and "exemplary
labels" by 1, because the page is creating that many fake objects and
the metrics were being sent in each class's __init__ method.
This changes to take advantage of SQLAlchemy's object lifecycle event
for "pending to persistent", which only triggers when an object is
actually persisted to the database. When this event happens, the
object's _update_creation_metric method is called, and all metric
updates have been moved into that method now.
This is mostly just rearranging, but a couple of functional changes:
* The "preview blocks" can now be clicked to switch themes, instead of
using the dropdown menu.
* Click events should be disabled in the fake posts, so we don't need to
worry about voting/labels/etc.
This actually involves the necessary backend changes for full comment
search, but I'm a little nervous about the privacy implications of that,
and don't want to enable it for everyone out of nowhere. So for now,
this just allows users to search *their own* comments or topics.
These views and templates are starting to get real ugly, and probably
need a major re-thinking soon.
Previously a topic with an unknown content type would display "Article".
This switches to "Link" which will have some different idiosyncracies,
but should be a little more correct in general.
When a chain of comments has been fully removed, this will display a
single marker like "Removed by admin: 8 comments by 2 users" instead of
individual "Comment removed by site admin" markers for each one.
This should be stored in the database so that we can take advantage of
it for searching/filtering, but I want to test it on the live site first
and get input about what other types of content get posted. It'll be
simpler to leave it in code only for that, and I can add it to the
database later.
I've always been pretty unhappy with the ugly way tags were implemented,
using @hybrid_property and needing to do strange things all over the
place to deal with converting underscores to spaces and vice versa, as
well as other idiosyncracies.
There are still a few oddities here and there, but overall this is much
better.
Now that there are topics being posted automatically by the "fake" user,
we don't want to be generating reply notifications for that user
whenever someone posts a comment in them. These serve no purpose and
would just pile up forever without being read. The same problem could
have already been happening if people replied back to a comment with the
"unknown user" author.
This was always inconsistent - I've been setting objects in some models,
and ids in other ones. Using ids can cause various errors, especially in
cases where one of the objects involved hasn't been committed to the
database yet and so hasn't been assigned its id.
For some reason (maybe only in the default/white theme), this was using
currentColor, making the "indentation" borders darker than they should
have been.
Renames the username_linked macro to link_to_user and takes a User
object instead of a username, so that we can handle more than just the
"unknown user", including the new generic "Tildes" user.
This adds the backend for scheduled topics, which can be set up to post
at a certain time and then (optionally) repeat on a schedule.
Currently, these topics must be text topics, and can have their title,
markdown, and tags set up. They can be configured to be posted by a
particular user, but if no user is chosen they will be posted by a
(newly added) generic user named "Tildes" that is intended to be used
for "owning" automatic actions like this.
Topic tags are no longer shown in listing pages by default (except
"important" ones such as "nsfw" and "spoiler"), and the layout and
emphasis on them has been adjusted in multiple other ways as well. A
preference to continue showing them in listings has been added, and this
preference is enabled by default for all users with the "topic.tag"
permission (ability to tag any users' topics).
Other changes:
* The tagging field is collapsed by default when posting a new topic. If
a user expands it and enters tags, it will be shown expanded in the
future. It will also be expanded by default after the user
adds/changes tags on a topic, and will also be set automatically for
users that have shown an interest in tagging recently (by submitting a
topic with tags or changing tags).
* "Content metadata" such as word count and duration is now shown next
to the group name (where tags were previously), instead of after the
title.
* Topic icons (favicons) are now shown next to the domain name, instead
of before the title. Blank icons no longer have a dashed border and
are now just an empty space.
* When on a topic's comments page, tags are now shown in the main
content area, below the "byline" information about who posted the
topic and when, instead of in the sidebar.
* Other minor layout changes to adjust for tags, as well as prepare for
some other upcoming changes like adding "actions" to topics in
listings.
We don't want inline code to increase the line-height of lines its in,
since that makes paragraphs have uneven line-heights and looks a little
strange.
There are some new capabilities in mypy 0.730, including nicer output
formatting, which this enables. I'd also like to be able to use the
specific error-code ignoring instead of the current all-encompassing
"type: ignore" comments, but there's currently an issue with that:
https://github.com/python/mypy/issues/7562
This uses Stripe's Subscriptions capability to set up recurring
donations. Requires setting up a Product for the recurring donation, and
defining its product ID in the INI file.
This seems to be a separate Semrush bot that doesn't listen to
robots.txt for the main SemrushBot UA. Their site also says to block
"SemrushBot-SA" and doesn't mention "-BA", but I don't know if that's a
mistake on their end or if this won't work.
Alembic 1.2.0 adds the ability to post-process revision scripts, so this
uses that capability to automatically run any new files through Black.
This always had to be done manually before, so it's nice to have it
taken care of automatically like this.
Deimos
deing
Jason Schwarzenberger