Both the initialize_db script and Alembic need to import all
DatabaseModel subclasses so that DatabaseModel.metadata has all the
models attached. Previously this wasn't being done properly, and the
intialize_db script wasn't creating the scraper_result table since it
hadn't been imported.
This commit creates a dedicated module to import all of those classes,
so that both those locations can simply import * from it instead of
needing to import all the models individually. This still isn't great
overall, but it should be less prone to mistakes at least.
Solarized's green color is pretty awful, not really attractive for
Exemplary comments. This switches Exemplary to blue, Joke to green, and
Offtopic to cyan. It also rearranges the order a bit so that the colors
in the labeling menu go in a progression that looks right.
Since the production version is using a default label weight of 0.5,
previously two users would have had to label the same comment Exemplary
for it to actually activate. This will probably be fairly uncommon so we
most likely don't want it to work that way. This reduces it to 0.5 so
that one label will trigger it, but it would probably be better to
specifically set it to the default label weight if that's feasible
somehow.
This is a bit of a hacky way of doing it in a few aspects and definitely
still needs some work, but it's a reasonable start. Specifically, a
major miss is that there's no way to remove an Exemplary label unless
you currently have one available. This should definitely be fixed, but
I'm not sure what might be the best approach yet.
This adds an "exemplary" comment tag which has a few special behaviors:
* Requires a reason, which is visible (anonymously) to the tagged
comment's author.
* Highlights the comment's left side in green
* Acts as a multiplier to the comment's voting
I think we'll still want to add a cooldown or something similar on using
this tag, but this covers the basic functionality for it.
Using orange had a bit of a "this is a bad thing" implication to it (and
was hard to tell apart from the red of Malice). It also makes a bit more
sense to have the more-negative tags (Noise and Malice) to the right.
No functionality impact yet, but changes these two tags into a single
one. The alembic migration is a bit tricky to deal with the potential
constraint violation if a user had tagged a comment with both "flame"
and "troll".
Previously, topic lists filtered down to a tag were using "descendant
of", so for example a listing based around the "ask" tag would show
topics that have the exact tag "ask" as well as any sub-tags, such as
"ask.survey". However, it wouldn't show a hypothetical "example.ask"
tag, where the "ask" was at the end.
This commit updates the method used to utilize lqueries to search for
tags that either start or end with the specified one (which includes
exact full matches as well).
Some (mostly minor) adjustments to the syntax highlighting:
* Take advantage of SCSS nesting
* Switch to using a <code> tag instead of a <div> (requires creating a
custom Pygments formatter class)
* Change "syntax_" prefix to "syntax-" for consistency
* Refactor Bleach attribute-checking function a bit and drop unnecessary
one (the "language-" class gets dropped before sanitization)
Adds a new comment sort named "relevance" (may change the name) and
makes it the default. This sort takes into account the tags that have
been applied to comments and sorts them differently based on them. This
will certainly require a lot more work and adjustments, but for now it
will help push down noise, offtopic, and joke comments.
In the future, this can probably also be used to apply some more
interesting sorting effects, like potentially placing new comments
temporarily near the top of the comment section to help them get a bit
of initial exposure in large threads.
This gives each comment tag a "weight" value, which is a float. This
weight can be set for each user to affect their future comment tags, and
if a user doesn't have a weight set it uses a global default value
that's set in the INI file. This global default is set to 1.0 for
development purposes, since that should make it so that a single tag
will have an effect. It should probably be set lower initially in
production so that single users can't cause tag effects.
Instead of lightening the text of collapsed comments and displaying them
in italics, this shrinks the font size a little but still keeps them at
the normal level of contrast. That should make them more easily readable
while still de-emphasizing them somewhat.
Now that a bit more info (link publication dates) is being shown inside
the topic content metadata sometimes, white-space: nowrap probably isn't
a very good idea and has a chance of causing issues on mobile. Just
letting it wrap should be fine most of the time, though it could look a
little strange when it wraps in the middle of a word count or date.
This uses the CSS :not(:target) selector to make collapsing have no
effect on the "target comment" (the one that's being linked to or
navigated to using the Link/Parent links).
This isn't really a full solution and the collapsing button will still
toggle between + and - when clicked, but it's an improvement anyway.
One of the weird behaviors so far with comment collapsing was that if a
top-level comment was collapsed due to being tagged as "noise", and then
all other top-level comments were also collapsed due to being made up
entirely of old comments, the end result would be all comments being
uncollapsed, including the "noise" one.
This probably still isn't a proper fix for it, but should be better. The
whole system of interactions between different collapsing processes
probably needs re-thinking or this is going to get very confusing as
more "reasons for collapsing" come in.
CommentTree was starting to get pretty messy with multiple staticmethods
that were used to process individual comments, as well as just
monkeypatching on the special attributes needed by comments inside the
tree.
This commit uses wrapt's ObjectProxy to create a CommentInTree class
that wraps Comments and adds the attributes/methods needed by
CommentTree. This can definitely still be improved further, but it's a
decent place to start.
This isn't working very well in a lot of cases, shouldn't be used until
I've got some workarounds for a lot of the issues that I'm finding.
This reverts commit 369f273f8e.
As part of scraping a link, Embedly will often remove tracking vars from
the query, follow redirects, and so on. This will start using the url
returned back from an Embedly result to replace the one that was
originally submitted when it was different (though the original one will
still be kept in the original_url column).
Uses the "published" time from the Embedly data to display the date that
a link was published if it was more than 3 days before the link topic
was submitted.
Not really a big deal, but deleted topics are getting sent back through
this consumer when the clean_private_data script erases their data,
since that changes the markdown and puts them into the topic.edited
queue. There shouldn't be any reason to process deleted topics and
re-add "blank" metadata (0 word count, no excerpt), so we can just skip
them.
By default, sqlalchemy will set the JSON value of null (the string
"null") in JSONB columns if you set them to None. I want it to set the
actual column to NULL, so they need to be defined with the none_as_null
argument set to True like this.
This is being done to be able to support some other collapsing behavior
starting to come in (for example, as an effect from comment tags).
However, it's starting to get pretty ugly at this point. I should
probably try to implement a wrapper class for Comments that are inside a
CommentTree so that I can move methods like this into that class instead
of needing them to be staticmethods on the CommentTree class itself.
This adds a consumer (in prod only) that uses Embedly's Extract API to
scrape the links from all new link topics and stores some of the data in
the topic's content_metadata column.
Previously, any topic processed by this consumer would have its
content_metadata completely replaced. This won't work once other
consumers or processes start being able to set that data, since we don't
know that this one will always run first.
This commit updates the method the consumer uses so that it will keep
any data that's already in the topic's content_metadata column if
necessary. It would probably be good to generalize this method out
somehow so that it can be used in other places more easily.
Previously, user topic tag filters weren't also filtering out any
"descendant" tags when they were hierarchical. For example, setting a
filter on "ask" wouldn't also filter out "ask.survey". This fixes that
behavior, though it's a bit awkward and maybe could be done better
somehow.
Deimos
Petr Stastny
Andrew Shu