Skip to main content

September 24

Hi all, hope you're doing well.

My health is particularly awful at the moment, so I have been struggling to get much of anything done in the last week or so, but updates are below as usual.

After doing the performance tests on the GetGalleryForDisplay procedure in the last update, I ran tests on 60,000 artwork records to test the performance of the procedure at higher scale (in this instance, using a base of 10,000 users).

I've made a lot of improvements to the procedure's performance and fixed a few bugs. It's never going to be mega fast, but given I'm testing on the smallest database server possible, a fraction of a second for most searches and ~1-1.5 seconds for more specific searches isn't terrible. I might need to consider relegating things like tag searches to the client-side code after results are returned, to further reduce the database load, but that needs further thought.

Until now, the system for gallery loading has been to load a given number of results from the server, and when the user has scrolled to the bottom of the gallery, to request the next set.

The time taken to run the GetGalleryForDisplay procedure doesn't change much with higher LIMITs, so it makes sense to request a higher number of artworks from the server each time. To prevent a huge number of artworks loading on the user's screen at once, the AJAX calls are now double-buffered: 100 results are requested from the server, and then those are displayed in groups of 25, with the next group being displayed after the user scrolls to the bottom of the gallery.

The result is that to the user, this seems identical to the previous system, but requires less AJAX calls, lower database load and faster loading when results are available to load client-side.

Notification improvements

I've made a lot of progress on the main notifications page and notifications dropdown, to properly display and format notifications, including notifications that are grouped together such as follows.

The styling still needs some work, particularly for the dropdown notifications; the borders are primarily there for development clarity at the moment, but they probably need to look nicer, alongside perhaps aligning the notification text to the same horizontal position for each notification so it's easier to read. The other things left to do are to add an option to the user_navigation_settings or user_notification_settings tables, to allow users to disable the filtering of artworks in their notifications, and to also enable filtering of artworks in the notifications dropdown. 

image.png

image.png

As part of fixing the remaining gallery client-side code bugs, I realised that making cropped versions of artworks for search results purposes isn't needed (e.g. for an image with a very distended aspect ratio). It can be implemented by adjusting the amount of the image visible on the screen; I initially didn't want to do that as it seemed inefficient, due to loading a larger file than necessary, but this is effectively forced for videos where creating a resized version of the video would be prohibitively expensive.

I haven't yet removed the relevant columns from the artwork tables, but I'll need to do that once I've checked for where those columns are still used.

Text Moderation: testing and improvements

AWS' AI text moderation does what it's supposed to do, but it will take time to adjust the thresholds for each category to work properly when screening artwork comments. The TOXICITY value gets very high for comments with profanity or sexual content, which in the context of an art site which allows NSFW content, isn't useful: e.g. a comment like "HUGE FUCKING TITS" is not toxic or abusive in an art site context when it might be in another website's context.

As such, I've made separate configurable threshold values in the configuration database table for each of the scores that are relevant - the HARASSMENT_OR_ABUSE, VIOLENCE_OR_THREAT, HATE_SPEECH and INSULT scores. For now I've set those to 0.5 for testing purposes (50% confidence), but more than likely those thresholds will need to be set higher to avoid flagging up non-problematic comments.

Besides the bug fix to the bookmarks gallery I mentioned earlier in this update, I also implemented an extra option for bookmark folders: choosing an icon to associate with the folder.

This isn't primarily aimed at being "flashy", but at helping users have an easier time visually distinguishing between their different bookmark folders, rather than all of them having a "Bookmark" icon. I still need to implement modal prompts for editing and deleting bookmark folders, and managing the bookmarks contained with them in an easy manner.

image.png

CDN error pages

I've mostly finished up the error pages for CDN files, primarily for the case when a file doesn't exist or is not publicly accessible. These error pages are slightly different from the webserver ones, since they don't have login information, but are broadly similar beyond that. I had to debug a weird 428 HTTP error when building them, but it turned out it was due to some BunnyCDN rate limits for the error pages CDN I had set and then completely forgotten about.

The last thing to do with these, besides polishing them up slightly, is to see if there are edge rules I can modify. At present, because of the Referer header requirements, refreshing a page like an image hosted on the CDN goes to the error pages, since the Referer header is no longer present. I've filed a support ticket with Bunny to ask them about this, as I can't find a way to whitelist empty Referer headers with their current configuration dashboard.

Misc

Minor Improvements / Bug Fixes
  • Fixed cropped versions of images for galleries having distended aspect ratios, causing the cropped image's container to have the wrong dimensions.

  • Fixed like and reblog galleries incorrectly filtering artworks when they aren't supposed to.

  • Fixed galleries with non-empty prefixes not properly moving elements up and down in their div arrays, due to a faulty ID selector in the deltaMap processing method. What an elusive bug that was.

  • Implemented cropped versions of additional images and videos in an artwork beyond the first, so that the buttons to view the extra media in the gallery area for that artwork function properly.
Documentation Updates
  • Documented how the Config classes and its associated classes work, where their values are stored in the database, the syntax and their caching policies.

  • Added documentation about the GetGalleryForDisplay procedure and its optimisations.
Back to top