January 30

Hi all, hope you're doing well ^^ updates below!

HTML / JS / CSS improvements, continued

Fixed fadeins and fadeouts on profile pages not always working correctly. Also been doing some interface improvements, e.g. allowing enter key to be used for form submissions and other areas to make things easier to use, as some areas don't do this yet.

Started to clean up the styling for the login and create account pages, still in progress.

Profile Pages: Albums

I've started designing the templates for the albums sub-page. The main design decision I need to make here is about whether to use a custom cover image on each album, or make it a requirement to use an existing submission inside that album. More than likely the latter will be easier, but it could also be viable to make a separate artwork category that is specifically for album covers (since in both cases, it needs to have NSFW content categories, etc.)

Login improvements

Implemented the forgotten password and lost 2FA device sections of the login pages, and tested email template documentation and scripts I made earlier. Thankfully worked surprisingly well, cleaned up some redundant areas in those scripts that had options for text templates for emails which aren't used now (only HTML templates are used).

I also need to polish up the email templates themselves at some point to have better styling.

Search Control Panels

I fixed a bug causing video and image exclusions in searches to not work properly, and further cleaned up the gallery code, though more work needs to be done there. Primarily it's that the gallery code stores some variables it doesn't really need to store, like user navigation settings, which should be retrieved from the local cache instead, and there's some repetitive areas of the code that can be slimmed down to make it easier to read.

I've also made the search panel for galleries into a modal that appears over the page when a button on the right is clicked, so it takes up less space and can be easily found when browsing through a page.

Artwork Comments implemented

Comments on artworks are now implemented. Next steps are to implement comment deletion and possibly editing, and also decide on styling for “reply to” comments and how to nest them neatly.

I also need to change the artwork submission code and settings areas to allow for users to enable or disable comments on their artworks, or on a specific artwork.

Hotlinking Protection

To prevent having bandwidth leeched, I configured Deserted Chateau’s CDNs to disallow hotlinking from external websites after running into an article about it on BunnyCDN by chance that shows the configuration setup.

Since this means images and other assets that need to be externally linkable (for example, images shown in emails), I made a new CDN distribution for hotlinkable assets that will specifically only exist for serving those assets.

TOS & Privacy Policy: moved to articles webserver

Maintaining a custom format for articles on the main site is a bit redundant and unnecessary extra work, and it makes more sense for all articles and documents for users (i.e. non-tech stuff) to be in one place. 

Messaging servers: nightmare bug fixed, code improvements

While I was improving the messaging server code, I ran into a weird nightmare of my own making: I redeployed the code having messed with it on the server, and didn't check what I had on there at the time, and it broke in a mysterious manner that gave arbitrary WebSockets errors.

I spent a day and a half investigating just about every part of the network infrastructure that goes between a user and the messaging servers, and found the problem was me using a custom port for the WebSockets protocol. Normally that wouldn't matter, but as all connections are going through CloudFront and WAF, custom ports don't get supported properly. To fix this, I changed the WebSockets servers to use the standard port 443, and disabled all Apache functionality on them. As there is no need for the messaging servers to support HTTP or HTTPS connections, this works fine.

I tested sending private messages through the WAF configuration, and it worked fine. I suspect I didn't get any errors as the TinyMCE editor used for private messages doesn't allow for a lot of custom content: I'll likely need to base64 encode message contents and sanitise them server side, as with profile page contents.

With the code working again, I cleaned up the messaging server code properly: I refactored some methods into a separate class, and documented the necessary changes for making the servers work over CloudFront and WAF in the documentation. I also figured out a way to improve the security of WebSocket sessions, by making sure that when users log out of the webservers, they are also logged out of the WebSockets servers at the same time (which is not as easy as it might seem).

Authenticating a user to the messaging servers means requesting a JSON web token from the webservers, proving the user is logged in. Until now, the user's messaging server session was held in a Redis entry, with a key made from the WebSocket connection ID and WebSocket connection user ID. As Deserted Chateau's user IDs don't need to be private, I realised a better solution, in which the Redis key can be known to both the webserver and the messaging server by way of the JWT.

The new system uses a Redis key formed of the user ID, and 64 random bytes, generated by the webserver, which are part of the JWT. As each user session on the webserver can only correspond to one messaging server session, the webserver can thus unlink the user's messaging server session using the Redis key, which is an extra security layer in case the normal JavaScript logout request wasn't sent by the client for any reason.

AJAX efficiency

Reducing AJAX calls by using browser localStorage (theme settings, notifications etc, which don’t need to be refreshed on every page). Important not only for webserver performance, but also because WAF costs are per million requests, so lots of AJAX requests will increase costs there.

I knew it had to be done eventually, but watching the number of requests in WAF while testing the messaging server problem reminded me of it. Not necessarily aiming for optimal efficiency, but there’s a lot of easy efficiencies that can be achieved.

To that end, I decided to implemented a basic caching system to simplify keeping track of the various session variables that should be cached clientside, using the localstorage-slim library to help with implementing it.

Clientside caching

The client-side caching system I've implemented is fairly simple: by utilising localstorage-slim's abstraction of local storage, there's no more need to convert data to JSON strings, and time-to-live values can be set on localStorage variables. It's all in a CacheManager class, where a list of variables that denote different cached values resides, alongside some helper methods.

This way, requests for localStorage values go through the CacheManager, and each of the session variables Deserted Chateau stores locally has a lookup key, a default TTL, and an AJAX form URL for when the local value doesn't exist and needs populating. Some values don't need to expire, so they don't have a TTL, but in all cases this means a massive reduction in unnecessary AJAX requests.

I'm thinking I might expand this a bit to allow users to customise how often their notification counts are updated, for those users who do not want notifications to update often (the standard is once per 15 minutes). On a side note, here's a graph from AWS WAF showing the total request counts to the test environments over a few hours today.

Stripe subscriptions

I improved the backend code that deals with directing the user to Stripe, so that existing users with expired subscriptions get directed to their existing Stripe customer ID rather than a new one.

So far, all the Stripe webhooks are working correctly at the moment. I'm keeping an eye on them to check for potential errors, and I'll need to review that area of the code more thoroughly before launch as part of normal pre-launch testing.

Navigation Options

Artworks can now be hidden for e.g. blocked users, and/or "hidden" users. The idea of a "hidden" user is that user X can "hide" user Y, so that user Y's artworks don't show up in search results, without having to block or mute them.

I still need to create settings pages for managing the lists of blocked/muted/hidden users, and possibly expand the settings area so that it can have submenus to make it a bit more manageable in future.

NSFW profile banners

I've implemented a way to let NSFW artists use an NSFW banner on their profile, by making it possible for the banner to display a placeholder if a visitor isn't logged in or has the NSFW category of that particular banner set to not display. I'll make a proper demonstration video of that in the next update if I find the energy.

Bug fixes

• Fixed videos not downloading when using download buttons (needed to add an extra edge rule in the CDN distributions)