TL;DR: What just happened‽

I got three brand new hard drives for the server with much more capacity. I was running low on storage space, so I decided that I would split my at-the-time current single drive setup into four. I set up one for /, one for /var, one for /srv, and one for /home. Unfortunately, Ubuntu did not like me doing this for /var in particular, and decided instead to wipe both the drive and the backup. Shenanigans ensued.

A Timeline of Events

July 16th, 2026

On the 16th, my three drives arrived in the mail. I had previously put everything on a 600GB SAS drive, and I was starting to run a bit low on space. These new, 1.2TB drives, would allow me to use the other three slots in my server, while also allowing me to expand space significantly. I decided that the best course of action would be splitting the drive into four, as I still had the 600GB one and thus a RAID would be wasting a large amount of space. After making sure that all three drives worked, I put them in and partitioned them. I wanted to put some sort of label on each, marking which one goes to which drive, so I played around with some of the various indicator lights for a bit. After awhile, I found that I could copy the drives to null using dd to get their activity lights going. This was a bit scary, but worked well. I then copied my /home folder to one drive, my /var to a second, and my /srv to a third. This took awhile, though by the time it was done I was ready to mount them. I decided to first test this out with /var. I wanted to make sure that the files I was mounting over weren't still taking space, so I made links to them in my home directory. After that, I mounted the drive over /var. To make sure it worked, I restarted the server. This is when everything broke, horrendously.

I had forgotten that Ubuntu Server in particular does not like it when /var is played around with, in part due to how Snap handles sandboxing. Each package acts as essentially its own partition on the drive, which does not play well with changing partitions. After seeing everything wiped on the new /var drive, I decided to unmount it and go back to the previous /var directory that I had linked in my home directory, as clearly something had gone wrong and I needed to revert. I assumed I would likely also need to reïnstall, though surely with my data on each drive that wouldn't be too much of a hassle. I restarted... and found everything also wiped in the old /var drive. I cannot fathom why this would be the default behavior from Ubuntu, but between this and this LTS kernel bug that I can't seem to get the maintainers to look at, I decided that was enough of Ubuntu Server. Shutting off all I/O on boot on the most recent LTS kernel was strike 1, wiping my /var drive was strike 2, and wiping it a second time was strike 3.

July 17th, 2026

It was clear to me that I needed to take action to salvage what was left of the /var directory, alongside backing up anything that was stored on the server. I used a very lovely tool (ext4magic) to recover all of the files from the /var drive onto my computer, compressed the rest of the files into a tar.xz, then brought it onto my computer. This process took awhile, and did not recover folder names for a majority of what was on /var. I started the process at around 10AM, and it finished up around 6PM. The way I did this was through my Ubuntu Server installation disc; I started a shell, logged in via SSH, then used SSHFS to get the files onto my computer. This worked decently well, though seeing my RAM usage slowly increase as I installed more packages to get this to work was certainly worrisome!

July 18th, 2026

I had been talking to some people that know a bit more about Linux than I do, and they had suggested Rocky Linux as an alternative. I downloaded it, burned the ISO to a disk, and... Rocky Linux doesn't support my CPU configuration. I have two Xeon CPUs that are almost two decades old now, so this was not particularly surprising. I briefly considered just installing Arch on the server, as that's what's on my computer, before deciding that was a terrible idea for obvious reasons. One distro that I knew supported my CPUs is Debian, as Ubuntu is built on top of Debian, and I had been running Ubuntu Server previously. So, I got a second DVD, burned the Debian ISO onto it, and put it into the server. I installed it, and... it didn't boot. Apparently my GRUB partition previously was somehow messing with it, so I did it a second time, and it worked this time! It did however mean that I had to copy everything back from my backup. This is a process I'm still working on, as I find smaller bits that didn't quite make it.

July 19th, 2026

The 19th is when I started copying files back, while also trying to recreate my /var drive. Almost all of the important files were found in either MAGIC-1, MAGIC-2, and MAGIC-3. MAGIC-1 had no folder names, MAGIC-2 had no file names, and I've yet to find a functional file (not counting the plaintext files that don't contain much of use) in MAGIC-3 (though almost all of it seems to be bits that the previous owner of this drive forgot to wipe). Most of the folders in MAGIC-1 were from apt, so those could safely be ignored. I eventually found the Oijon folder, though it was missing all of the most important files, those being the ones for the main page! I had a backup, though that backup was a bit outdated for my liking. The website also relies on a database, I hadn't seen anything resembling a MySQL database yet, and I was not liking the prospects of recreating it by hand. I decided to make a quick little status page that'd explain what was going on. I took most of the CSS from the Debian "It works!" page, and wrote the first entry:

Over the next few days to weeks, the various services here will be returned back to normal. Most of what was lost was backed up, though a few notable exceptions (the Oijon website in particular) is quite outdated. All data outside of the web server is intact. The same cannot yet be said about the MySQL databases. Once the recovery process is complete, a blog post will be made at the Oijon blog.

July 20th, 2026

The missing files for the Oijon website have been found! Unfortunately, the file names were not preserved, however the content of them themselves was. This may take some time to reconstruct, though it appears there should not be any major data loss. Attempts on recovering the MySQL databases are still ongoing, though look promising.

On the 20th, I figured out that the missing files were in MAGIC-2. I suspect this was due to these files being read by apache2 when Ubuntu did its /var wipe, though I can't be certain if this is the case. Either way, I looked at each file, matched it to where I remember its file name going, and got a working-ish PHP website. The main issue was the MySQL database. Most of the content of the website is stored on there, and I really, really did not want to remake it by hand. I decided to do a search via baloo for the file extensions needed, however my backup directory was too large, and after about 12 hours of searching Dolphin crashed due to using too much memory. This was not ideal, and I feared that perhaps parts of the database would end up in MAGIC-2 or MAGIC-3. I looked through them and found no trace of the database. I found it incredibly unlikely that the drive that had just about nothing overwritten would've just had the entire database overwritten and nothing else, so I assumed that must mean that the database was somewhere in MAGIC-1.

July 21st, 2026

Databases have been restored! All that now remains are some services that are not user-facing. The Oijon website will be up shortly!

Eventually, I found the database. Turns out it was in a folder that I had skipped on accident. Finding the database fully intact, I went and put it into the MySQL directory on the server. This was rather scary, as the Ubuntu version of MySQL server is several versions behind the Debian one, and I was not certain they'd be compatible. Thankfully, they were! The database was restored with no data loss. Once I got this up, I got to work getting this website public-facing again. There were a few things that I needed to fix, apparently I had been enabling modules in apache2 incorrectly before, leading to some with null configurations. This was not particularly ideal, but was fixable. I used apt to download apache2 into the cache, brought it to my user directory, extracted it, then got the missing files. I didn't want to reïnstall the entire thing outright, as if I had done that, I would've lost my configs. Granted, I know there's at least 7 backups of those, but I don't remember off-hand where those backups are. Once I had that set up, I moved over the final missing file of the server (downloads), and brought it back up. There's still some things down, but I'm the only one impacted by those services. Soon all will be back to normal.

Some takeaways

Much of what happened here was due to there not being any (recent) off-site backups. Though I had backups, they were on the server itself, which was running low on space. I need to find some sustainable way to make off-site backups. Having them on my computer is not particularly sustainable, as I am also running low on space there. I'm eyeing a few servers on eBay that are very affordable, that if I get some SAS drives in, could work for this.

This was not a particularly fun experience. I hope this never happens again. That said, the drives I have will someday fail. That is something I need to prepare for, with off-site backups.]]>Tue, 21 Jul 2026 05:48:00 -0400https://oijon.net/blog?post=13Let me start this post by saying that I am not a cybersecurity expert! As can be seen on this website, I mainly make software, usually software that's somehow related to linguistics. This is a bit of a detour from what I usually talk about, but I feel it's both important enough and not publicly known enough that I should post it here. If you're from Higher Logic, I'm glad I have your attention now, please reply to my email I sent two months ago!

TL;DR, what's the vuln?

In vanilla/nbbc specifically, custom tags do not properly sanitize input, allowing for malicious JavaScript to be inserted and run by merely loading the page it's on.

Where it all begins

Around two months ago, I woke up to an @everyone ping in the Art Fight Discord server. Seeing how this was before this year's event started, I thought perhaps it was the theme reveal or maybe some pre-release merchandise. What I found instead was a rather vague announcement saying there was a "recent security issue" and to check this newspost. Knowing that the Art Fight community isn't the most technical, I decided to watch #general to see some... interesting... takes made by users, such as "You can check if your account's been affected by checking your Gmail settings" and "XSS attacks give access to all of your passwords and email account". Mildly entertained and now a bit more awake, I got into a voice call with a friend of mine and read the full announcement. Reading it worried me quite a bit, as I quickly realized that Art Fight used PHP and BBCode, both of which are used on this website! Once the more in-depth announcement was posted, I noticed the owner in #general, and started to ask how I could get into contact with the developer team, not expecting a response due to how fast chat was moving at the time. However, this surprisingly got me in contact with the head developer! When talking to him, I quickly learned that Art Fight was using vanilla/nbbc as their BBCode parser, and no CVE exists for this. I also recieved the malicious comment in question, the JavaScript meant to steal login credentials replaced with alert(1):

[imgleft]http://<>deleteme9012#[url=' onerror=this.remove();alert(1)//'][/url][url]http://deleteme9012.com/<>?a=[/img][img]http://deleteme9012.com/<>?a=1'"a[/imgleft][url=http://deletme9012.com/<>] [/url][/url]

It should be noted that neither "deleteme9012.com" nor "deletme9012.com" are registered domains (at least at the time of testing), causing the image loading to always fail, thus always triggering the onerror attribute.

Testing

After getting some tea and searching for existing CVEs, I started testing. Starting with the presumption that this is applicable to all [img] tags, I tried yet failed to reproduce this vulnerability with nbbc. I decided to come back to it, and decided to test with some other BBCode parsers. I had a hunch that this would work with most other BBCode parsers, but as my experience with BBCode is mostly limited to phpBB, I simply searched up "php bbcode parser" and tested the ones that came up. Once I got it to work with genert/bbcode, I realized I'd be testing quite a few parsers, so I set up a KDE vault and continued testing there. Next I tested jBBCode, and got it to work with the following BBCode:

[b]Hello World![/b][img]https://oijon.net/invalid_img.png[/img][img]https://oijon.net/invalid_img.png#\"onerror=\"alert(1);

I got a similar result using chriskonnertz/bbcode, with the following BBCode triggering the vulnerability:

[img]https://oijon.net/invalid_img.png?"onerror="alert(1);" [/img]

After getting these (+ a few more where I couldn't find contact info) working, I decided to focus on vanilla/nbbc once more. Despite my assumption holding true for other parsers, nbbc required some sort of custom BBCode tag in order for this to work. To do this, there's two different modes of custom tags, simple and enhanced. I acquired the vulnerable [imgleft] tag from the head developer of Art Fight, and tweaked it to work for both modes. These tweaked custom tags consisted of the following:

1. $bbcode->AddRule('imgright-simple, Array(
2. 'simple_start' => '<img style="float:right" src="',
3. 'simple_end' => '">',
4. 'class' => 'inline',
5. 'allow_in' => Array('listitem', 'block', 'columns', 'inline', 'link'),
6. ));
8. $bbcode->addRule('imgright-enhanced', Array(
9. 'mode' => BBCode::BBCODE_MODE_ENHANCED,
10. 'template' => '<img style="float:right" src="{$_content}"></img>',
11. 'class' => 'inline',
12. 'allow_in' => Array('listitem', 'block', 'columns', 'inline', 'link'),
13. ));

Using these two custom tags, the following BBCode allows for XSS attacks:

[imgright-simple]https://oijon.net/img/logo.png#[url=' onload=alert(1);//'][/url][/imgright-simple]

[imgright-enhanced]https://oijon.net/img/logo.png#[url=' onload=alert(2);//'][/url][/imgright-enhanced]

What's interesting here is that, in the built-in [img] tag, [url] tags are automatically moved outside the [img] tag, thus preventing this kind of vulnerability from occuring. After all this, I went and contacted each maintainer, gave each a summary of the issue, and awaited replies. Unfortunately, not a single maintainer bothered to reply to my emails.

Response from maintainers

There are a few parsers not listed that I was able to reproduce this vulnerability, however could not find any contact details for the maintainers. As I technically have not disclosed the vulnerability to those maintainers, I will not say which parsers are in this category.

Overall, I am incredibly disappointed at the state of BBCode parsers. Out of all the maintainers I contacted (one of which being a rather large company), not a single one bothered to either reply back or attempt to fix the issue. Perhaps this is simply because BBCode is not as popular as it once was, but surely one should actually maintain their project if there's no indication that it's no longer being maintained. I am currently in the process of getting CVEs registered, I hope that process goes well. Perhaps I'll make my own parser, but if I end up doing that, it'll be quite awhile before I actually get around to that.

From my understanding, when a vulnerability is being used in-the-wild, it's best to disclose it 45 days after disclosing it to the maintainer. I decided to give the maintainers a bit more time, hoping that at least one of them would respond, but that does not seem to be the case. As of today, it's been 66 days since I contacted maintainers about this issue. To me, this is an issue of severe neglect among most BBCode parsers. When software is neglected, eventually vulnerabilities appear. I can only hope that no other website is impacted by this.

Timeline

All times in Eastern Daylight Time

• 5/24/2025:
  • 5:00am: An attacker takes advantage of this vulnerability on Art Fight.
  • 12:11pm: The malicious comment is removed from Art Fight.
  • 6:00pm: The attack is patched on Art Fight via an undisclosed method specific to Art Fight and not applicable to the parser.
• 5/25/2025:
  • 11:40pm: A preliminary announcement about the attack is made publicly on the Art Fight Discord.
• 5/26/2025:
  • 12:20am: A more in-depth announcement is made, with some frequently asked questions.
  • 1:57am: I get in contact with Art Fight's head developer.
  • 3:34am: I get a similar vulnerability working with genert/bbcode.
  • 4:11am: I email the developer of genert/bbcode.
  • 5:29am: I get a similar vulnerability working with jbowens/jBBCode.
  • 7:01am: I get a similar vulnerability working with chriskonnertz/bbcode.
  • 8:35am: I email the developer of chriskonnertz/bbcode.
  • 8:54am: After some digging, I email the developer of jbowens/jBBCode.
  • 5:12pm: I am able to reproduce the vulnerability with vanilla/nbbc.
  • 5:55pm: Per nbbc's security policy, I open a security advisory on their GitHub.
• 5/31/2025:
  • 11:41pm: After getting no response on the GitHub security advisory, I send a follow-up email to Higher Logic
• 7/30/2025:
  • I accept that none of these maintainers are going to either reply or fix the issue.
• 7/31/2025:
  • I write this blog post.

Anyways, Cliffside's essentially the localization update. Localization should be (mostly) functional now, so it should be possible to translate the program into any language (yippee!). I have more localization features planned that will be added over time, but for now I think I'm going to try to find a way to streamline these releases a bit more...

I used LaTeX to write something longer than a page for the first time today, and it was quite enjoyable. At first I was worried about things being more painful than they needed to be, but it worked out well. I think in the future, I'll make manuals for all software here aimed at an end-user. I think that javadocs would be more useful to a developer than a manual, so I'll refrain from making manuals for things like OLing (at least for the time being). I'm especially happy with how the figures turned out, as I personally think they make the text flow much better. I went ahead and printed out the manual, mainly because I think it's neat to have a physical thing from something I've been working on. I made the .tex file for it a part of the repo, so other people can add and change things to the manual as they see fit. I'm not sure how LaTeX handles other authors, but I hope that everyone who contributes can add their name as authors to the manual! The next thing I plan on doing is letting the manual display in the application itself. Susquehanna already has the UI of a book, so having each page of the PDF take up a page in the book UI would be ideal. I think that'll be for the next iteration though.

This is a Minecraft mod that I made for a good friend of mine for a project we are both working on. None of the items added are obtainable in survival, and as such this mod will not be all that useful without creative. The main reason why this mod is not open source is because it is meant for a project that has yet to be released. This mod adds four gems, and corresponding blocks. These are the red gem, yellow gem, green gem, and blue gem. More will probably be added to the mod as the project progresses!

Apologies about not posting about 0.0.2! I just completely forgot to make a devlog entry, at least I remembered this time! Here's what's new:

• New icons!
• Lexicon is now editable! Now you can add word to your conlang!
• Phonology tab revamp! I noticed that it was impossible to add diacritics with 0.0.1 and 0.0.2, so I fixed that!
• Sounds are now removable from phonologies!

I don't really have much else interesting to share, I plan on making the lexicon tab have more fields that are editable, as right now the logs start yelling about every word. There's also currently a bug where homonyms and synonyms will be added to a word multiple times, so I'll have to fix that before adding more fields to display in the Lexicon. I plan on getting the Orthography tab up and running before I release 0.0.3, so stay tuned for that! Looking forwards, I plan on adding the Grammar tab to 0.0.4, and then after that just keep adding new features to each tab. I'm hoping that by the end of 2023, Susquehanna will be a fully functional, all-in-one conlang manager!