Sunday, January 22, 2023

Book review: Riddley Walker


This is a rather unique book. It tells the odyssey of the eponymous Riddley Waler who lives in a far future dystopia after a nuclear war reduced human civilization back to the iron age. Essentially, a series of events happen that lead to the protagonist being forced to leave his community where he interacts with the larger post-apocalyptic society he finds himself in and its various political, mythological and philosophical elements, stumbles upon the re-invention of gun powder, and in the end becomes a traveling story teller.

Plenty of books have been written with the premise of nuclear war destroying civilization. What makes this book unique is the writing style. To signify that significant time has passed, all the spelling in the book is non-standard. For example, here is a quote:

The worl is ful of things waiting to happen. Thats the meat and boan of it right there. You myt think you can jus go here and there doing nothing. Happening nothing. You cant tho you bleeding cant. You put your self on any road and some thing wil show its self to you. Wanting to happen. Waiting to happen. You myt say, 'I dont want to know.' But 1ce its showt its self to you you wil know wont you. You cant not know no mor. There it is and working in you. You myt try to put a farness be twean you and it only you cant becaws youre carrying it inside you. The waiting to happen aint out there where it ben no more its inside you.

As you can imagine, this is pretty frustrating to read a times. Sometimes I think something means something, and discover chapters later that I misinterpreted it. I suspect if I read it a second time I would get a lot more out of it, but I also don't know I want to. At the same time I'm glad I read it at least once. It does feel quite different from any other books I've read.

One of things I like most about this book is the world truly feels alien. Too often science fiction books have aliens with superficial differences that feel basically the same as any modern day western culture. Although these are of course not aliens, I really did feel that this book depicted a society very different from our contemporary society. It wasn't just America with one quirk changed.

A large part of the book deals with the myth around "Eusa". I originally thought that this was talking about the United States and nuclear war, but it appears it is actually about St. Eustace. Or perhaps the intention is to be both? It is certainly interesting when reading this book how you can think one thing and then end up re-evaluating it all later.

The most closest comparison to another book is probably certain scenes in Cloud Atlas, although it seems that David Mitchell was taking very direct inspiration from this novel. However if you ignore the unique language, I think The Shadow of the Torturer is kind of similar. Which is interesting, as I didn't really like that book, but I got the same sort of feel from it. I suppose its the journey through a different world vibe that is similar.

In conclusion, certainly an interesting book but also a very frustrating book. I liked it but I don't think I would ever want to read it again.

Friday, January 20, 2023

The Vector-pocalypse is upon us!



Yesterday, a new version of the Vector skin was made default on English Wikipedia.

As will shock absolutely no one who pays attention to Wikipedia politics, the new skin is controversial. Personally I'm a Timeless fan and generally have not liked what I have seen of new vector when it was in development. However, now that it is live I thought I'd give it another chance and share my thoughts on the new skin. For reference I am doing this on my desktop computer which has a large wide-screen monitor. It looks very different on a phone (I actually like it a lot better on the phone). It might even look different on different monitors with different gamuts.

So the first thing that jumps out is there is excessive whitespace on either side of the page. There is also a lot more hidden by default, notably the "sidebar" which is a prominent feature on most skins. One minor thing that jumps out to me is that echo notifications look a little wonky when you have more than 100 of them.

On the positive though, the top bar does look very clean. The table of contents is on the left hand side and sticky (Somewhat similar to WikiWand), which I think is a nice change.

When you scroll, you notice the top bar scrolls with it but changes:

On one hand, this is quite cool. However on reflection I'm not sure if I feel this is quite worth it. It feels like this sticky header is 95% of the way to working but just not quite there. The alignment with the white padding on the right (I don't mean the off-white margin area but the area that comes before that) seems slightly not meeting somehow. Perhaps i am explaining it poorly, but it feels like there should be a division there since the article ends around the pencil icon. Additionally, the sudden change makes it feel like you are in a different context, but it is all the same tools with different icons. On the whole, I think there is a good idea here with the sticky header, but maybe could use a few more iterations.

If you expand the Sidebar menu, the result feels very ugly and out of place to me:

idk, I really hate the look of it, and the four levels of different off-whites. More to the point, one of the key features of Wikipedia is it is edited by users. To get new users you have to hook people into editing. I worry hiding things like "learn to edit" will just make it so people never learn that they can edit. I understand there is a counter-point here, where overwhelming users with links makes users ignore all of them and prevents focus on the important things. I even agree somewhat that there are probably too many links in Monobook/traditional vector. However having all the links hidden doesn't seem right either.

On the fixed width

One of the common complaints is that the fixed width design wastes lots of screen real estate. The counter argument is studies suggest that shorter line lengths improve readability.

As a compromise there is a button in the bottom right corner to make it use the full screen. It is very tiny. I couldn't find it even knowing that it is supposed to be somewhere. Someone had to tell me that it is in the lower-right corner. So it definitely lacks discoverability.

Initially, I thought I hated the fixed-width design too. However after trying it out, I realized that it is not the fixed width that I hate. What I really hate is:

  • The use of an off-white background colour that is extremely close to the main background colour
  • Centering the design in the screen

 I really really don't like the colour scheme chosen. Having it be almost but not quite the same colour white really bothers my eyes.

I experimented with using a darker colour for more contrast and found that I like the skin much much better. Tastes vary of course, so perhaps it is just me. Picking a dark blue colour at random and moving the main content to the left looks something like:


 Although I like the contrast of the dark background, my main issue is that in the original the colours are almost identical, so even just making it a slightly more off-white off-white would be fine. If you want to do a throwback to monobook, something like this looks fine to me as well:

I don't really know if this is just my particular tastes or if other people agree with me. However, making it more left aligned and increasing the contrast to the background makes the skin go from something I can't stand to something I can see as usable.

Sunday, December 4, 2022

Hardening SQLite against injection in PHP

tl;dr: What are our options in php to make SQLite not write files when given malicious SQL queries as a hardening measure against SQL injection?


One of the most famous web application security vulnerabilities is the SQL injection.

This is where you have code like:

doQuery( "SELECT foo1, foo2 from bar where baz = '" . $_GET['fred'] . "';" );

The attacker goes to a url like ?fred='%20UNION%20ALL%20SELECT%20user%20'foo1',%20password%20'foo2'%20from%20users;--

The end result is: doQuery( "SELECT foo1, foo2 from bar where baz ='' UNION ALL SELECT user 'foo1', password 'foo2' from users ;-- ';" );

and the attacker has all your user's passwords. Portswigger has a really good detailed explanation on how such attacks work.

In addition to dumping all your private info, the usual next step is to try and get code execution. In a PHP environment, often this means getting your DB to write a a php file in the web directory.

In MariaDB/MySQL this looks like:

SELECT '<?php system($_GET["c"]);?>' INTO OUTFILE "/var/www/html/w/foo.php";

Of course, in a properly setup system, permissions are such that mysqld/mariadbd does not have permission to write in the web directory and the DB user does not have FILE privileges, so cannot use INTO OUTFILE.

In SQLite, the equivalent is to use the ATTACH command to create a new database (or VACUUM). Thus the SQLite equivalent is:

ATTACH DATABASE '/var/www/html/w/foo.php' AS foo; CREATE TABLE (stuff text); INSERT INTO VALUES( '<?php system($_GET["c"]);?>' );

This is harder than the MySQL case, since it involves multiple commands and you can't just add it as a suffix but have to inject as a prefix. It is very rare you would get this much control in an SQL injection.

Nonetheless it seems like the sort of thing we would want to disable in a web application, as a hardening best practice. After all, dynamically attaching multiple databases is rarely needed in this type of application.

Luckily, SQLite implements a feature called run time limits. There are a number of limits you can set. SQLite docs contain a list of suggestions for paranoid people at In particular, there is a LIMIT_ATTACH which you can set to 0 to disable attaching databases. There is also a more fine grained authorizer API which allows setting a permission callback to check things on a per-statement level.

Unfortunately PHP PDO-SQLITE supports neither of these things. It does set an authorizer if you have open_basedir on to prevent reading/writing outside the basedir, but it exposes no way that I can see for you to set them yourself. This seems really unfortunate. Paranoid people would want to set runtime limits. People who have special use-cases may even want to raise them. I really wish PDO-SQLITE supported setting these, perhaps as a driver specific connection option in the constructor.

On the bright side, if instead of using the PDO-SQLITE php extension, you are using the alternative sqlite3 extension there is a solution. You still cannot set runtime limits but you can set a custom authorizer:

$db = new SQLite3($dbFileName);
$db->setAuthorizer(function ( $action, $filename ) {
        return $action === SQLite3::ATTACH ? Sqlite3::DENY : Sqlite3::OK;

After this if you try and do an ATTACH you get:

Warning: SQLite3::query(): Unable to prepare statement: 23, not authorized in /var/www/html/w/test.php on line 17

Thus success! No evil SQL can possibly write files.

Thursday, November 24, 2022

TV Show review: Stargate SGU

 If I could sum up this show in 2 words, I think it would be "wasted potential". There's a lot I really like about this show but the writers play it way too safe and it never really seems to come together into something truly interesting.

This was the third spin off in the Stargate Franchise, and if the original stargate TV show is Star Trek The Next Generation, and Atlantis is DS9, then this would be the franchise's Star Trek Voyager.

And it has a very similar set up to Star Trek Voyager: They get flung half way across the universe and cannot get back. They aren't really prepared for the mission. There are enemies who (eventually) get stranded with them who they don't trust but nonetheless integrate into the crew.

The other show that seems an obvious influence would be Battlestar Galactica (BSG). I'd even go as far to say that this show is essentially what would happen if Voyager and BSG had a baby that dressed up as Stargate for halloween. You can especially see the BSG influence in where the drama of the show is focused. Voyager was very much alien of the week. SGU focuses inwardly on its main cast and their struggles, in a more serialized fashion. The crew don't trust each other. There are cliques. They are stressed. They struggle with their emotions. The civilians and the military mistrust each other, just like in BSG. Additionally, BSG's Baltar was clearly an influence on how the lead scientist Dr Rush was depicted.

However, unlike BSG, this show doesn't really commit to being serialized, and as a result the characters never really grow. Any time something interesting happens to change the status quo, it gets reset in the next 2 or 3 episodes. For example, two of the character's dead girlfriend gets resurrected as a computer program in the ship - then 2 episodes later a contrived situation happens where they have to be "quarantined" in AI jail, never to be seen from or thought of again. Plots like this are common, where something happens that implies the characters will have to change and adapt, but just as you're excited to see how that plays out, the status quo ante is restored. Nothing ever seems permanent and you don't get the pay off for teased change. The worst example is probably Col. Telford, who switches from being obnoxious, to evil, to good, is marooned but comes back, is killed but then cloned in an alternate time line, etc. The character gets swapped around so much it is simply ridiculous.

In many ways, one of the best plot lines in the show, involves having an alternate timeline of the crew be sent back a thousand years, and the main timeline crew meeting their descendants and being shown archival footage of their alternate selves from a thousand years ago. This allowed the writers to show what might have been for these characters, and it was the most compelling character development in the show. I suppose the writers felt safer making bold choices with these alternate versions of the character, since the real characters didn't necessarily need to abide by them. However I can't help but think what a great show this would have been if this type of character development took place throughout.

Ultimately, this show felt like it didn't quite know what it wanted to be. It used patterns from both serialized and episodic TV shows, resulting in something that was a bit in-between which satisfied neither. It teased complex characters, but mostly failed to commit to actually developing them, instead playing things safe. Most frustrating of all, at times it did do interesting things, and you could see the potential. By the end, I really did like this characters, and wished I knew more about them. Thus why I think "wasted potential" is the best descriptor for this show.

Sunday, September 11, 2022

Why don't we ever talk about volunteer PMs in open source?

 Recently, on Wikipedia, there was an open letter to the Wikimedia Foundation, asking them to improve the New Page Patrol feature.

This started the usual debate between, WMF should do something vs It is open source, {{sofixit}} (i.e. Send a patch). There's valid points on both sides of that debate, which I don't really want to get into.

However, it occurred to me - the people on the {{sofixit}} side always suggest that people should learn how to program (an unreasonable ask), figure out how to fix something, and do it themselves. On the other hand, in a corporate environment, stuff is never done solely by developers. You usually have either a product manager or a program manager organizing the work.

Instead of saying to users - learn PHP and submit a patch, why don't we say: Be the PM for the things you want done, so a programmer can easily just do them without getting bogged down with organizational questions?

At first glance this may sound crazy - after all, ordinary users have no authority. Being a PM is hard enough when people are paid to listen to you, how could it possibly work if nobody has to listen to you. And I agree - not everything a PM does is applicable here, but i think some things are.

Some things a volunteer person could potentially do:

  • Make sure that bugs are clearly described with requirements, so a developer could just do them instead of trying to figure out what the users need
  • Make sure tasks are broken down into appropriate sized tickets
  • Make a plan of what they wish would happen. A volunteer can't force people to follow their plan, but if you have a plan people may just follow it. Too often all that is present is a big list of bugs of varying priority which is hard for a developer to figure out what is important and what isn't
    • For example, what i mean is breaking things into a few milestones, and having each milestone contain a small number (3-5) tickets around a similar theme. This could then be used in order to promote the project to volunteer developers, using language like "Help us achieve milestone 2" and track progress. Perhaps even gamifying things.
    • No plan survives contact with the enemy of course, and the point isn't to stick to any plan religiously. The point is to have a short list of what the most pressing things to work on right now are. Half the battle is figuring out what to work on and what to work on first.
  • Coordinate with other groups as needed. Sometimes work might depend on other work other people have planned to do. Or perhaps the current work is dependent on someone else's requirements (e.g. new extensions require security review). Potentially a volunteer PM could help coordinate this or help ensure that everyone is on the same page about expectations and requirements.
  • [not sure about this one] Help find constructive code reviewers. In MediaWiki development, code much be reviewed by another developer to be merged in. Finding knowledgeable people can often be difficult and a lot of effort. Sometimes this comes down to personal relationships and politely nagging people until someone bites. For many developers this is a frustrating part of the software development process. Its not clear how productive a non-developer would be here, as you may need to understand the code to know who to talk to. Nonetheless, potentially this is something a non-programmer volunteer can help with.

To use the new page patrol feature as an example - Users have a list of 56 feature requests. There's not really any indication of which ones are more important then others. A useful starting point would be to select the 3 most important. There are plenty of volunteer developers in the MediaWiki ecosystem that might work on them. The less time they have to spend figuring out what is wanted, the more likely they might fix one of the things. There are no guarantees of course, but it is a thing that someone who is not a programmer could do to move things forward.

 To be clear, being a good PM is a skill - all of this is hard and takes practice to be good at. People who have not done it before won't be good at it to begin with. But I think it is something we should talk about more, instead of the usual refrain of fix it yourself or be happy with what you got.

p.s. None of this should be taken as saying that WMF shouldn't fix anything and it should only be up to the communities, simply that there are things non-programmers could do to {{sofixit}} if they were so inclined.


Sunday, August 21, 2022

Weekly roundup - aug 21

 Some things i read or saw this week that i thought were interesting.

Natural perspective

I found this blog post fascinating. Basically it talks about how human preception is different than what a photograph would be. For example if there is a big object of interest in the distance it usually looks larger. I never thought much about this before,  but now that it has been pointed out to me, it rings very true. 

Kill the hero save the (narrative) world

This was a talk fron the GDC conference, by Hannah Nicklin, who is the narrative lead of a video game called Mutazione, talking about the narrative structure of video games. Essentially the speaker was arguing that many video games follow a hero's journey type of plot where the story follows a protagonist's journey to becoming a hero. They feel that this is a structure that works really well in movies: the director can direct your focus to character traits and growth. The 2 hour length is also very suitable to developing a single character's journey. They argue that video games would be better suited to follow the structure of an emsamble cast tv show. They think that this allows a better balance between players being able to do whatever they want but still getting the plot across as the focus is less on the affect of actions on the main character's phsyche and more driven by characters interacting with a community of other characters.

I know very little of video games, so i don't know how true the premise rings. However i found the reasoning quite interesting, and it gave me a lot to think about.

Galatea, Versu, Character Engine

This was a talk from 2018 by Emily Short about making non playable characters in video games feel like real people with inner lives. She also talks a bit about the pros and cons of interactive fiction, which i have always find interesting.

Saturday, August 13, 2022

Book review: Reamde



It has been a while since I have read a Stephenson novel, so when I saw this 1044 pages of light reading on the library shelf, I thought I would give it a go.

While I still enjoyed this novel, overall it was one of my least favourite Stephenson novels that I have read. I think perhaps the type of novel this was - essentially a thriller set in basically our current world - showcased Stephenson's weaknesses and hid his strengths, as a writer.

Stephenson is pretty famous for writing giant novels, and this is no exception. However usually it feels like a lot more is going on to justify the length. In this novel, the plot seems much more straight forward, and instead of using the pages to explain complex concepts or explore the world, we instead get 150 page long car/boat chases.

I think fundamentally the reason why Neal Stephenson novels are usually really fun, is their setting. He essentially writes novels set in various nerd utopias. The difference between him and most other science fiction writers writing in utopia settings, is most of those types of novels spend a lot of time arguing for the utopia or telling you how good it is. In Neal Stephenson novels, there is less of that, and the novel is more just set there. We might see how much a character likes the setting through their eyes, and how excited they are by it, but it still seems less directly aimed at the reader. I think this eases suspension of disbelief, because it feels less preachy and allows one to get absorbed in the "coolness" of the fantasy without thinking about all the ways it falls apart. For example, both Ananthem and The Big U are romanticized versions of university life (albeit romanticized in opposite directions), Cryptonomicon is a romanticized version of the cypherpunk movement, Snow crash is kind of romanticized cyberpunk. [I read these a long time ago, I might not be remembering them that well]

In this novel, the only romanticized aspect, is a world where everyone thinks MMORPGs are really cool. Unfortunately, a world where slightly more people play MMORPGs, is a lame utopia.

Additionally it has very little to do with the plot. It does provide the inciting incident, where a ransomware computer virus infecting the MMORPG encrypts some mobsters' private files. However this felt arbitrary - the MMORPG aspect really didn't matter.

Even worse, we spend hundreds of pages on this MMORPG world and the ongoing re-alignment of the in-world factions (The so-called "War of realignment"). Just as it seems to be building to something, we cut away to the main plot, never bothering to resolve or revisit what was building. This is especially sad, because in a lot of ways, it was more interesting than the main plot. Instead it is relegated to being a rather trite metaphor for a theme in the main plot about people coming together to form a chosen tribe over whatever situations they were born into.

Perhaps that's the biggest disappointment in this book: The threads don't really come together to form something bigger. There are a bunch of interesting premises at various points that don't really seem to contribute much to the overall whole. There is the design of the MMORPG/War-of-realignment, which is talked about a lot, but doesn't really go anywhere. There are the motorcycle gangsters with katanas, which are introduced as a back-story for a side character, who just ends up being shot with no katana fights. There are the main character's relatives who are some sort of fundamentalist christian gun-nut cult, but when the action comes down they mostly just hide in their house like normal people.

Why bother with all these tantalizing details when everyone in the end just acts like a normal person. Their uniqueness seems unnecessary to the novel, which feels like a major let down.

Last of all and pretty unrelatedly, the female characters felt like they were written weirdly, in a hard to define way. The main character, Zula, spent most of the novel kidnapped, and there was a bizarre amount of discussion on the differing levels of chivalry her various kidnappers showed her. As if holding the door open for her somehow makes up for the whole kidnapping at gunpoint. Several characters debate whether Yuxia is a "sister" or girlfriend material, where "friend" or "person they just met last week under very traumatic circumstances" apparently didn't cross anyone's mind. All the romantic pairings seemed kind of random and forced (Except maybe Olivia & Sokolov). Yuxia had barely even exchanged any words with Seamus before suddenly being together. Csongor and Zula also had pretty limited interaction before falling in love.

In the end, i did enjoy it, but definitely not his strongest novel.