libnpk

When I started looking at Maratis, it had pretty much everything I could want. I really wanted to get stuck in with it and fiddle with it, to get to know how it worked so I could more easily extend it when I needed to later. I spent a while browsing the forums to see what people might find useful, which I could also use.

The answer was data packages. After some discussion with Anael, the creator and owner of Maratis, he suggested libnpk as a potential option for this. I took a look at the library and found it to be very nice to work with, it has a very simple interface and allowed for a lot of flexibility.

So, this became the first of my little tasks in extending Maratis. The package system itself was nice and simple to plug in, however, loading files from it turned out to be a little bit of a headache. The issue here is that any part of the engine could potentially call fopen to open a file. This could be inside a third party library, or within the engine code itself. There’s no real “nice” way to do this. My initial suggestion was to create replacement functions with the same signature as fopen etc and then to patch them into the running executable at run time. This seems pretty hacky, but I think is a quite elegant way to do things, as it would immediately allow for any third party library to load from any location, regardless of whether the library is open or closed source, or whether the data comes from a raw file, within a package, or even across the network.

Unfortunately, there was one main reason that I didn’t do this, and that is that, much as you can do this patching in Windows, Linux and OS X (it’s how debuggers read and write information to a running process) it is forbidden to use in iOS. I guess that’s acceptable, as you could potentially hack the OS with the functionality there. Ah well.

So, the solution adopted was just to create wrapper functions (M_fopen etc) which work identically to their stdio counterparts, apart from that, rather than return a FILE*, they return an MFile*. Maratis Core then implemented a stdio file, which was the default fallback, and the MPackageManager registered itself as a file opener, and would return a MPackageFile*, if there was a file in the package. Simple.

Then there’s my pet hate that libraries tend to just expect you to give them a const char* filename, or a FILE*. Thankfully, the libraries which Maratis use would take a buffer, or in libsndfile’s case, custom file read callbacks. The only one that wouldn’t was tinyxml, which Anael changed to load binary files in published projects, not needing any xml files.

The only thing left was make this usable for people. I added a publish button to Maratis editor which creates a “published” directory, packages all of the data up then copies the libraries and executable. I hopefully made it in a way that would allow more publishing steps to be added from plugins, although I’m not entirely sure at this stage how a plugin will be able to get hold of the maratis editor instance. That’s an issue for another time though.

What’s still needed to be done is to allow users to change the teakey for the package, to allow secure encryption, and to also allow different extensions to be used, rather than .npk. Much as neither are really necessary, it’s nice to be able to protect your data when you distribute.

That just about wraps it up for this. If you have any thoughts or comments about this, give me a shout in the relevant forum thread.

Leave a Reply

Your email address will not be published. Required fields are marked *

Diary of a Slacker