Technology and Lent

filed under: , , , , , , , , , , , , , , , , ,

4106 days ago

This website

The plugin that provided the email notification feature for this website broke with a recent update to my blog software, and it’s not going to be easy to fix. So I’m not going to fix it.

Of course, that means that the affected people aren’t going to hear that they’re affected. Sorry about that! (I’ll tell personally the few I know.)

In fact, I’m likely to switch to ikiwiki …eventually. Textpattern seems to have lost its mojo, and there have been some long-standing issues with it (like no native tagging support!) that seem unlikely to ever get fixed. And I’m hip to the cool technologies now, so a more infrastructure-like framework (i.e. ikiwiki, with git) for my blog feels like a better answer. And schmonz volunteered to do most of the work :)

That also means I’ll probably abandon tru_tags …more than I already have. There hasn’t been anything to do with it in a long while, at least not that I felt was worthwhile to be done. Most of the features that remain to be implemented require a major refactoring of the core Textpattern code, and that just seems very unlikely to happen (by me or anybody else) any time soon. So hopefully it will remain useful to the people who still use it.

This year’s Lent

I have utterly failed at this year’s Lent give-up. I have been better at going to bed at a reasonable hour, sometimes for days at a time. But I simply can’t do everything I need/want to do in my life with the few hours that leaves me between work, kids, and chores. So sleep will continue to lose to projects – although less-so than it used to. There are some nice perks to getting more sleep – I’m much more on-the-ball and willing to take on mental tasks that otherwise seem hard. But that extra value doesn’t offset the lost value from just not being able to do all the things I need to do.


Speaking of Lent, I also broke a 5-year streak of vegetarianism a week or so ago. Benjamin, Liam, and I had some extremely delicious tilapia, also breaking both boys’ life-long vegetarian streaks. Kristina chose not to participate.

We had a bunch of reasons for deciding to do it. And a bunch of reasons to not do it (i.e. to stay vegetarian). I may blog about all the tradeoffs some day soon, but for now, suffice it to say that it was a very close decision, and I’m not sure what’s next.

I made a web app!

If you recall, I started using SmugMug for my online gallery a few years ago. But when I made the switch, I left behind an old gallery site (on Menalto Gallery 1) that I’ve been meaning to clean up for a long time. It broke a while ago, motivating me to finally migrate off that old software – to ZenPhoto, which had been my long-standing plan. It took a few days getting ZenPhoto to work (when it should have been easy!), but I got it there, and I shut off the old site.

I also started this exchange with the ZenPhoto dev in which I start by being too grumpy and then he finished by insisting that his software simply must be unsupportable for him to support it. Net effect: I had to get off ZenPhoto.

But I had no alternate destination for self-hosting my images. My long-term goal is to migrate the images to SmugMug, but I want to filter them down from “every picture I took during that time period” to “just the best ones, tagged and rated” (like all the other pictures I post to SmugMug). And it will take Nathan-weeks of work to get that done, so it keeps getting put off. So in the short term I just needed a new self-hosting product, and there just aren’t any good alternatives. They’re all either old or ugly or badly designed or some combination of those three.

So I made one myself. I’ve never made a web app from scratch before, but I am quite comfortable in perl, had used Catalyst from a prior job, and I’d heard then that Mojolicious is better. So I tried it.

And wow, was it easy. Probably 8 hours total from “install mojolicious” to “the gallery is up and running on the new software”. That’s only just a little more than I spent trying to get ZenPhoto to work. Many kudos to Mojolicious, perl, and pkgsrc.

Now… ZenPhoto does way more stuff. (TONS more… too much, actually.) And this new software isn’t really ready for someone else to use it. And it has no tests. And it only does one extremely simple thing (i.e. serve nested directories of images, in name-sorted order, with no metadata).

But the code is small, easy to read, and easy to modify. (Roughly 300 lines of code, 115 lines of CSS, and 80 lines of HTML template.) The site looks really good (in my opinion). And it doesn’t require a database – just a directory full of images. And with some app-level caching and the help of Mojolicious’s preforking web server and great documentation for setting it up under apache mod_proxy, it’s about as fast is it could possibly be on my old host and slow network connection.

So ZenPhoto is out and my home-grown software is in. Here’s hoping it doesn’t need maintenance!

Add a comment [1]

Why I switched from Menalto Gallery to SmugMug

filed under: , , , , , ,

5221 days ago

Two posts ago, I described my struggle to figure out how to fit my Menalto Gallery album structure into SmugMug. In a comment to that post, one of the Gallery developers asked why I switched from Gallery to SmugMug. I started to reply in the comments on that post, but the reply got to be long enough that I thought it deserved its own post instead; this is that post.

Note that it’s fairly mind-boggling (even to me) that saw my post and responded to it in a single day; I wonder how he did that?



Most of my experience is with Gallery 1, although I do have a site using Gallery 2. Back when I first started using Gallery 1, it was really the only choice (this was before Flickr, even!) that had the ability to handle a large gallery like mine, had the key features I wanted, and would preserve all my images in original form. So it was really my only choice.

For a long time it did what I needed, and I was grateful for that. I told people about it, and about why I chose it. I know one or two people who ended up using it because of me, but most of my friends and acquaintances ended up using Flickr or similar sites.

But even though I chose it, and was able to use it, I didn’t like it very much. It was clumsy, hard to modify, and too complex. It was just my only real option, so I stuck with it. About once a year, I would try to find something else, and I never could.

So when Gallery 2 was announced, I liked the redesign philosophy, but I thought you all were nuts to try to rewrite a product from the ground up. I was certain that it would be a year or more before Gallery 2 was even close to ready, and in the meantime Gallery 1 support would wane, so I was pretty sure I’d be forced to choose another product. That didn’t quite happen – you folks kept supporting Gallery 1 – but it took a very long time to get Gallery 2 out.

So when it was actually released, I was pleasantly surprised… but then it took an even longer time for Gallery 2 to catch up to some of the basic features from Gallery 1 that I needed. (I forget what they were, now.) So I still didn’t have the replacement for Gallery 1 that I was hoping for.

But eventually Gallery 2 did have the necessary features, so I gave it a try… and it was confusing. Sure, maybe the code was much better this time around, and it was certainly more themeable… but it wasn’t easily themeable, and it was confusing as heck to administer and to teach my users about. So I gave up on Gallery 2, decided to live with Gallery 1, and to search more earnestly for a replacement.

Back then, I had my hopes set on ZenPhoto, but it didn’t quite have everything I needed, either. I certainly liked their “simplicity first” approach, though. It did eventually get to the point where it had all the features I needed, and that was enough for me to install it and start working in earnest to switch over to it.

And then I had a system failure that suddenly forced me to host my gallery on my own local machine, which had me terrified – if my house burned down, with it would go all my pictures. So I had to figure out a better hosting option. I considered renting a virtual server somewhere, but it’s hard to find a low-cost NetBSD host, and I didn’t really want to host on Linux. So I looked around at other gallery options, and found SmugMug.

SmugMug isn’t perfect, but it’s pretty great, and I like that they care (a lot!) about usability, and that they have humans on their support staff, and that they’ll take good care of my pictures. I like that I don’t have to administer the site myself. I like that they are a commercial venture, and are therefore forced to prioritize customer needs first, or fail as a business. I like that the features I find lacking, now, are features that I never even dreamed up, back on Gallery.

So now that Gallery 3 is coming out… it seems somewhat surreal. I first heard about it in an April Fool’s post on ZenPhoto, and when I followed the link to the real announcement of Gallery 3, I really thought it was an elaborate April Fools joke from you folks. I couldn’t believe you’d have the gall to say “when we went back to do it right, we did it wrong, so now we’re going to do it right again!” But it turns out that that’s really what you were saying, and that you really are giving it a third go-round.

Sure, maybe Gallery 3 will be better, and maybe someday I will find a reason to move off SmugMug. If so, I’m sure I’ll look at Gallery 3 (or 5, or 9?) and see if it fits the bill. But my default stance will be wariness – I don’t trust that you’ll ever get it right, or feel that you have gotten it “good enough” to just keep supporting (or evolving) the platform you are already on.

Of course, it now seems ironic that you posted your comment in a post titled Perfectionism, pragmatism, and progress. It looks like we all struggle with the balance between those issues! Perhaps we should both take the lesson from this – that our customers won’t give us many chances to find our balance.

Add a comment [10]

Emergency hard drive recovery: success!

filed under: , , ,

6378 days ago

Waaaay back in April of 2006, my cousin Karen (and her husband David) had a hard drive failure, resulting in the loss of all their digital pictures of their baby daughter Ella. They didn’t have backups, and they didn’t have prints. Computer shops weren’t able to help, and clean-room data recovery was too expensive. Eventually, they gave up on it and sent an email to the family asking if we could send them copies of any pictures we had.

I, being a geek, offered to take a look at the drive and see if I could get anything off of it. They, lacking any better options, sent the drive to me so I could give it a shot.

I dropped the drive into my NetBSD machine and sure enough, the BIOS recognized it. That meant that the drive was still physically working, and that I might have a chance at getting whatever data was left off the drive. I was able to mount the drive (read-only) and read data from it, which meant that there was a good chance of finding at least some of the pictures.

I did a bunch of googling and learned a lot about home-brew data recovery. To sum it up, I learned that you need:

  1. A copy of the dd utility that supports the conv=noerror argument. (netbsd’s dd does)
  2. A handy program that knows how to find images on a raw drive image. (See below)

I was able to make a copy of the bad drive (onto a good drive of mine) using dd and the instructions on this page. There were a lot of bad drive sectors encountered during the copy, which was to be expected given the fact that the drive had failed in the first place, but I was hopeful about finding at least some of the images. And having the drive copy was a big win – it meant that I could work with a copy that wouldn’t get worse if the hard drive took a dive.

The hard part was finding a program that could recover the images. For a long time I thought I was going to have to write my own, and that seemed a daunting task, but I finally found this guy who had this exact same problem and already wrote this program. It was exactly what I needed – a program that would find and extract images from a raw drive!

The only problem was that jpg-recover was running at about 15kb/s. At that rate, getting through the 80gb hard drive would have taken about 60 days to finish. I didn’t have 60 days (without interruption!) to wait.

So I dug into the code and discovered that it was horribly innefficient. It was reading data one byte at a time, checking for an image after each byte read, and just generally not being smart about performance. I set about improving it.

I was able to do so. My version takes more memory (a configurable amount) but it runs much faster: at about 12000kb/s. That’s 800x faster :) At that rate, it only took about two hours to finish, finding (after some tuning) 4,422 potential images, of which 187 were uncorrupted pictures of Ella – including this one.

That felt good :)

I’ve published my version of the program as jpg-recover-faster. It’s a perl script, so you’ll need perl. I make no guarantees about the lack of bugs – use this script at your own risk. It may eat your children. ;)

You’ll want to read the comment at the top of the script before using it, and the other pages listed above will help you figure out how to use it. Feel free to post comments here with questions or suggestions.

Add a comment [9]

Recipe / HOWTO for printing from linux/unix/BSD to Windows

filed under: ,

6386 days ago


This recipe tells how to configure a linux/unix/BSD machine and a Windows machine such that the unix machine can print to a printer attached to the Windows machine, even if it isn’t a postscript printer. This could also be adapted to work for OSX client machines.

How it works

Starting from the unix machine, a print job travels thusly:

  1. User prints something to a CUPS-configured print queue
  2. CUPS generate a postscript version of the printout, using Ghostscript and/or foomatic and/or gimp-print
  3. CUPS passes the postscript file to Samba which passes it over the network to a network-shared (virtual) postscript printer on the Windows box
  4. The virtual printer takes the poscript file and gives it RedMon which gives it to Ghostscript (on the Windows box), which prints it out to the actual printer

There’s a bit of work on the Windows side to set up Ghostscript and the virtual printer, and another bit of work on the unix side to set up Samba so you can print over the network, and then another bit of work to set up CUPS so it can accept printjobs and route them to Samba.


First post!

filed under: ,

6688 days ago

...from my new NetBSD machine, running Firefox on X, that is :)

Just a quick note to the geeks out there that I’ve gotten X working on my NetBSD machine, along with:

  • Xfce as the window manager, although I tried KDE and Gnome also
  • XDM as the machine logon screen
  • Firefox as the browser
  • Java 1.5 working as a Firefox plugin (which required Linux emulation, via Suse)
  • Thunderbird as the email client
  • OpenOffice as the office suite
  • The BitStream fonts so web pages don’t look ugly

So that’s pretty cool – I was just able to log in graphically, check my email using Thunderbird, open a Word doc attachment and make changes. Then, just for kicks, I played some online java games, to prove that I could :)

But here’s the best part: I did (nearly) all of it using pkgsrc, just by typing “make install” in the right places. (And yes, the linux emulation just happened automagically.) Damn, but I love NetBSD.

Add a comment [6]