packages website in progress

Generally speaking, I don’t like writing posts just to give a status report, since there’s nothing to show … especially in this case … but I guess I’ll make an exception.

I’ve been working on the packages website rewrite over the holidays, and it is really starting to come together.  I’ve decided to short circuit the process and instead of going for a full-featured site that I’d like to have, just duplicate the bare minimum of the old site to get it up and running as fast as possible.

I don’t want to raise hopes too much, but it’s gonna be a really cool site — not the first launch, but what it’ll eventually become.  I’ve rewritten all the code to access the portage tree, and I have classes to access just about everything.  Doing that has given me a tremendous amount of flexibility, and it just makes things easier to bang out.

There have been some rough spots that I have run into that are some hurdles.  Right now I’m working on determining masked packages.  Always a bit tricky, but thank goodness that’s the hardest part I’m running into right now.

I don’t have a timeline for when I expect the site to be up.  I’m actually ready to start rewriting the frontend for it right now.  I’m looking at getting a new design theme for it, and I’ll probably wait to get that before launching it.  I already have a new domain for it, too.  Much shorter, easier to remember and type.  Realistically speaking, I think the site is probably going to be going live somewhere near the end of December.

Hosting is still a problem.  I don’t know what I’m going to do with it, partially because I’m not sure how much CPU or traffic load it’s going to generate.  I have three options right now — host it at home, on my Linode or on a shared hosting account — neither of which I’m really excited about since I think they’ll all have different things they’ll choke on, network traffic, CPU usage and speed, respectively.  So it’s gonna be very likely that it gets bumped around a bit at first.

One last thing I wanted to mention.  I almost decided to completely abandon the project completely, since it seemed like no one really cared that I had even taken it offline.  I have had a few people poke me though, and ask about it’s status, and I’ve seen it mentioned in a few other places.  This is one of those projects that I’m never sure if anyone ever finds it useful, so lack of feedback makes me want to drop it completely.  Inversely, any feedback makes me want to work on it and keep it going.  So, I’m really curious to hear what you liked about the old site.

Also, along that same vein, something else I’m going to do with the new site is build out features based on requests only.  I have a lot of ideas that I’d like to put in there, but instead of doing that, I’m going to hold off on them and just build out what the users want.  Everytime someone tells me how they use it, it surprises me because I never imagined it being accessed in that way.  So, again, feedback is critical.

Well, that’s all for now.  Again, I don’t know how soon I’ll have a super slick website up.  Chances are, that I’ll put an XML API up first (another new feature) or the RSS feeds since that doesn’t require any nice eye candy.  We’ll see.

my hardware closet

I finally called and cancelled my Comcast cable TV subscription this weekend.  I’ve been meaning to do it forever.  I can’t remember the last time I was even watching TV on a regular basis, though I think it was probably around 6 months or so.  With my media center up and running so well, I generally just watch something from there or rent it on Netflix these days.

Now, the question is, what to do with all the hardware?  I have three (yes, THREE) Tivos with unlimited subscriptions.  Two of them are the klunky one-tuner first generation of the Series2 boxes, but the third is an HD Tivo, which is very nice.  I could sell them, but considering the price I shelled out (for the HD one, at least, savvy consumerism got me the first two for real cheap … under $50 each) I hate to part with it.  I keep thinking I’ll get cable again some day, and by that year sometime in the future, I’ll pat myself proudly on the back and say, “way to hang on to a piece of hardware for so long!  Now go get the compressed air.”

I’m cursed with the pack rat mentality, though.  I hang onto stuff far too long in the oft chance that someday, I *might* need it.  On Saturday, I woke up a little early, and as is normally the weekend routine, I get the feeling that I must turn my entire world around by 11 a.m.  This time, it was the closet in my living room which has the distinction of dedicating 85% of its storage space to electronics that I might need sometime before the next century.  The other 15% is a mix between my puzzles, dust bunnies, air, movie posters, and movie t-shirts.  I have a red t-shirt promoting “Searching for Bobby Fischer”, I kid you not… I’ll even take pictures to prove it.

I swear I’ve been carrying this collection of cables and equipment for at least ten years or so, probably ever since I haven’t been living at home.  It never really bothered me that I don’t use most of the technology anymore (or ever, really), you just never know when you’re going to need a floppy IDE cable.  Really!  I’m all about being prepared, but for the wrong circumstances.  I can just see the day when I’ll be someone’s hero for helping them be able to flash the BIOS on their 15-year old Dell desktop.  I still have the floppies to put it on, too.

So, this weekend was the closet’s demise.  I grabbed a bunch of plastic bags from the kitchen (the bachelor’s preferred method of storage and transportation for all things non-essential) and started filling them up with stuff.  My method of deciding what to keep and what to throw out was pretty simple: if I couldn’t remember the last time I used it, it gets tossed.  Normally it could be times like these that a selective memory can cause problems down the road, but I had so much junk anyway, I don’t think it’ll cause a problem.  Besides, the memory problems go both ways — when I do need a new cable or piece of hardware, I can’t get mad at myself because I’ll have forgotten I used to own one anyway.

All in all, I filled up something like eight to twelve bags of stuff.  I don’t remember how many it was, but I do recall that when I took it to the thrift store and started unloading, I had so much stuff that it took two guys to carry it all away, and one of them kept laughing because the stream was endless.  I think most of it was cables.  There were some notable things that I’d been hanging onto for a long time, “just in case,” some of which were: my old Gamecube, an 8 GB IDE harddrive, my old home-theater-in-a-box speakers (which were about as powerful and had as much wattage as two light bulbs), three PCMCIA wireless cards, a few wireless USB dongles, and a slew of PCI slot brackets.  I elected to hold onto the floppy drive — it was a sound investment in 1990, and it’s a sound investment today.

That’s not really the interesting part, though.  There is still all the stuff that I decided to keep because it held some kind of value, but I don’t have the energy or drive to see them through the process of being sold on the secondhand market.  Nothing makes a closet grow quite like a pack rat mentality combined with the laziness of avoiding the hassle of making spare change.  I’ve still got my Tivos, for instance.  There’s an old (now) AMD Athlon64 desktop that is pretty nice — top of the line of about 5 years ago.  Runs really quiet, too.  Then there’s a used Gateway desktop I remember I bought on Craigslist for some reason a while ago, and I’ve never used.  I’m holding onto that one because it came with Windows, and I might someday want *another* Windows XP Home key, so I can just use that one.  It probably wouldn’t work, anyway, but hey … hope lingers longer than logic.

I almost dragged off my original Xbox to the thrift store, too.  It hasn’t been as fun as I’d hoped it would have been, and having one console (PS2) with corded controllers is enough for me.  Plus it’s a bit wheezy.  Probably just needs a new fan, or harddrive.  Dunno.  I also decided to keep all my TV tuner cards, even though I never use those either … especially now without a cable subscription.  One of them was the Plextor external USB one that has MPEG4 hardware encoding (very nice).  I think my brother wanted that one.  Maybe it’ll be a nice holiday surprise, as in, I’ll be surprised if I manage to make it to the post office before Christmas 2010 to mail it off.

Oh yah, and there’s an MSI Mini-ITX motherboard with an Intel Atom that I’ll probably never try to revive, but I hate to donate it since I still think I could get at least $20 for it somewhere.  Then there’s my new Motorola RAZR phone that I used for about a week before I switched to Verizon.  That’s gotta be worth something.  I’ve also got two MP3 players, an iPod Nano and a Sansa something, each 4 GB … too small for me to do anything with … but I’ll hang onto them just because.  I think I still have a portable Sony Walkman cassette player, too.

In actuality, I’d like to get rid of all of the stuff, provided I can do it through a simple way … meaning I don’t have to do any work, and non-creepy people flock to my house with cash in hand.  I doubt it’d happen, but hey, if you live near Salt Lake and are interested … drop me a line.  That’s about as proactive as I’m gonna get about it.  If you ever need your BIOS flashed, too, I could probably do that as well.

packages website going offline for a while

I’ve been on a roll to clean house lately, and part of that is simplifying my hardware setup.  One thing that needs to be ripped out completely is my old server, which is getting to be a real pain to maintain.  Mostly it’s just my personal stuff on there, but the ebuild packages website is also running on there right now.  Between now and Tuesday, I’m going to take it down since I’m going to be rearranging my hardware setup anyway.  I’m not going to bring the old website back online, either.  The code for the new one is almost complete, and it will use a lot less resources.  There’s gonna bet lots of cool stuff on the new one: better feeds, simpler interface, new domain name and hopefully a new design as well.  Oh, and the scripts aren’t dependent upon portage anymore, which is the real crutch right now.  I have to run an old version of portage ( that isn’t even in the tree anymore, and it’s making updates painful or impossible.

The new site will also run on my dedicated Linode, where I think I’ve finally correctly managed the apache issues, so that means there will be less arbitrary downtime as I screw around with my box here at home.  I really hate running servers at home that other people are dependent on, because I like the freedom to change things around without affecting anyone.  Right now, the old site is so CPU intensive, that I can’t move it over to the VPS.

The code for the new site is much cleaner.  The entire thing is rewritten in OOP classes to access the portage tree, which makes my job incredibly easier.  Not to mention it’s a lot faster.  It’ll still be a bit before I get it online, but killing it will inspire me to push it along.  I’m tired of having this thing limp around when it’s just a dead albatross around my neck right now.  So, farewell.  The new one will be better. :)

hardware setup

I got woken up this morning at about 5:30 because my server’s fan was so loud.  Seriously.  Actually, I think I had trouble sleeping anyway, but it was unusually noisy, and when I woke up, the first thing I thought was, “what the heck is that noise?”  The rear fan in the ATX case of my server is precariously placed, and if the elements don’t align quite right, it rattles quite a bit, and that’s what happened here.

This is the server that houses all my media files, so I can’t really just rip it out and replace it with something else.  It has two 750 GB drives right now which make up my entire library space.

I shut the sucker down for a bit so I could pop it open and see if I could adjust the fan and blow out some of the dust in there.  While it was powered down, it was so quiet in the room, I couldn’t believe it.  I forgot how much noise these things make.

Since I was already up, I decided to look into some ways to either reduce the noise pollution or find an alternative storage setup.  I’ve got a spare Mini-ITX system and a spare external SATA drive enclosure, so I decided to fire that up and see how good my transfer rate would be if I just used external drives with a fanless low-powered Mini-ITX.  By the time I left for work, I was in the middle of transferring a bunch of media files over to the new harddrive, so I guess I’ll find out later.  But the transfer rate was at about 8 MB/s, so I think it’s safe to say that it’ll work out pretty nice.

I’m hoping that this setup will work in the future for my dream scenario: a quiet file server.  I figure if I can buy a few 1.5 TB harddrives, and plug them all into external SATA enclosures, then I should be good to go.

I went ahead and bought a Western Digital Green 1.5 TB harddrive this morning, too, to replace my other drives in the server.  I know the Green line of harddrives isn’t the fastest of the line, but I think that, for my circumstances, it’ll run just fine.  They run between 5200 and 7200 RPM.  Less speed should mean less heat, which would make me worry not as much about having it as an external drive being passively cooled.  I’m not gonna be using it as the OS, for the Mini, I’m already running that off of a 4GB USB drive, and that runs plenty fast.  The only thing it’ll do is just be serving up media over the wired network.  As long as the read speed is decent, I probably won’t have any complaints.

closed captioning on dvds (and ripping them)

In ripping my DVDs, I try to future-proof it as much as I can, by putting in as many elements as I *think* I might need or want someday down the road.  One of those elements is subtitles.  There are three types of subtitles that can be on DVDs — VobSub, closed captioning and SDH — and the first two can be extracted fairly easily.  I have no idea how to access the SDH ones.  I think you need either a newer DVD player or a Blu-Ray one.

I’ve been ripping my TV shows, and so far I haven’t seen any really hard and fast rules on what to expect with them on DVD.   Part of the reason is that I just haven’t been paying much attention to subtitles until recently.

I was playing with ripping one show last night, and I saw the CC logo on the back of the case, so I went to check the rest of my library to see which other ones had it.  Nearly my entire library of Warner Bros. DVDs displayed the logo — even for much older cartoons (Looney Tunes, Scooby Doo) — once again staying consistent with the fact that the studio puts a lot of effort into the quality of their releases.


I just started playing with extracting CC though, and just barely wrote the code to my DVD ripper to extract them, so I have no idea what the other series are like, if they have subtitles or not — VobSub or CC.  I usually don’t find out until I actually go to rip them.

Extracting the closed captioning subtitles is a lot easier and faster than getting the VobSub streams.  For Linux (and Mac and Windows) there’s a nifty OSS program called ccextractor.  Once you have your VOB video file on your harddrive, just run that on the movie, and it will create an SRT subtitle file of the closed captioning text.  It’s great, and really fast, taking probably under a minute on a 60-minute video on my box.  Comparatively, when ripping a VobSub stream, you need to read the DVD directly which causes its own bottleneck, and then demux the entire stream.  It takes probably around 3 to 5 minutes for an episode of the same length.

Another thing I like about the closed captioning titles is that because they are extracted as SRT, it’s easy to look through them since they are just text files.  If you’re really anal, you can correct typos yourself.  The VobSub subtitles are all bitmaps.  I’ve also noticed that on some DVDs, where there were issues with framerates or something else, that the VobSub timestamps will be off … and sometimes either they will show up clumped together at the beginning of the film or the sync will be way off.  I think that this has to do with the dumping process, somewhere, but I’m not sure.  I’ve never really taken the time to pin down the source.

So, with closed captioning being easier and faster to extract, as well as editable and the timestamps haven’t had any issues for me (yet), it’s quickly becoming my preferred subtitle format.

There’s only one small issue with using ccextractor, and that is you won’t know if there are any captions in the VOB until after it’s made its trial run.  The program will create an .srt file regardless when you run it, but the file will be empty if it couldn’t find any.  That’s the only drawback.  With VobSub, you can know if there are subtitles just by probing the DVD using lsdvd or something similar.

Muxing it into matroska is simple, too.  Just pass it as a file argument and you’re done.

As a sidenote, while my bend application that I wrote and use to rip DVDs would be a major pain to setup for someone else, I’ve rewritten it recently so that it uses individual classes to access every object directly: DVD, DVD track, DVD VOB, Matroska file.  They are standalone classes written in PHP if anyone wanted to use them, feel free.  You would also need my tiny class of shell functions as well, since they all make calls to it.

The DVDVOB one makes it simple to extract the subtitle stream.  In fact, all the classes make things relatively simple.  They have made writing my code so much simpler.

ratchet and clank: a crack in time

One thing I will readily admit is that I’m not a veteran gamer.  Not having a video game console for most of my life will kind of do that to you.  Thankfully, though, to an abundance of free time, a large HDTV and no pesky girlfriend to spend money on, I can resolve that issue gradually.  Not knowing what the heck kind of games is out there has its drawbacks now and then (I have a really small library of games), but it does have it’s positive upswings too, for instance, when I “discover” something new quite by accident.


This weekend while puttering around, I decided to check out the latest demos on the Playstation Network, and downloaded one for Ratchet and Clank Future: A Crack in Time.  There were two demos, interestingly enough, one for Ratchet and one for Clank.  I got Ratchet’s.  As is with most demos, I wasn’t expecting much (it’s a pretty high failure rate when it comes to interest), but I fired it up to give it a whir.

The game dropped me in a weird landscape with a gun, and I kind of groaned to myself .. not another shooter.  I suck at shooters, which is one reason I’ve been avoiding console gaming a lot.  But, as I started playing it, here’s the sequence of thoughts that went through my head:

  1. How the heck do I shoot my gun?
  2. How do I shoot my special weapon?
  3. Holy crap, I have hoverboots?
  4. Oh my gosh, this game is fun!
  5. Repeat last step, about 500 times.

Heh.  It was awesome.  So much, in fact, that when I finished, it said that the game came out last Tuesday!?  Well, you know what that means to a man who has spent his whole life evolving a careful sense of patience and self-mastery …  I paid full price for it at Best Buy 15 minutes later.

I’ve since played through probably a third of the game, and I’m just now starting to lose steam.  Wow, it is fun.

First of all, the graphics are absolutely amazing.  It’s just eye-popping nice.  I’m actually surprised that they’d put so much effort into the artwork for a game that is mainly for an audience much younger.  But the quality shines through in every aspect so far.

The game has one of my favorite features of all time, too — no penalty for deaths.  It just starts  you back up where you left off.  I love that.  I’m sure one of the reasons I never had a console growing up was because I realized early on that it was an issue-aggragator rather than a calming, enjoyable experience.  Let’s just say it’s a good thing that those Nintendo controllers were hard plastic — they could take a lot of damage at high velocity speeds.

Another really nice touch about the game is it has a degree of free-range movement.  There are lots of different places you can go, and while the storyline is linear, you can take breaks from it and go back to where you were before to finish things up or just screw around.  Its fun.  It gives you enough freedom to screw around if you want, or get into a serious adventure if you’re up for the run.

The only thing I don’t like about it is playing as Clank.  So far, it’s just a chore.

The storyline is great, too, and there is some great dialogue / writing in there.  Really original and funny.  Something else I really enjoy too is the ability for the character to upgrade himself and his weapons.  I grew up with the gold-box of AD&D gaming, and I always loved levelling my guys.  This isn’t the same level of complexity, but still I think it’s a nice touch that your character’s abilities improve as the guys get harder in the game.  It only seems fair.

All around, a really awesome game.  And it turns out, as I discovered, that there’s apparently a whole series of Ratchet and Clank games, too.  I’m really skeptical that any of the others would be nearly as good as this one, but who knows.  At least they’re not as expensive by now.