lds-scriptures-3.0 data dumps (well, some)

The next version of the lds-scriptures project release is taking forever. The main reason (aside from me always putting it off) is that it will most likely be the last release since it fulfills all my original goals, and so I’m obsessing over every little detail to make sure it’s right.

That said, the *schema* is actually complete and ready to go. It pretty much has been for a while, but yesterday I got around to doing some more optimizations. I also thought I had the Access ones cleaned up, but I had a few issues in there which have now been resolved. I still have a few nitpicks about it, but I’ll probably let it slide.

So, downloads for the actual data are available right now for MySQL, PostgreSQL, SQLite 2 and 3, and Access 97, 2000 and 2002. You can get them here: http://spaceparanoids.org/nephi/lds-scriptures-3.0/

I still have a few formats to export to (CSV, XML, etc.), so I’m not finished, and then I need to write the documentation because the entire schema has changed since the 2.0 release series. The major change is that the data is completely normalized now. Another small bonus is I threw in a sample view there. Oh, and the obligatory bugfix is that I accidentally missed some of the text in one of the Doctrine and Covenants sections. Whoops.

Anyway, I’m tired of waiting for my perfect release to make it out the door, so at the very least, there’s the raw data that everyone’s going to want, anyway. :)

I’m glad that at least that’s done!

star trek on blu-ray

All the original Star Trek movies (1 through 6) came out on Blu-Ray today in one large collector’s set. Normally I’d be really excited about something like this, but I’m pretty meh about the whole thing, and am gonna pass on this and wait for a couple of reasons.

The first reason is, I don’t like “collection” packs. I prefer to buy stuff individually, and avoid them wherever possible (for movies, at least). Plus, it goes without saying that Star Trek V and Star Trek VI are painful to watch. The first four are the only ones worth watching, and I actually like Star Trek III quite a lot myself for some reason. Wrath of Khan is undeniable, of course. Ah, heck, I like em all. :)

Another reason is they didn’t do such a great transfer on them. It looks like Star Trek II: The Wrath of Khan is the only one that they went back to the original film and did an HD transfer. All the other ones, they just took a current digital copy and cleaned it up. Double meh.

The third reason is, this is Star Trek … and Paramount has a long history of milking fans for all the money its worth by dangling a barebones release then coming out with a much better package later on in the game. Just look at Star Trek: The Original Series on DVD for an example. I think it’s been released three times, and it just barely came out on Blu-Ray too.

So, me, I’ll be a-waitin for the individual films. And a better HD transfer.

One thing is certainly for sure, though … if TNG, DS9 or Voyager ever get Blu-Ray releases, you can bet your bippy I’m gonna snatch those up in no time flat. I’ve been watching Deep Space Nine on and off, and they really did not do a great job on the digital transfer — they used a really low compression level for it, and I can always see blurring on it, even on my old 27″ CRT (480i). That’s pretty bad. By comparison, I’ve been watching season five of Voyager recently, and on my HDTV, the picture is still very nice. They did a good job on it.

upgrading myth, part two

I’ve almost finished the upgrade process for MythTV.  Last night I ported all my patches towards 0.21 and so far everything is working great.  This time I took it a step further and modified the code so it would look for all cover art in a central directory, instead of locally (filename.jpg for individual files, folder.jpg for directories).  Kind of cool, I think, that I’m starting to make the changes directly in MythVideo for my personal preferences instead of working around Myth’s functionality by creating symlinks and stuff on the filesystem.

Usually I would avoid going into the code, but I’m slowly getting more comfortable doing it.  I think the Qt3 docs could stand being a tad bit more verbose, though.  In a lot of cases where I was confused, it was a matter of not understanding exactly what the function was returning, and I’d have to execute it myself to figure it out.

Aside from that, all the “original” stuff is in, but I found two more bugs that I want to hunt down.  I thought I had this first one licked, but I guess not — if you have a file on the filesystem that is not stored in the database, then it will display the filename sans extension as the title at the top of the page (fex: Mr._Belvedere).   I’ve already fixed it so that it will show the “correct” title (replace _ with spaces) when displaying the grid of all files, but I can’t seem to find the variable where it’s showing the other title and it’s a little hard going through the code trying to figure out what everything is doing.  I haven’t figured *that* much out.  In fact, I know my way around just a few functions in videogallery.cpp and that’s about it.  Once it gets outside of that, I’m back in Wonderland.  I’m lost.

Add on top of that there’s not much documentation, if any, in the code and it makes it a bit of slow going.  Ah well, it’s only a matter of time most likely.

The second bug is really annoying, and it’s interesting in that it’s one of those phantom ones that was always like … “wait a minute, did something just change?”  There was a nagging feeling that something had changed, but I could never pin my finger on it.  I finally found it.  What happens is, when MythVideo has scanned the files and put the metadata into the database, it will do some kind of intelligent sorting based on the titles for display.  That is, titles prefixed with “A”, “And”, “The”, etc. will have the prefix dropped (can’t remember the correct grammatical term for those) and then sorted by the following words in the title.  I only caught it because I had added an MP3 file for the LP of “A View To A Kill” (and yes, there are children’s records that tell the story of the 007 films — how cool is that?) and I was testing some stuff in that folder and actually saw it’s position change before and after it was put into the metadata database.

That one, I have absolutely no idea where it’s getting sorted at all.  I don’t have a clue, and it could be anywhere, since all the MythVideo displays implement that, not just the Gallery view.  I just need to send a note off to the mythtv user’s mailing list and hopefully someone will know.  That would be nice to get rid of, though.  I don’t like the inconsistency.

In fact, I never use the metadata database at all.  Pretty much all I see is the cover image, which I supply manually, and the title.  And I rename all my videos so that the title is part of the filename, including the episode order for the ones that are a part of a series.  I already ripped out the stuff to display the metadata before playback, since I’m not interested in that.  So, I know that it’s somehow related to the metadata database, since the titles *and* the orders are affected once they are in there.  Just gotta figure out where.

Other than that, everything has gone really smooth, and it’s been nice.  One thing I did notice as well was that 0.21 just seems generally snappier and much more responsive.  It could be my imagination, but I don’t think so … there just seems to be less latency and wait when navigating and moving around places.  It’s pretty nice.

gui development

Whenever I work on writing some patches for MythVideo, it always occurs to me that using MythTV is extreme overkill for what I’m doing.  I’m using a very small subset of its features, just creating a gallery view that is highly tweaked towards my preferences.  And that’s it.  Pretty simple, really.  Recently, though, the idea has been festering in my mind that I could probably write my own, and it’d be a good starter app for learning some C++.  I’ve found that learning a new skill goes invariably much faster when I have a practical example to apply it to.  I can’t just pick up a new language just because it seems like fun.  That just doesn’t go anywhere.

Anyway, writing a small GUI app to replace MythVideo for my needs seems like it’d be pretty simple.  It’s nothing really more than a visual filesystem frontend to my multimedia library, with cover images to individual files and folders.

The thing that is really driving me, though, is that in spite of the simplicity of the basic functions, I am extremely picky on presentation, navigation and UI.  I’ll never be able to pick up an “out of the box” media frontend package that is becoming so popular on the market now.  I just have so many ways that I want to organize and display the stuff that I’d never be able to live with a narrow set of parameters.  I don’t have an interface just to watch videos.  Mine is setup so that I can watch an entire series of a TV show, have it resume playback from each indivual file by itself and in relation to each other.  I also have random playlists that I generate myself.  And then I categorize, tag and shuffle my files all around the place.  Plus I’m getting to add movies as well as MP3s — not just music, but records, sometimes with accompanied video (kind of a slideshow effect).  Simple solutions just don’t fit me.

So, I finally bought a Qt programming book online today, so I’ll see where that takes me.  I don’t know why I never really thought about trying to integrate learning C++ with a GUI language as well … it just makes more sense, plus I have the example project that I can hack on.

And everytime I dive into C++ I just love it even more.  It’s cryptic, it’s confusing, it’s extremely strict and barks at you if you do the slightest thing wrong.  I’m a real perfectionist when it comes to writing code that I intend to be reliable and I love how C++ just whips you into shape.

I’ve toyed with the idea of rewriting my DVD ripping scripts in C++, but I don’t really see the point.  It’d be an idea, but not really a driven one.  I would, however, love to learn how to hack on libdvdread a bit to either fix some small bugs I’ve run into, or try to help me address some of the more rare issues that I’ve run into with my collection.  For instance, I got a bad DVD in a TV series the other day, and while my settop DVD player could kind of skip around it, dvdread absolutely choked and stopped reading.  Just little stuff like that would be fun to try and figure out if I could workaround.

Anyway, we’ll see.  I picked up a copy of this book for $5.  The reviews don’t look that great, but it might get me started a bit.  That’d be nice.

upgrading myth

I started doing something last weekend I always swore I’d never do — upgrade MythTV on my Mini ITX.

Normally, the process wouldn’t be hard, except that I’ve built a custom image that is running on a solid-state flash disk that is only 256 mb in size. And yes, it’s Gentoo. It doesn’t have a full blown Gentoo install, of course, but it’s certainly a very stripped down version of one. I’ve had the image on there for probably a year and a half or more, and have been extremely content to just leave it alone as it works just fine.

As time goes on, though, I’ve been hacking on MythVideo a bit here and there, adding a few tweaks to suit my needs. This weekend I finally got another navigation one hammered out that took me a few hours to figure out due to my lack of C++/Qt programming skills. It was worth it though — it was one annoying UI decision I didn’t like (I won’t go in details, but it’s how the menu position is selected when navigating back and forth through the video gallery).

The problem is that making any kind of changes to the Mini’s image is always a pain for a couple of reasons. For one, if I screw something up badly, it’s a bit difficult to get into the box. I don’t have a USB bootable stick laying around, and in fact I’ve never had much luck getting one to work .. and I don’t think this box will boot off of one anyway. Not sure. That means if something really goes haywire then I have to boot over the network, which is a bit of a pain to setup sometimes. Fortunately, it didn’t come to that this time.

Another issue is that because space is so limited, part of the filesystem is loaded from a read-only squashfs image. That means a bit of tedious testing when it comes to unpacking the image, removing the old files, adding the new ones in place, repacking it, remounting it, and restarting the application each time just to test anything. That adds a lot of time to the process.

This time around, I did something pretty smart, in my opinion. I don’t know why I didn’t think of this before. The squashfs image is loading /usr/lib. This time, I installed Myth to /usr/local so that I could leave the libraries alone and replace MythTV easily. Just create a new image for that one, and drop it in anytime.

Anyway, tonight I just barely got it working with the new install. I went from a very old 0.20 install to the latest 0.21 in portage (0.21_p19961). In fact, one of the reasons I had to upgrade is because I can’t even get 0.20 to compile anymore. Normally I wouldn’t care, but I figure I may want to write more patches, and it’s getting to be painful trying to maintain an older version that I can’t even duplicate in case of a problem.

One other nice thing that’s changed since I last built this is I have my still-somewhat-newish ThinkPad to build it on. So I just build the binaries on my x86 laptop, strip out all the crap I don’t need or want, create a new squash image, and drop it right in. Relatively speaking, it has all gone rather smoothly. I think I’ve probably spent about 20 hours on it since Sunday. I vaguely recall it taking at least two weeks the first time I put it together. And of course, it took me something like three months to even get X, Qt, LIRC and all the other stuff on such a small bootable image. That was a really bumpy ride, but I wouldn’t trade the knowledge to do it for anything.

That’s the other thing — I could *easily* spend less than $20 and get double the harddrive space (512MB) and be able to forego half the problems. Or, I could even spend another $20 and get a 1 or 2 GB drive and have a *lot* of room to play around in. Oddly enough, I’m really going against my normal approach and instead of throwing money at the problem, I’m taking it as a matter of pride that I can endure so much pain and still get it on such a small drive. And really, I don’t need the extra space anyway. Once it’s done, it’s done and done. It’ll be just like before, where I’ll leave it alone for 18 months or more, and I’ll be completely happy with it the whole time.

Just for the record, here’s the harddrive current status. I managed to clean up a lot of cruft this time around and freed up a lot of space:

Filesystem            Size  Used Avail Use% Mounted on
/dev/hda1             238M  184M   42M  82% /
udev                   10M   60K   10M   1% /dev
/usr/usr.lib.img       36M   36M     0 100% /usr/lib
/usr/usr.local.img     18M   18M     0 100% /usr/local
shm                   474M     0  474M   0% /dev/shm
svcdir                2.0M  116K  1.9M   6% /var/lib/init.d
//willy/media         1.4T  624G  726G  47% /var/media

And finally, here’s what the menu currently looks like … still unpatched, and with the default theme. That’s for another day. I’ll cover what the patch changes, too. Someone may find it useful. In the meantime, I think it’s time to try and catch up on sleep.  I’m really tired.

myth1

Oh, and those icons are from the gartoon theme set.

some cases for copyright violations, part three

This (hopefully last) scenario about a case for willful copyright violation is one that I’m much less inclined to agree with as morally okay, but it still presents an interesting problem.  The situation is similar to the last one, in that certain goods that could be replicated digitally were never available commercially, but in this case, they were at one time, but aren’t anymore.

Actually, dang it, it looks like I’m splitting off into two possible scenarios *again* … the other one I wanted to cover was a case of newer, recent media that has never been available commercially.  Lemme do the other one, first.

A great example of a category that covers both scenarios, though, would be abandonware.  Abandonware can be either computer games that were once released for personal computers, and so people obviously had physical copies, or ROMs for video games where the consumer would buy hardware, but couldn’t duplicate it themselves.

For old computer games, that were once published and had a legal channel to purchase a copy, there is still the option of tracking down a copy somewhere, but it is usually difficult to do and cost prohibitive.  Collectors are willing to do it, because of the sense of ownership of owning an original piece of the production, but for people who just want to play the game, there is a demand without any marketed supply.

I’ve really waffled with the moral considerations of this one for a long time, and I personally don’t have a clear cut decision on what I think is “right.”  It seems obvious, though, that if there is the chance of getting a legal copy, even as a collector, that that would be the right thing to do, and in a lot of cases that is is what I’ve done.  I’ve hunted down copies of old games (LOOM comes to mind) that I never played but wanted to, and then once I had an actual copy, I’d feel justified in getting a digital one somewhere that I could run on an emulator (assuming I couldn’t use the original disks for whatever reason … usually because I don’t have a floppy 5 1/4″ drive).  But, I’ll admit, I’ve also downloaded copies of games that I’ve wanted to try out just to see what I think of them as an attempt to demo it and see if it’s worth my time to track down.  Generally, though, I’ll either delete the copy once I’ve found it not worth my time or track down a copy once I decide it is.

Computer games is just one example though.  A much more modern one would be television shows and movies.  There are, for some reason or another, modern day shows on television that just never make it to DVD.  And I’m not talking about stuff from 20 years ago that I’m being nostalgic about, either, I’m talking about really recent shows from this decade.  One example, just off the top of my head, is an older show called “Still Standing” that is running in syndication right now on a couple of channels on TV late at night.  It’s a great, hilarious, original family sitcom that I love watching, and if there were DVDs out, I’d most likely buy them.  It ran from 2002 to 2006, and hasn’t seen so much as a blip on the radar of TV shows on DVD.  I have no idea why.

Now in a case like this, I’m also not sure what the “right” thing to do would be, but I’ll at least share my own line of thinking on the matter.  This is my take — that since they were originally designed for public consumption, and publicly aired at one time for anyone to see, anywhere, then I don’t have any issues with downloading copies of them and watching them.  In my mind, it’s just a case of extreme time-shifting.  I couldn’t watch it 7 years ago, so I’m just now getting around to it.  It was originally free and public, and so how could there possibly be a harm to the market — especially since there’s no other way to watch them?

A second example is a crime drama that’s being aired right now, called “Cold Case.”  It’s a show that I think is fun and worth watching, but it’s not out on DVD either, and so if I want to watch them, I’m pretty much on my own again to find the archives since there’s no market for them.  Another case of demand with no supply.  CBS’ website does have some clips of the show (big whoop), but again, I don’t understand the logic of not putting them all online somewhere … aside from the infrastructure cost, of course.  It just doesn’t make sense in my mind how studios can broadcast a show, for free consumption, and then have an issue when it is, technically, rebroadcast by being downloaded somewhere else.  Of course, I should also throw in that I don’t mind having commercial breaks during those since I recognize that’s their source of revenue.  Personally, though, I’ve been voluntarily ignorant of TV commercial influence since I was about 5, but that’s a whole other story entirely.

A third example is of a show that started to release some DVD sets, but then stopped after the first season — Malcolm in the Middle.  In that case, I do know what the issue is — licensing.  Everyone who originally signed up now wants a bigger piece of the pie now that it’s getting redistributed.  And, as a result, of course, it ends up in *zero* revenue versus any at all.  Once again it shows how boneheaded people can be with their rights.  It’s seriously like little children who say if you can’t play how they tell you to, then they’re just going to spitefully take their toys and play by themselves, always ignorant of the fact that people adapt and do just as well *without them* and the missed opportunity is really the loner’s.

Anyway, it’s really interesting how “piracy” is creating the market and supplying the demand that is out there that businesses either cannot create or are not willing to invest time into.  It’s rather interesting, really, that an ocean of people would willingly provide the infrastructure and shoulder the costs to share all this information, media and entertainment by themselves.  What’s even more interesting is what people *will* do when there are legally licensed marketplaces to acquire the exact same stuff digitally (a la iTunes) — they’ll pay for it!

I don’t really think it’s fair to “blame” copyright owners for not seeing the demand and investing their futures into providing virtual marketplaces for all these digital goods.  After all, there is a risk in doing so.  But on the flipside, I also don’t think it’s reasonable for them to chase down scofflaws who are copying stuff when there’s really no way to legally get it in the first place.  There needs to be a happy medium somewhere.  I have no idea what it is, though I have a few ideas — a simple licensing framework website where the digital goods are hosted and sold … kind of a cloud for cpoyrighted digital good or something, who knows.  I’m not a businessman, thank goodness, just a savvy consumer.  But I imagine that if the entry barrier into the marketplace was lower for investors to setup shop, that things would quickly streamline and we’d see “piracy” dwindle quite a lot.  I don’t think people are being intentionally bad breaking copyright laws, I just see them as being reasonable and practical first.