down the rabbit hole: my encoding setup

At some point, I really want to detail my entire setup for organizing, cataloging, ripping, encoding, and storing my entire DVD collection, because frankly, it’s kind of cool and it’s really fun. Plus, I’ve spent a lot of time on it, and I think it’d be good to get the information out there.

Or in other words, take the braindump that is on dvds.beandog.org and put it into a huge document.

That’d be fun.

Here’s all the components (in no real order) that I have:

  • Command-line tools to get all the metadata from the DVD I can and drop it into a database.
  • Website to collect and organize DVDs and its properties (tracks, chapters) into libraries (TV Shows, Movies) and episodes or content.
  • Command-line scripts to extract the data from the database and create scripts to extract everything.
  • Do the encoding.

Every time I try to describe it, it gets really complex. I basically have to merge multiple components with multiple programs written in multiple languages to encode content to one specification to play back on one application.

It works. :)

Here’s a sample encode running right now on my server, to kind of illustrate every step I have:

HandBrakeCLI --markers --cfr --title 7 --encoder x265 --quality 18 --rate 60 --encoder-preset medium --encopts level-idc=50:colorprim=smpte170m:transfer=smpte170m:colormatrix=smpte170m --audio 1 --aencoder copy --input /home/steve/Media/Laundry-Basket/1.567.1867.MASK..iso --output 1.567.1867.16866.MASK..mkv

I seriously don’t know how I would *organize* all the data, because of its size and complexity, and thinking about that and knowing I can go so many different directions with it and me being so pedantic about writing it, kind of makes it impossible. I guess maybe it might make sense to split preference from technology. Or something. For me, they just go hand in hand. I mean, what good is all the tech involved if you can’t get the social experience you expect (good content in good viewing situation).

Who knows. It rocks, though. :)

Advertisements

happy pie day!

I have a somewhat annual tradition I like to do around Thanksgiving, which is go visit random friends and family that I know, drop off a pie, and at the same time pick up any spare clothes that they may have, and then take them to the homeless shelter.  I’ve never really been a big fan of Thanksgiving, since I’m kinda meh about turkey, and I don’t really like sitting around all day.  Thanksgiving, however, is a perfect holiday when everyone else is home, so it’s easy to stop by almost any place and pick something up.

I’m going to try and do this again this year, but this time around be a bit more organized.  So … here’s specifically how it works.

I make lots of chocolate cream pies, and if you live in either Salt Lake County or Utah County or Davis County, I’ll swing by when it works for you, deliver one, and ask if you have any spare clothes or items you can donate to the homeless in Utah.  I’ll be out and about all day WEDNESDAY Nov. 26th through SUNDAY Nov. 30th, and can come any time if there’s a preference.

What you can donate: clothes, obviously, are really appreciated, but here are a list of things that *I* myself am always surprised to hear what they need:

  • towels
  • disposable razors
  • sleeping bags
  • diapers

As far as clothes go, we need them of ALL sizes (including big and tall, and plus sizes) and for ALL ages — yes this means babies, small children and teenagers along with adults.  The biggest ones out of anything need to be UNDERWEAR and SOCKS — again, for all ages and sizes.  One time a volunteer at a homeless shelter told me “they wear them until they literally disintegrate and fall apart.”

If you can donate something, great.  If you *don’t* have anything at your house to give, please go to the store and spend $10 on any of the above items.  It’s small amount to spend, but it’ll make a huge difference!

I haven’t decided yet which place specifically I’ll be taking the donations to, but chances are I’ll spread it out based on what I can pick up.  Here’s places I’ve taken stuff to / helped out at before:

So, if you’re up for it, send me an e-mail, message me on Twitter or Facebook, or text me if you have my phone #, and let me know this:

  • possible days / times to come by, and best days to come by — since I’m gonna be all across the Wasatch front, flexibility is appreciated
  • if your pie needs to be sugar-free
  • your address
  • any weird instructions I need to get to your house
  • a phone number I can call you on when I get lost trying to understand weird directions on how to get to your house

That’s it!  Happy Pie Day!

clothesline project

Every year, UVU (my school) hosts what’s called The Clothesline Project.  It’s a project that is designed to raise awareness about domestic abuse of all kinds — physical, emotional, sexual, and also things leading to death and suicide.

The layout is that survivors of abuse create t-shirts where they tell their story or share a message about what happened or what they went through.  In some cases, the t-shirts are written by relatives of people who were abused and were killed.  It’s pretty intense stuff.

_photo06_05_2f_b5aaea4df6ba__1365017388000

I went last to last year’s exhibit, and went again this year for my psychology class.  The first time I went, I looked at almost every t-shirt there, and I was at the exhibit for probably an hour and a half.  On multiple occasions I was so grossed out that I almost threw up.  A lot of them were terribly traumatizing.  I remember driving home afterwards, and I was so overcome by emotion that I was sobbing uncontrollably.  Afterwards, I was deeply depressed for about a week.  It certainly raised some awareness in me.

I learn a lot from reading first-hand accounts of anything related to situations like this.  It gives a clear, non-academic approach of what it feels like for someone to go through these things.  The stories are informative, to see how they cope, how they escape, how some of them let go, and so on.  There are all kinds of endings as well.  Sometimes their family or friends don’t believe them, sometimes the perpetrator dies or gets thrown in jail and is convicted.  Other times they get a divorce, or get married to someone else, or just flee the situation completely.

A common problem that I see in a lot of the stories are this — people do not speak up when they are being abused, or do nothing about it.  In some cases, someone else in their family was also being abused, but neither one knew.  It is so important to speak up, to tell someone!  Abuse has many side effects on the person receiving it.  It severely mess up their emotions and take away from them a proper healthy reference of how things like relationships, sex, and emotions are supposed to be.  The best comment comes from one of the shirts below: “Silence is your enemy.  Talking is your medicine.”

I took some snapshots with my phone this year, because I wanted to post some of the stories on here.  I only managed to get a few, because I showed up at the display when there was only about twenty minutes before closing.  On top of that, I opted to only take pictures of shirts that I thought I’d be able to read later from a photo.

I’m posting the pictures and the text some of the t-shirts on here.  Be warned that these are graphic, verbose, and terrifying.  Proceed with caution.

Continue reading “clothesline project”

bully

So, I went and watched the movie “Bully” tonight.  It was good.  I’ve got kind of mixed feelings about it, probably because of the many ways I look at the stories.  Part of me was interested to see what students are going through.  Part of me was thinking about what social settings had to exist for a setting like that to exist.  And then I was thinking about how school administration seemed like politics a little bit, and I wondered if teachers had any idea that they’d be called upon learning how to do mediation when they were getting their degrees.

The story about the teenagers who committed suicide is really sad.  I’m really glad that the film didn’t focus just on that angle, though.  They followed a couple of students specifically, and then had footage of bullying in general, and students just dealing with it in a general sense.  It was tough to watch, and made me feel bad for the guys.  It also renewed the feeling that I really wanna do something about it.  During the film I pulled out my phone for a second to check the time, and seeing the background on my cell phone — a picture of me and my little brother, Steven — really hit me, and made me realize that I *am* doing something.  That was kinda cool. :)

I don’t know much about bullying to have an opinion.  I can’t really draw on my own experiences, since I was never bullied, and I don’t remember anyone around me getting bullied.  Either I wasn’t really observant, or it wasn’t going on much.  I dunno.  All through school I kind of just stayed in the background.  Nobody bothered me and I didn’t bother anyone.  Some of the scenes were about the students riding on the school bus, and I actually thought it was weird to have so many people on there.  I remember that the bus was hardly ever half full, and having two people in one seat was rare.  So, a lot of it, I couldn’t really relate.  I was just kind of watching it.

The thing that made me sad (more than the bullying, actually, go figure) was how the adults in the lives of the kids tried to help them out.  The kids were pretty much getting the message of “well, you should do something about it,” and “it’s not really that bad, kids do that.”  A big part of that reason was that the kids getting bullied wouldn’t tell their parents how bad it was.  And in the cases where they did and the school administration would address it, the kids and parents would call them out on it and say how nothing was really changing.  It brings up a lot of questions regarding maintaining order in schools, providing the students somewhere they can feel safe, and whose job it really is to be an influence on the bullies.

The stories about the suicides were sad, but for me it didn’t really dig into me as hard as the other stuff.  I have kind of a different perspective on suicide, in the sense of that I can *understand* why they would see it as an out.  I dunno if that’s common, or if you have to be really interested in counseling to know how that works.  The thing that is really crazy in my mind though is that these guys are committing suicide at such a young age, and that others usually don’t have any clue that they’re pushing their peers so far off the cliff until it’s too late.

The part that was really hard for me was seeing the kids themselves being bullied as they were in the middle of things — they were suffering all these things, they were trying to make sense of this — “why would they do this?” “can we just be friends?” “why isn’t anyone at school doing anything?” — and then getting mixed messages from their parents as well.  In every case, the parents had no idea how bad things were until either the kid snapped (one took a gun on a bus), they were completely ignored and isolated by the community after coming out (a lesbian), or they saw the actual footage of the film.

I’d recommend seeing the film.  It was really good, and put together well.  I was hesitant to go see it, since I knew this is an emotional issue, and I thought it’d be easy to draw on that emotion and make a movie that was just sensationalizing it a little bit.  It wasn’t that way at all, though.  It came across to me as a sincere documentary that looked at the problem, explored it very well, and showed the stories of how they *really* are.  I love movies that are raw in that sense, where they are just about *life*.  In that vein, I’d recommend seeing “Boy Interrupted” as well.  That movie is also really gritty (and about suicide).

As strange as it may seem, I love movies like this where they display actual raw emotion, what the people are going through.  I prefer things like this not to be watered down or come with an obvious agenda.  Just exposing human life for what it’s like is good enough (and sad enough, in some cases).  I wish there were more films like this (and if you know of any, let me know).

Out of the entire film, one scene stood out to me the most.  It was in the assistant principal’s office (who, she was only in the film for maybe five minutes herself) who called in a student to talk to about bullying.  The kid came in looking just like any other kid, not sure what was going on, but that was about it.  She (the principal) pointed down to her desk, a picture, I’m imagining, of a student that was being bullied, and asked what his relationship was to him.  The poor kid just instantly lost the color to his face, and noticeably tensed up as he realized he was in trouble.  That made me feel really bad, that getting a shock like that, that you’re doing something wrong is suddenly and abruptly brought to your attention.  He genuinely had a look of “wow, I didn’t know that was wrong,” partly because he looked like a really innocent kid in addition to how daunted he was by being accused of bullying.  I kind of read into it that he was going along with things, but didn’t really realize the effect he was having.  In contrast, there was another kid who was also called into her office to talk about it, and he had an attitude of denial and how it wasn’t happening, and it wasn’t a big deal.  For the first kid, I thought to myself, there’s got to be a better way to bring this to his attention and correct it.  I feel really bad for anyone who gets the banhammer dropped on them unexpectedly.  That’s something I work really hard not to do with people, so it makes me sad when I see it happen to someone else.

I felt really bad for all the kids — the bullies and the ones being bullied.  I wish there was some easy answers, but I think there are two things that would help — learning how to communicate better with students, and having everyone learn to be kind.  There were a lot of times when bullies were being punks, and the other kids just kind of rolled with it.  That’s a social phenomenon in itself, which is pretty normal … people don’t typically step in when something unfair is going down, and in a lot of cases, will just pile on the aggression, because it seems to make the most sense.  I dunno, there’s a lot of variables in it that make it a difficult challenge, but I still think there’s some simple concepts that would help (communication, kindness, courage).

I dunno how I would handle it if someone came to me and told me they were getting bullied.  I’d honestly never really thought about it before, again, mostly because it’s something I didn’t really ever know much about it.

There’s a lot of great videos on youtube about bullying.  I’ll end on a positive one. :)

freebsd, quick deployments, shell scripts

At work, I support three operating systems right now for ourselves and our clients: Gentoo, Ubuntu and CentOS.  I really like the first two, and I’m not really fond of the other one.  However, I’ve also started doing some token research into *BSD, and I am really fascinated by what I’ve found so far.  I like FreeBSD and OpenBSD the most, but those two and NetBSD are pretty similar in a lot of ways, that I’ve been shuffling between focusing solely on FreeBSD and occasionally comparing at the same time the other two distros.

As a sysadmin, I have a lot of tools that I use that I’ve put together to make sure things get done quickly. A major part of this is documentation, so I don’t have to remember everything in my head alone — which I can do, up to a point, it just gets really hard trying to remember certain arguments for some programs.  In addition to reference docs, I sometimes use shell scripts to automate certain tasks that I don’t need to watch over so much.

In a typical situation, a client needs a new VPS setup, and I’ll pick a hosting site in a round-robin fashion (I’ve learned from experience to never put all your eggs in one basket), then I’ll use my reference docs to deploy a LAMP stack as quickly as possible.  I’ve gotten my methods refined pretty well so that deploying servers goes really fast — in the case of doing an Ubuntu install, I can have the whole thing setup close to an hour.  And when I say “setup” I don’t mean “having all the packages installed.”  I mean everything installed *and* configured and ready with a user shell and database login and I can hand over access credentials and walk away.  That includes things like mail server setup, system monitoring, correct permissions and modules, etc.  Getting it done quickly is nice.

However, in those cases of quick deployments, I’ve been relying on my documentation, and it’s mostly just copy and paste commands manually, run some sed expressions, do a little vim editing and be on my way.  Looking at FreeBSD right now, and wanting to deploy a BAMP stack, I’ve been trying things a little differently — using shell scripts to deploy them, and having that automate as much as possible for me.

I’ve been thinking about shell scripting lately for a number of reasons.  One thing that’s finally clicked with me is that my skill set isn’t worth anything if a server actually goes down.  It doesn’t matter if I can deploy it in 20 minutes or three days, or if I manage to use less memory or use Percona or whatever else if the stupid thing goes down and I haven’t done everything to prevent it.

So I’ve been looking at monit a lot closer lately, which is what I use to do systems monitoring across the board, and that works great.  There’s only one problem though — monit depends on the system init scripts to run correctly, and that isn’t always the case.  The init scripts will *run*, but they aren’t very fail-proof.

As an example, Gentoo’s init script for Apache can be broken pretty easily.  If you tell it to start, and apache starts running, but crashes after initialization (there’s specifics, I just can’t remember them off the top of my head) the init script thinks that the web server is running simply because it managed to run it’s own commands successfully.  So the init system thinks Apache is running, when it’s not.  And the side effects from that are that, if you try to automatically restart it (as monit will do), the init scripts will insist that Apache is already running, and things like executing a restart won’t work, because running stop doesn’t work, and so on and so forth.  (For the record, I think it’s fair that I’m using Apache as an example, because I plan on fixing the problem and committing the updates to Gentoo when I can.  In other words, I’m not whining.)

Another reason I’m looking at shell scripting as well is that none of the three major BSD distros (FreeBSD, NetBSD, OpenBSD) ship with bash by default.  I think all three of them ship with either csh or tcsh, and one or two of them have ksh as well.  But, they all have the original Bourne shell.  I’ve tried my hand and doing some basic scripting using csh because for FreeBSD, it’s the default, and I thought, “hey, why not, it’s best to use the default tools that it ships with.”  I don’t like csh, and it’s confusing to try and script for, so I’ve given up on that dream.  However, I’m finding that writing stuff for the Bourne shell is not only really simple, but it also adds on the fact that it’s going to be portable to *all* the distros I use it on.

All of this brings me back to the point that I’m starting to use shell scripts more and more to automate system tasks.  For now, it’s system deployments and system monitoring.  What’s interesting to me is that while I enjoy programming to fix interesting problems, all of my shell scripting has always been very basic.  If this, do that, and that’s about it.  I’ve been itching to patch up the init scripts for Gentoo (Apache is not the only service that has strange issues like that — again, I can’t remember which, but I know there were some other funky issues I ran into), and looking into (more) complex scripts like that pushes my little knowledge a bit.

So, I’m learning how to do some shell scripting.  It’s kind of cool.  People always talk about, in general, about how UNIX-based systems / clones are so powerful because of how shell scripting works .. piping commands, outputting to files, etc.  I know my way around the basics well enough, but now I’m running into interesting problems that is pushing me a bit.  I think that’s really cool too.  I finally had to break down the other day and try and figure out how in the world awk actually does anything.  Once I wrapped my head around it a bit, it makes more sense.  I’m getting better with sed as well, though right now a lot of my usage is basically clubbing things to death.  And just the other day I learned some cool options that grep has as well, like matching an exact string on a line (without regular expressions … I mean, ^ and $ is super easy).

Between working on FreeBSD, trying to automate server deployments, and wanting to fix init scripts, I realized that I’m tackling the same problem in all of them — writing good scripts.  When it comes to programming, I have some really high standards for my scripts, almost to the point where I could be considered obsessive about it.  In reality, I simply stick to some basic principles.  One of them is that, under no circumstances, can the script fail.  I don’t mean in the sense of running out of memory or the kernel segfaulting or something like that.  I mean that any script should always anticipate and handle any kind of arbitrary input when it’s allowed.  If you expect a string, make sure it’s a string, and that it’s contents are within the parameters you are looking for.  In short, never assume anything.  It could seem like that takes longer to write scripts, but for me it’s always been a standard principle that it’s just part of my style. Whenever I’m reviewing someone else’s code, I’ll point to some block and say, “what’s gonna happen if this data comes in incorrectly?” to which the answer is “well, that shouldn’t happen.”  Then I’ll ask, “yes, but what if it *does*?”  I’ve upset many developers this way. :)  In my mind, could != shouldn’t.

I’m looking forward to learning some more shell scripting.  I find it frustrating when I’m trying to google some weird problem I’m running into though, because it’s so difficult to find specific results that match my issue.  It usually ends up in me just sorting through man pages to see if I can find something relative.  Heh, I remember when I was first starting to do some scripting in csh, and all the search results I got were on why I shouldn’t be using csh.  I didn’t believe them at first, but now I’ve realized the error of my ways after banging my head against the wall a few times.

In somewhat unrelated news, I’ve started using Google Plus lately to do a headdump of all the weird problems I run into during the day doing sysadmin-ny stuff.  Here’s my profile if you wanna add me to your circles.  I can’t see a way for anyone to publicly view my profile or posts though, without signing into Google.

Well, that’s my life about right now (at work, anyway).  The thing I like the most about my job (and doing systems administration full time in general) is that I’m constantly pushed to do new things, and learn how to improve.  It’s pretty cool.  I likey.  Maybe some time soon I’ll post some cool shell scripts on here.

One last thing, I’ll post *part* of what I call a “base install” for an OS.  In this case, it’s FreeBSD.  I have a few programs I want to get installed just to get a familiar environment when I’m doing an install: bash, vim and sometimes tmux.  Here’s the script I’m using right now, to get me up and running a little bit.  [Edit: Upon taking a second look at this — after I wrote the blog post, I realized this script isn’t that interesting at all … oh well.  The one I use for deploying a stack is much more interesting.]

I have a separate one that is more complex that deploys all the packages I need to get a web stack up and running.  When those are complete, I want to throw them up somewhere.  Anyway, this is pretty basic, but should give a good idea of the direction I’m going.  Go easy on me. :)

Edit: I realized the morning after I wrote this post that not only is this shell script really basic, but I’m not even doing much error checking.  I’ll add something else in a new post.

#!/bin/sh
#
# * Runs using Bourne shell
# * shells/bash
# * shells/bash-completion
# * editors/vim-lite

# Install bash, and set as default shell
if [ ! -e /usr/local/bin/bash ] ; then
	echo "shells/bash"
	cd /usr/ports/shells/bash
	make -DBATCH install > /dev/null 2>&1
	if [ $? -ne 0 ]; then
		echo "make install failed"
		exit 1
	fi
else
	echo "shells/bash - found"
fi
if [ $SHELL != "/usr/local/bin/bash" ] ; then 
	chsh -s /usr/local/bin/bash > /dev/null 2>&1 || echo "chsh failed"
fi

# Install bash-completion scripts
if [ ! -e /usr/local/bin/bash_completion.sh ] ; then
	echo "shells/bash-completion"
	cd /usr/ports/shells/bash-completion
	make -DBATCH install > /dev/null 2>&1
	if [ $? -ne 0 ]; then
		echo "make install failed"
		exit 1
	fi
else
	echo "shells/bash-completion - found"
fi

# Install vim-lite
if [ ! -e /usr/local/bin/vim ] ; then
	echo "editors/vim-lite"
	cd /usr/ports/editors/vim-lite
	make -DBATCH install > /dev/null 2>&1
	if [ $? -ne 0 ]; then
		echo "make install failed"
		exit 1
	fi
else
	echo "editors/vim-lite - found"
fi

# If using csh, rehash PATH
cd
if [ $SHELL = "/bin/csh" ] ; then
	rehash
fi

multimedia reference guide: handbrake

I wrote about the x264 reference I put together yesterday. I just got finished with a version for handbrake presets as well.

This time I was surprised how much Handbrake changes stuff. However, I have never been one to argue with the results, because they are always gorgeous. The decombing and deinterlacing filters are what really sold me on using it, since so far it’s taken all but a few of my DVDs and managed them just fine. That had been a major pain in my side with all DVD rippers for years.

I’m thinking of putting together a small libav to x264 comparison for flags, and then adding the x264 command lines for all the presets as well. If anyone thinks of anything similar they’d like to see, let me know.

truth and trivia

At work today, I randomly commented to my friend, Jason, “It’s interesting to note what drops out of your life when your time gets filled with important things.” That seems to be the trend my schedule is taking lately. Not to say my schedule is a paragon of efficiency and order. I just had pudding for dinner. But I have noticed that as my surplus of resources diminishes, things change. And it’s curious to note what gets dropped.

It makes me think of this talk I heard some time:

“When compared to eternal verities, the questions of daily living are really rather trivial. What shall we have for dinner? Is there a good movie playing tonight? Have you seen the television log? Where shall we go on Saturday? These questions pale into insignificance when times of crisis arise, when loved ones are wounded, when pain enters the house of good health, or when life’s candle dims and darkness threatens. Then truth and trivia are soon separated. The soul of man reaches heavenward, seeking a divine response to life’s greatest questions: Where did we come from? Why are we here? Where do we go after we leave this life? Answers to these questions are not discovered within the covers of academia’s textbooks, by dialing information, in tossing a coin, or through random selection of multiple-choice responses. These questions transcend mortality. They embrace eternity.”