znurt.org cleanup

So, I finally managed to getting around to fixing the backend of znurt.org so that the keywords would import again.  It was a combination of the portage metadata location moving, and a small set of sloppy code in part of the import script that made me roll my eyes.  It’s fixed now, but the site still isn’t importing everything correctly.

I’ve been putting off working on it for so long, just because it’s a hard project to get to.  Since I started working full-time as a sysadmin about two years ago, it killed off my hobby of tinkering with computers.  My attitude shifted from “this is fun” to “I want this to work and not have me worry about it.”  Comes with the territory, I guess.  Not to say I don’t have fun — I do a lot of research at work, either related to existing projects or new stuff.  There’s always something cool to look into.  But then I come home and I’d rather just focus on other things.

I got rid of my desktops, too, because soon afterwards I didn’t really have anything to hack on.  Znurt went down, but I didn’t really have a good development environment anymore.  On top of that, my interest in the site had waned, and the whole thing just adds up to a pile of indifference.

I contemplated giving the site away to someone else so that they could maintain it, as I’ve done in the past with some of my projects, but this one, I just wanted to hang onto it for some reason.  Admittedly, not enough to maintain it, but enough to want to retain ownership.

With this last semester behind me, which was brutal, I’ve got more time to do other stuff.  Fixing Znurt had *long* been on my todo list, and I finally got around to poking it with a stick to see if I could at least get the broken imports working.

I was anticipating it would be a lot of work, and hard to find the issue, but the whole thing took under two hours to fix.  Derp.  That’s what I get for putting stuff off.

One thing I’ve found interesting in all of this is how quickly my memory of working with code (PHP) and databases (PostgreSQL) has come back to me.  At work, I only write shell scripts now (bash) and we use MySQL across the board.  Postgres is an amazing database replacement, and it’s amazing how, even not using it regularly in awhile, it all comes back to me.  I love that database.  Everything about it is intuitive.

Anyway, I was looking through the import code, and doing some testing.  I flushed the entire database contents and started a fresh import, and noticed it was breaking in some parts.  Looking into it, I found that the MDB2 PEAR package has a memory leak in it, which kills the scripts because it just runs so many queries.  So, I’m in the process of moving it to use PDO instead.  I’ve wanted to look into using it for a while, and so far I like it, for the most part.  Their fetch helper functions are pretty lame, and could use some obvious features like fetching one value and returning result sets in associative arrays, but it’s good.  I’m going through the backend and doing a lot of cleanup at the same time.

Feature-wise, the site isn’t gonna change at all.  It’ll be faster, and importing the data from portage will be more accurate.  I’ve got bugs on the frontend I need to fix still, but they are all minor and I probably won’t look at them for now, to be honest.  Well, maybe I will, I dunno.

Either way, it’s kinda cool to get into the code again, and see what’s going on.  I know I say this a lot with my projects, but it always amazes me when I go back and I realize how complex the process is — not because of my code, but because there are so many factors to take into consideration when building this database.  I thought it’d be a simple case of reading metadata and throwing it in there, but there’s all kinds of things that I originally wrote, like using regular expressions to get the package components from an ebuild version string.  Fortunately, there’s easier ways to query that stuff now, so the goal is to get it more up to date.

It’s kinda cool working on a big code project again.  I’d forgotten what it was like.

another semester done

I just finished my Fall semester for 2012 today at UVU.  This was, by far, the hardest semester I’ve ever had since I’ve been in school.  It was brutal.  I had three classes which carried with it more work than I was expecting, and I spent a lot of time in the past four months doing nothing but homework.  I was talking to my cousin tonight about it (while we were doing some late-night skateboarding in the winter, which, it’s actually really nice out here right now), and I mentioned that the stress was a huge burden on me.  Stress is normal, but I’ve learned that if something heavy is really going on, I notice I will stop being cheery.  I don’t really get somber, but it’s more like, just focused and serious all the time.  Which can be a real bummer.

But, the semester is finished, and it’s freed up a lot of time and has taken that huge burden off of me.  I got good grades, and along with that, and some great friends that really stepped up at the last minute and helped me out, it’s really gotten me humbled and grateful to God and everyone that stood by me.  I’m really glad this semester is done.

One thing I learned from this last jaunt around is that I’ve decided I’m never taking online classes again.  I had two this semester, and one on campus.  Looking back, I’ve always had a range of issues with online courses.  Either I don’t understand the material very well because I can’t chat with the professor one on one, or I slack the whole time (I did 50% of the coursework in one day.  I’m not kidding).  The worst one though is I never really feel like I “get” the material.  I jump through hoops, get a grade, and move on, but it doesn’t seem like I learned anything.

So, I’m sticking to just two classes from here on out, and doing them all on-campus.  That’ll be manageable.

For now I’m really looking forward to not so much having more time, but having less stress.  I’ve been wanting to work on some cool side projects, and I also have been itching to go skating … a lot.  So tonight I went on a two-hour run with my cousin down Main Street in Bountiful, and it was really cool.  We call it a “mort run” since we start at the top of a hill and go all the way down to the mortuary.  It’s smooth all the way down and  you can just push around and then either skate back up hill or walk.  It’s a good workout.

The best part tonight though was debating whether or not we should go to the drive-through at Del Taco, knock on the window and ask for something.  We didn’t, but we circled the place like eight times and probably freaked out the employees while we debated it.  Eventually, we realized he didn’t have enough cash to buy something on the dollar menu (he was a penny short), so we spent half an hour wandering around downtown looking for lost change.  It was pretty fun. :)

Soooooooooooo ….. projects.  One thing I have time to look into now is znurt.org.  It’s broken.  I’ve known it’s been broken.  It would take me probably less than an hour to fix it.  I haven’t made the time, for a lot of reasons.  It’s actually been on my calendar reminding me over and over that I need to get it done.  I’m debating what to do about the site.  I could just fix the one error and move on, but it’s still kind of living in a state of neglect.  Ideally, I should hand the project over to someone else and let them maintain it.  I dunno yet.  Part of me doesn’t wanna let it go, but I guess a bigger part doesn’t care enough to actually fix it so … yah.  Gotta make a decision there.

Other than that, not much going on.  I moved to a new apartment, back into a complex.  I like it here.  I have a dishwasher now, which I’m really grateful for (I haven’t had one in the last three apartments).  The funny thing about that is I seriously have so few dishes, that filling up the entire thing with all of mine it’s half full.

Anyhoo, I am really looking forward to moving on.  My big thing is I wanna get some serious skating time in while I’ve got the time.  That and enjoy the holidays with friends and family.  I’m looking forward to next semester too.  I’ve got a class on meteorology and another on U.S. history.  I’m almost done with generals.  The crazy part about all of this?  Since I went back to school two years ago, I’ve put in 30 credit hours.  Insane, for someone working full time.  I tell you what.

bully

So, I went and watched the movie “Bully” tonight.  It was good.  I’ve got kind of mixed feelings about it, probably because of the many ways I look at the stories.  Part of me was interested to see what students are going through.  Part of me was thinking about what social settings had to exist for a setting like that to exist.  And then I was thinking about how school administration seemed like politics a little bit, and I wondered if teachers had any idea that they’d be called upon learning how to do mediation when they were getting their degrees.

The story about the teenagers who committed suicide is really sad.  I’m really glad that the film didn’t focus just on that angle, though.  They followed a couple of students specifically, and then had footage of bullying in general, and students just dealing with it in a general sense.  It was tough to watch, and made me feel bad for the guys.  It also renewed the feeling that I really wanna do something about it.  During the film I pulled out my phone for a second to check the time, and seeing the background on my cell phone — a picture of me and my little brother, Steven — really hit me, and made me realize that I *am* doing something.  That was kinda cool. :)

I don’t know much about bullying to have an opinion.  I can’t really draw on my own experiences, since I was never bullied, and I don’t remember anyone around me getting bullied.  Either I wasn’t really observant, or it wasn’t going on much.  I dunno.  All through school I kind of just stayed in the background.  Nobody bothered me and I didn’t bother anyone.  Some of the scenes were about the students riding on the school bus, and I actually thought it was weird to have so many people on there.  I remember that the bus was hardly ever half full, and having two people in one seat was rare.  So, a lot of it, I couldn’t really relate.  I was just kind of watching it.

The thing that made me sad (more than the bullying, actually, go figure) was how the adults in the lives of the kids tried to help them out.  The kids were pretty much getting the message of “well, you should do something about it,” and “it’s not really that bad, kids do that.”  A big part of that reason was that the kids getting bullied wouldn’t tell their parents how bad it was.  And in the cases where they did and the school administration would address it, the kids and parents would call them out on it and say how nothing was really changing.  It brings up a lot of questions regarding maintaining order in schools, providing the students somewhere they can feel safe, and whose job it really is to be an influence on the bullies.

The stories about the suicides were sad, but for me it didn’t really dig into me as hard as the other stuff.  I have kind of a different perspective on suicide, in the sense of that I can *understand* why they would see it as an out.  I dunno if that’s common, or if you have to be really interested in counseling to know how that works.  The thing that is really crazy in my mind though is that these guys are committing suicide at such a young age, and that others usually don’t have any clue that they’re pushing their peers so far off the cliff until it’s too late.

The part that was really hard for me was seeing the kids themselves being bullied as they were in the middle of things — they were suffering all these things, they were trying to make sense of this — “why would they do this?” “can we just be friends?” “why isn’t anyone at school doing anything?” — and then getting mixed messages from their parents as well.  In every case, the parents had no idea how bad things were until either the kid snapped (one took a gun on a bus), they were completely ignored and isolated by the community after coming out (a lesbian), or they saw the actual footage of the film.

I’d recommend seeing the film.  It was really good, and put together well.  I was hesitant to go see it, since I knew this is an emotional issue, and I thought it’d be easy to draw on that emotion and make a movie that was just sensationalizing it a little bit.  It wasn’t that way at all, though.  It came across to me as a sincere documentary that looked at the problem, explored it very well, and showed the stories of how they *really* are.  I love movies that are raw in that sense, where they are just about *life*.  In that vein, I’d recommend seeing “Boy Interrupted” as well.  That movie is also really gritty (and about suicide).

As strange as it may seem, I love movies like this where they display actual raw emotion, what the people are going through.  I prefer things like this not to be watered down or come with an obvious agenda.  Just exposing human life for what it’s like is good enough (and sad enough, in some cases).  I wish there were more films like this (and if you know of any, let me know).

Out of the entire film, one scene stood out to me the most.  It was in the assistant principal’s office (who, she was only in the film for maybe five minutes herself) who called in a student to talk to about bullying.  The kid came in looking just like any other kid, not sure what was going on, but that was about it.  She (the principal) pointed down to her desk, a picture, I’m imagining, of a student that was being bullied, and asked what his relationship was to him.  The poor kid just instantly lost the color to his face, and noticeably tensed up as he realized he was in trouble.  That made me feel really bad, that getting a shock like that, that you’re doing something wrong is suddenly and abruptly brought to your attention.  He genuinely had a look of “wow, I didn’t know that was wrong,” partly because he looked like a really innocent kid in addition to how daunted he was by being accused of bullying.  I kind of read into it that he was going along with things, but didn’t really realize the effect he was having.  In contrast, there was another kid who was also called into her office to talk about it, and he had an attitude of denial and how it wasn’t happening, and it wasn’t a big deal.  For the first kid, I thought to myself, there’s got to be a better way to bring this to his attention and correct it.  I feel really bad for anyone who gets the banhammer dropped on them unexpectedly.  That’s something I work really hard not to do with people, so it makes me sad when I see it happen to someone else.

I felt really bad for all the kids — the bullies and the ones being bullied.  I wish there was some easy answers, but I think there are two things that would help — learning how to communicate better with students, and having everyone learn to be kind.  There were a lot of times when bullies were being punks, and the other kids just kind of rolled with it.  That’s a social phenomenon in itself, which is pretty normal … people don’t typically step in when something unfair is going down, and in a lot of cases, will just pile on the aggression, because it seems to make the most sense.  I dunno, there’s a lot of variables in it that make it a difficult challenge, but I still think there’s some simple concepts that would help (communication, kindness, courage).

I dunno how I would handle it if someone came to me and told me they were getting bullied.  I’d honestly never really thought about it before, again, mostly because it’s something I didn’t really ever know much about it.

There’s a lot of great videos on youtube about bullying.  I’ll end on a positive one. :)

the gospel of simplicity

I had an interesting thought tonight.  “Lord, I want to join the battle.”  I love working with youth, talking to them, helping them out the best I can.  The thing that worries the most is not the decisions that they’ll make, but rather that I haven’t prepared myself enough.  I want to be spiritually ready all the time, to be up to any challenge that comes my way.  That’s a pretty tall order.  When I feel like I need to reach that lofty goal, I start to think of big ways to change my life, and how to get there amazingly fast.

What I’m having to learn over and over is that the the gospel is not about moments of energy and excitement.  It’s not big projects that need to be undertaken, or major changes to my schedule.  It’s not zealotry or extreme attitudes.  Instead, it’s about making a decision, day by day, to follow Christ.

Like many Christians, I wear a cross.  It’s a necklace that I put on every morning before I head out for the day.  I don’t have to put it on, but as I do, it’s a really personal reminder that I’m making a choice — that, yes, this is something I want to do, and take it upon myself willingly.  And what’s cool is that I have to make that decision every day — not as a group, but individually.  Every morning I make the choice.

I still have the habit of wanting to jump into things with full heart and spirit, and at times get almost a patriotic pledge to do more.  I think of big changes I can make so that I’m somehow getting more spirituality into my life.  It starts to become a project, some huge overreaching goal that I can build with lots of effort and work.  This leads problem that I will start to think there is something “special” out there that I should be doing, to find that extra measure of spiritual input.  Big goals require big commitments, which leads to big changes.  Rip out all the old stuff, and put in the new.  Everything old must go. There’s some method out there to tap this great well of spiritual power that I haven’t found yet, some secret sauce that the Lord will reveal to me as I push with so much effort and drive.

However, that is going about it the wrong way.  I love how the Lord puts things into perspective.  From Matthew 24:

26. Wherefore if they shall say unto you, Behold, he is in the desert; go not forth: behold, he is in the secret chambers; believe it not.

There are no secret angles, no shortcuts, no hidden mysteries for only a select few to find.  I do not need to go out into the desert, something that would take a lot of resources and dedication — somewhere only a few could go if they had the right equipment, stamina, and drive.

Instead, He has made it clear that it is the basic principles of the gospel, that all men, women and children can exercise, where they are.  Consider, for example, taking the basics to a higher level over time as you make it a part of your life.

Prayer is the simple act of talking to God.  Reading the scriptures is having God talk to me.  Fasting teaches self-control.  Like any skill, I can improve, and do better over time.  Instead of saying token prayers, I can learn how to calmly and quietly express my soul to God, and know that he hears.  Instead of reading the scriptures out of a sense of duty and daily obligation, I can study them and look more closely, trying to understand God’s will.

The basics, if expanded on, can bring about great results.  I know that that’s true, because as I decrease or increase in those simple things, I can notice a difference.

My crazy mind still likes to flirt with the idea that there is some great knowledge that I need to acquire before I can commit.  A nebulous mass of content that I must completely understand before I can move forward.

Again, the Lord puts things into perspective, making it so much simpler:

13. Enter ye in at the strait gate: for wide is the gate, and broad is the way, that leadeth to destruction, and many there be which go in thereat:
14. Because strait is the gate, and narrow is the way, which leadeth unto life, and few there be that find it.

The way that I read this is that my task  is to enter into the gate that leads unto eternal life.  He doesn’t say anything about winning the race, or how fast I should be going, or how soon I need to get there.  At the very beginning, He just wants me to go in the right direction.

It’s not hard to make that choice, but it’s hard for me to understand and accept that it’s so simple.  It really is, though, and when I think about how easy it is, I realize that it’s something I can do.  And the Holy Ghost confirms to me that it is true.  I like the Lord’s way much better than mine.

simpsons treehouse of horror buying guide

Note: I found this in my drafts of old posts, and this one never got published.  I wrote it in October of 2011, so the list may have changed a bit since then.

For those of you who know me, I really don’t like TV or movies with violence or gore in them. Yet, somehow, I am totally fascinated by them. Oddly enough, I’ll read all about horror movies and slasher flicks sometimes, and never watch them. I think part of the reason is I get *really* scared by them. Anyway. I especially love the Simpsons Treehouse of Horror episodes, because they are just awesome, and not as hardcore.

I promised my little brother that I’d get some for Halloween for us to watch. I don’t think he’s seen any of them. Edit: I showed him some last year. :)

Being the collector type that I am, I did some research, and lo and behold, FOX has released these in the most backwards incomplete way possible. In short, of the 21 seasons available to buy of the Simpsons, 17 of them are available to purchase, either through Amazon Video or DVD.

What’s crazy is that while Amazon Video sells them in “seasons”, they are really just totally random episodes thrown together. On top of that, the one DVD that is available is also episodes from random seasons, and two of them crossover with what is packaged in season 2 on Amazon Video. The rest, you can buy individually from the Simpsons seasons on Amazon Video.

It’s confusing, I know, but here’s how they released them:

Treehouse of Horror – Season One:
1990 I
1993 IV
1996 VII
1999 X
2002 XIII
2005 XVI

Treehouse of Horror – Season Two:
1991 II
1994 V
1997 VIII
2000 XI
2003 XIV
2006 XVII

Treehouse of Horror – DVD:
1994 V
1995 VI
1996 VII
2001 XII

So, for the crazy completist in your life, I’ve organized them in correct chronological order, with the link of how to buy them. Ultimately, you’re going to have to get them all this way, both seasons plus the DVD, regardless of crossover, if you want the most complete amount of episodes.

01: ssn1
02: ssn2
03: N/A
04: ssn1
05: ssn2, DVD
06: ssn2, DVD
07: DVD
08: ssn2
09: N/A
10: ssn1
11: ssn2
12: DVD
13: ssn1
14: ssn2
15: N/A
16: ssn1
17: ssn2
18: N/A
19: indy
20: indy
21: indy

gentoo, openrc, apache and monit – proper starting and stopping

I regularly use monit to monitor services and restart them if needed (and possible).  An issue I’ve run into though with Gentoo is that openrc doesn’t act as I expect it to.  openrc keeps it’s own record of the state of a service, and doesn’t look at the actual PID to see if it’s running or not.  In this post, I’m talking about apache.

For context, it’s necessary to share what my monit configuration looks like for apache.  It’s just a simple ‘start’ for startup and ‘stop’ command for shutdown:

check process apache with pidfile /var/run/apache2.pid start program = “/etc/init.d/apache2 start” with timeout 60 seconds stop program = “/etc/init.d/apache2 stop”

When apache gets started, there are two things that happen on the system: openrc flags it as started, and apache creates a PID file.

The problem I run into is when apache dies for whatever reason, unexpectedly.  Monit will notice that the PID doesn’t exist anymore, and try to restart it, using openrc.  This is where things start to go wrong.

To illustrate what happens, I’ll duplicate the scenario by running the command myself.  Here’s openrc starting it, me killing it manually, then openrc trying to start it back up using ‘start’.

# /etc/init.d/apache2 start
# pkill apache2
# /etc/init.d/apache2 status
* status: crashed
# /etc/init.d/apache2 start
* WARNING: apache2 has already been started

You can see that ‘status’ properly returns that it has crashed, but when running ‘start’, it thinks otherwise.  So, even though an openrc status check reports that it’s dead, when running ‘start’ it only checks it’s own internal status to determine it’s status.

This gets a little weirder in that if I run ‘stop’, the init script will recognize that the process is not running, and reset’s openrc’s status to stopped.  That is actually a good thing, and so it makes running ‘stop’ a reliable command.

Resuming the same state as above, here’s what happens when I run ‘stop’:

# /etc/init.d/apache2 stop
* apache2 not running (no pid file)

Now if I run it again, it checks both the process and the openrc status, and gives a different message, the same one it would as if it was already stopped.

# /etc/init.d/apache2 stop
* WARNING: apache2 is already stopped

So, the problem this creates for me is that if a process has died, monit will not run the stop command, because it’s already dead, and there’s no reason to run it.  It will run ‘start’, which will insist that it’s already running.  Monit (depending on your configuration) will try a few more times, and then just give up completely, leaving your process completely dead.

The solution I’m using is that I will tell monit to run ‘restart’ as the start command, instead of ‘start’.  The reason for this is because restart doesn’t care if it’s stopped or started, it will successfully get it started again.

I’ll repeat my original test case, to demonstrate how this works:

# /etc/init.d/apache2 start
# pkill apache2
# /etc/init.d/apache2 status
* status: crashed
# /etc/init.d/apache2 restart
* apache2 not running (no pid file)
* Starting apache2 …

I don’t know if my expecations of openrc are wrong or not, but it seems to me like it relies on it’s internal status in some cases instead of seeing if the actual process is running.  Monit takes on that responsibility, of course, so it’s good to have multiple things working together, but I wish openrc was doing a bit more strict checking.

I don’t know how to fix it, either.  openrc has arguments for displaying debug and verbose output.  It will display messages on the first run, but not the second, so I don’t know where it’s calling stuff.

# /etc/init.d/apache2 -d -v start
<lots of output>
# /etc/init.d/apache2 -d -v start
* WARNING: apache2 has already been started

No extra output on the second one.  Is this even a ‘problem’ that should be fixed, or not?  That’s kinda where I’m at right now, and just tweaking my monit configuration so it works for me.

freebsd, quick deployments, shell scripts

At work, I support three operating systems right now for ourselves and our clients: Gentoo, Ubuntu and CentOS.  I really like the first two, and I’m not really fond of the other one.  However, I’ve also started doing some token research into *BSD, and I am really fascinated by what I’ve found so far.  I like FreeBSD and OpenBSD the most, but those two and NetBSD are pretty similar in a lot of ways, that I’ve been shuffling between focusing solely on FreeBSD and occasionally comparing at the same time the other two distros.

As a sysadmin, I have a lot of tools that I use that I’ve put together to make sure things get done quickly. A major part of this is documentation, so I don’t have to remember everything in my head alone — which I can do, up to a point, it just gets really hard trying to remember certain arguments for some programs.  In addition to reference docs, I sometimes use shell scripts to automate certain tasks that I don’t need to watch over so much.

In a typical situation, a client needs a new VPS setup, and I’ll pick a hosting site in a round-robin fashion (I’ve learned from experience to never put all your eggs in one basket), then I’ll use my reference docs to deploy a LAMP stack as quickly as possible.  I’ve gotten my methods refined pretty well so that deploying servers goes really fast — in the case of doing an Ubuntu install, I can have the whole thing setup close to an hour.  And when I say “setup” I don’t mean “having all the packages installed.”  I mean everything installed *and* configured and ready with a user shell and database login and I can hand over access credentials and walk away.  That includes things like mail server setup, system monitoring, correct permissions and modules, etc.  Getting it done quickly is nice.

However, in those cases of quick deployments, I’ve been relying on my documentation, and it’s mostly just copy and paste commands manually, run some sed expressions, do a little vim editing and be on my way.  Looking at FreeBSD right now, and wanting to deploy a BAMP stack, I’ve been trying things a little differently — using shell scripts to deploy them, and having that automate as much as possible for me.

I’ve been thinking about shell scripting lately for a number of reasons.  One thing that’s finally clicked with me is that my skill set isn’t worth anything if a server actually goes down.  It doesn’t matter if I can deploy it in 20 minutes or three days, or if I manage to use less memory or use Percona or whatever else if the stupid thing goes down and I haven’t done everything to prevent it.

So I’ve been looking at monit a lot closer lately, which is what I use to do systems monitoring across the board, and that works great.  There’s only one problem though — monit depends on the system init scripts to run correctly, and that isn’t always the case.  The init scripts will *run*, but they aren’t very fail-proof.

As an example, Gentoo’s init script for Apache can be broken pretty easily.  If you tell it to start, and apache starts running, but crashes after initialization (there’s specifics, I just can’t remember them off the top of my head) the init script thinks that the web server is running simply because it managed to run it’s own commands successfully.  So the init system thinks Apache is running, when it’s not.  And the side effects from that are that, if you try to automatically restart it (as monit will do), the init scripts will insist that Apache is already running, and things like executing a restart won’t work, because running stop doesn’t work, and so on and so forth.  (For the record, I think it’s fair that I’m using Apache as an example, because I plan on fixing the problem and committing the updates to Gentoo when I can.  In other words, I’m not whining.)

Another reason I’m looking at shell scripting as well is that none of the three major BSD distros (FreeBSD, NetBSD, OpenBSD) ship with bash by default.  I think all three of them ship with either csh or tcsh, and one or two of them have ksh as well.  But, they all have the original Bourne shell.  I’ve tried my hand and doing some basic scripting using csh because for FreeBSD, it’s the default, and I thought, “hey, why not, it’s best to use the default tools that it ships with.”  I don’t like csh, and it’s confusing to try and script for, so I’ve given up on that dream.  However, I’m finding that writing stuff for the Bourne shell is not only really simple, but it also adds on the fact that it’s going to be portable to *all* the distros I use it on.

All of this brings me back to the point that I’m starting to use shell scripts more and more to automate system tasks.  For now, it’s system deployments and system monitoring.  What’s interesting to me is that while I enjoy programming to fix interesting problems, all of my shell scripting has always been very basic.  If this, do that, and that’s about it.  I’ve been itching to patch up the init scripts for Gentoo (Apache is not the only service that has strange issues like that — again, I can’t remember which, but I know there were some other funky issues I ran into), and looking into (more) complex scripts like that pushes my little knowledge a bit.

So, I’m learning how to do some shell scripting.  It’s kind of cool.  People always talk about, in general, about how UNIX-based systems / clones are so powerful because of how shell scripting works .. piping commands, outputting to files, etc.  I know my way around the basics well enough, but now I’m running into interesting problems that is pushing me a bit.  I think that’s really cool too.  I finally had to break down the other day and try and figure out how in the world awk actually does anything.  Once I wrapped my head around it a bit, it makes more sense.  I’m getting better with sed as well, though right now a lot of my usage is basically clubbing things to death.  And just the other day I learned some cool options that grep has as well, like matching an exact string on a line (without regular expressions … I mean, ^ and $ is super easy).

Between working on FreeBSD, trying to automate server deployments, and wanting to fix init scripts, I realized that I’m tackling the same problem in all of them — writing good scripts.  When it comes to programming, I have some really high standards for my scripts, almost to the point where I could be considered obsessive about it.  In reality, I simply stick to some basic principles.  One of them is that, under no circumstances, can the script fail.  I don’t mean in the sense of running out of memory or the kernel segfaulting or something like that.  I mean that any script should always anticipate and handle any kind of arbitrary input when it’s allowed.  If you expect a string, make sure it’s a string, and that it’s contents are within the parameters you are looking for.  In short, never assume anything.  It could seem like that takes longer to write scripts, but for me it’s always been a standard principle that it’s just part of my style. Whenever I’m reviewing someone else’s code, I’ll point to some block and say, “what’s gonna happen if this data comes in incorrectly?” to which the answer is “well, that shouldn’t happen.”  Then I’ll ask, “yes, but what if it *does*?”  I’ve upset many developers this way. :)  In my mind, could != shouldn’t.

I’m looking forward to learning some more shell scripting.  I find it frustrating when I’m trying to google some weird problem I’m running into though, because it’s so difficult to find specific results that match my issue.  It usually ends up in me just sorting through man pages to see if I can find something relative.  Heh, I remember when I was first starting to do some scripting in csh, and all the search results I got were on why I shouldn’t be using csh.  I didn’t believe them at first, but now I’ve realized the error of my ways after banging my head against the wall a few times.

In somewhat unrelated news, I’ve started using Google Plus lately to do a headdump of all the weird problems I run into during the day doing sysadmin-ny stuff.  Here’s my profile if you wanna add me to your circles.  I can’t see a way for anyone to publicly view my profile or posts though, without signing into Google.

Well, that’s my life about right now (at work, anyway).  The thing I like the most about my job (and doing systems administration full time in general) is that I’m constantly pushed to do new things, and learn how to improve.  It’s pretty cool.  I likey.  Maybe some time soon I’ll post some cool shell scripts on here.

One last thing, I’ll post *part* of what I call a “base install” for an OS.  In this case, it’s FreeBSD.  I have a few programs I want to get installed just to get a familiar environment when I’m doing an install: bash, vim and sometimes tmux.  Here’s the script I’m using right now, to get me up and running a little bit.  [Edit: Upon taking a second look at this — after I wrote the blog post, I realized this script isn’t that interesting at all … oh well.  The one I use for deploying a stack is much more interesting.]

I have a separate one that is more complex that deploys all the packages I need to get a web stack up and running.  When those are complete, I want to throw them up somewhere.  Anyway, this is pretty basic, but should give a good idea of the direction I’m going.  Go easy on me. :)

Edit: I realized the morning after I wrote this post that not only is this shell script really basic, but I’m not even doing much error checking.  I’ll add something else in a new post.

#!/bin/sh
#
# * Runs using Bourne shell
# * shells/bash
# * shells/bash-completion
# * editors/vim-lite

# Install bash, and set as default shell
if [ ! -e /usr/local/bin/bash ] ; then
	echo "shells/bash"
	cd /usr/ports/shells/bash
	make -DBATCH install > /dev/null 2>&1
	if [ $? -ne 0 ]; then
		echo "make install failed"
		exit 1
	fi
else
	echo "shells/bash - found"
fi
if [ $SHELL != "/usr/local/bin/bash" ] ; then 
	chsh -s /usr/local/bin/bash > /dev/null 2>&1 || echo "chsh failed"
fi

# Install bash-completion scripts
if [ ! -e /usr/local/bin/bash_completion.sh ] ; then
	echo "shells/bash-completion"
	cd /usr/ports/shells/bash-completion
	make -DBATCH install > /dev/null 2>&1
	if [ $? -ne 0 ]; then
		echo "make install failed"
		exit 1
	fi
else
	echo "shells/bash-completion - found"
fi

# Install vim-lite
if [ ! -e /usr/local/bin/vim ] ; then
	echo "editors/vim-lite"
	cd /usr/ports/editors/vim-lite
	make -DBATCH install > /dev/null 2>&1
	if [ $? -ne 0 ]; then
		echo "make install failed"
		exit 1
	fi
else
	echo "editors/vim-lite - found"
fi

# If using csh, rehash PATH
cd
if [ $SHELL = "/bin/csh" ] ; then
	rehash
fi

freebsd

I’ve started looking at FreeBSD at work this week, because I was reading some blog posts about how MySQL performs well on a combination of that and ZFS together.  I haven’t gotten around to getting ZFS setup yet, but I have been looking into FreeBSD as an OS a lot, and so far, I like it.

This makes the second distro in the past year that I’ve really started to seriously look into, the other one being Ubuntu.  I’m still trying to wrap my head around the whole FreeBSD design structure and philosophy, and for now I’m having a hard time summing it up.  In my mind, it kind of feels like a mashup of functionality between Gentoo and Ubuntu.  I like that there is a set group of packages that are always there, kind of like Ubuntu, but that you can compile everything from source, like Gentoo.

What has really surprised me is how quickly I’ve been able to pick it up, understand it, and already work on getting an install up and running.  I think that having patience is probably the primary reason there.  Figuring out how things work hasn’t really been that hard, but I say that because of past Linux experience that has helped me figure out where to look for answers more easily.  That is, when I get stuck on something, I can usually figure it out just by guessing or poking around with little effort.

Years ago, if I would have looked at any BSD, I would have been asking “why?”  I still don’t know why I’m looking at it, other than I believe it’s not a good idea to put all your eggs in one basket.  At work we already support CentOS, Gentoo and Ubuntu, and it’d be awesome to add FreeBSD to the list.

I’m really enjoying it so far.  It’s easy to install packages using the ports system.  I tried going the route of binary packages at first, but that wasn’t working out so well for me.  Then I tried mixing ports and packages, and that wasn’t doing too great either, so I switched to just using ports for now.

The only thing I don’t like so far is how it’s kind of hard to find what I’m looking for.  I totally chalk that up to me being a noob, and not as any real flaw of the distro or it’s documentation — I just don’t know where to look yet.  Fortunately, ‘whereis’ has saved me a lot of time.

The system seems familiar enough and easy to use for me, coming from a Linux background.  In fact, I really can’t find many differences.  The things I have noticed are that it uses much less memory, even on old underpowered boxes, and that it is relatively quick out of the box.  I never would have guessed that.

I’m curious to see how ZFS integrates into the system, if at all.  I like the filesystem, and it’s feature set, but that’s about it for now (I got to play with it a bit as we had a FreeNAS install for a few months).  If it’s a major pain to integrate it, I’m probably not going to push for it right now — I’m content with riding out the learning curve until I feel more comfortable with the system.

So, all in all, it’s cool to find something different, that doesn’t feel too different, but still lets me get my head in there and figure out something new.

If you guys know of any killer apps to use on here, let me know.  I’m kind of wishing I had an easier way to install stuff using ports aside from tromping through /usr/ports manually looking for package names.

what i’m reading: “real boys”

Summer is rough for me.  I take fewer classes, I have lots more free time, and things are generally a lot less unstructured.  This means my life is full of chaos.

One thing I’ve noticed about school, recently, is that if I’m not taking any psychology courses, I become indifferent about working towards a degree.  It’s hard slogging through generals for any student, but in my case, where there’s limited amounts of time and money to spend on pursuing an education, it just feels like it’s not worth the hassle.

So, summer is a little rougher for me, and I’m looking forward to Fall and Spring semesters again.

In the meantime, and I’ve been doing this for awhile, I always have one book about psychology or counseling that I’m reading.  Right now, I’m making my way through a great book called “Real Boys.”

If I could summarize the book, it’s basically documenting the effects of boys not expressing their feelings.  I was going to expound on that, but that’s just about how it goes.  I also use the term ‘boys’ here from the author’s context, not mine. He tends to cover a large age group, from about eight to sixteen.

Flipping it open tonight, the page I started on perfectly expressed the “why” I want to work with youth so much — or, that is, the kinds of problems I want to encounter and help people out with:

When boys become hardened, they become willing to endure emotional and physical pain–even to risk their lives–if it means winning the approval of their peers.  Boys can become so thoroughly hardened that they literally anesthetize themselves against the pain they must cope with.  And they are often left unsupervised at an earlier age than girls and are usually discouraged by adults from engaging in help-seeking behaviors at their time of greatest vulnerability or need, boys learn to remain silent despite their suffering.

Incredibly sad commentary, of course, but also accurate.

I suppose that the solution could be summed up in “love your kids,” but what I see happening is that culture is a strong influence of how to love them — when to cut them loose, when to have them “man up,” and so on.  Culture is a poor guide for determining personal milestones.

I’ve been learning more about counseling and people not just with what I read, but as I casually observe people and realize how simple things are.  The realization is dawning on me that humans are alike emotionally, wanting the same basics subsets of love and caring: respect, communication, validation, correction and instruction.  Things that people do that are “weird” or “out there” are most times going to be tied back to some fundamental need that is unaddressed.  And in the cases where that is the case, there can be a check for internal chemical imbalances (depression, schizophrenia, OCD, mood disorders etc.) where medication can do a lot of good in providing more stability.

On a personal level, not an academic one, from helping out others, I’ve noticed how important it is that people have someone that will look them in the eye and listen to them.  I’ve noticed that just looking at someone directly often times can slightly startle someone, since it is so unexpected.  I’ve seen though, how talking calmly and directly to someone will both relax them and engender some trust.  People just want to be listened to.

Anyway, it’s all fascinating stuff, and I love reading up on it, and discovering new things.  In a lot of ways, I’m finding that counseling is based on really simple principles of caring and communicating.

rebooting my mini-itx

It’s been a long time since I’ve worked on much anything computer-related as a hobby.  Things have changed quite a lot in the past year.  I moved to a much smaller apartment in Salt Lake, which is about a third the size of my old place.  The idea was to trim the fat and focus on going back to school, which is my major direction in life these days.  When I moved in, I didn’t have room for setting up a desktop computer anywhere, so it’s been just my netbook and me.  That suits me plenty fine, though, I wasn’t really using it that much either.  I had just upgraded to a six-core so I could rip DVDs much faster, and now it was sitting headless wherever I could find room, and even then, only used occasionally.

It’s not just at home that things have been changing.  At work I got to make the transition from programmer to full-time sysadmin, and I’m absolutely loving it.  I knew I was getting tired of coding, and I had always enjoyed just taking care of servers, and now I get to do that all day long. When I initially started as a sysadmin, I didn’t think our small company would have enough work for me to do after a few months.  In actuality, I’m kept busy all the time.  The part I like the most is that part of my job is doing research, how to do things better, more efficiently, anything to make the workload easier.  It’s fun.

On top of all that, my school attendance is starting to ramp up more, and I’ve been consistently drifting to adding more classes to my workload.  All this stuff has basically booted Linux out of my life as a hobby, and so now I need things to “just work” without hassle, so I leave my installations alone.

One thing I’d been neglecting a little bit was my entire HTPC setup.  I hadn’t been using it much lately just because I would mostly stream some Netflix (yay, Doctor Who!).  My setup has been a beast though, normally running for months on end without the slightest hiccup.  What started to happen though is that I would come back to using it, switching my HDMI input over, and the box would be powered off for some reason.  Most of the time, I would either power it back on and go on with life or just ignore it.  Until one day it wouldn’t power on at all, and I just shrugged it off and determined to look at it later.

Well, later turned out to be finals week, when my brain has been working overtime, and I seriously needed a hobby.  I pulled out my main frontend and started looking at it to see what was going on.  It was plugged in properly and everything looked legit, but when I hit the power, the CPU fan would start up for a second and then everything would stop.  After fiddling with it for a bit, I started to notice that something was smelling burnt.  Once that happened, I abandoned my diagnosis.  Even if I did manage to get it working, I didn’t want it to catch everything on fire.

At the same time, my external USB drive enclosure died on me.  So even if I could have gotten it working, I still wouldn’t have had a way to watch my shows.  Them giving out on me hasn’t bothered me in the least — the entire setup has been running flawlessly for years, and I’d managed to get a lot of mileage out of them.

Now I had to decide what I was going to do.  I have a lot of hardware, but in pieces.  I have four mini-ITX boards altogether, two of them are VIA C7 chipsets, and the other two are Zotac boards both running low-powered Celeron CPUs (around 35W if I remember correctly).  The power supplies for the VIA boards use 20-pin connectors and only run at about 80W, and aren’t enough to handle the Zotac boards which use 24-pin connectors.  So I have this mix of hardware, and nothing powerful enough to act as a frontend.

There are some great packaged systems out there now where for between $200 to $300 you can get an entire package in one go that does exactly what I’m putting together myself. I considered the idea of just starting over, but I decided that it’d be cheaper to just salvage what I could.

So this week I ordered a new USB HDD enclosure, and I also ordered a new power supply for the main Zotac board.  I found a site that sells really small power supplies for mini-ITX boards, called picoPSU.  The design eliminates a lot of the hardware that I would normally need to get all the power to my box.  I was really skeptical about them when I first heard of it, but did some looking around and it looks like it’s exactly what I need.

In the meantime, I ripped out my motherboard out of my desktop, and put both Zotac boards in there to make sure they still work, and thankfully they do.  I got the old setup pieced together using my desktop case, and fired up the old system to play around with it.

I had started to forget how much time I put into this thing.  I forgot that I had put countless hours stitching this thing together, running a custom build of Gentoo suited to run on small environments.  On top of that I made hacks to mythvideo and got those working to polish off some rough edges.  It just started to come back to me how much I’d worked on this … and how much fun it was. :)

I played around with my frontend a little bit, and fired up a few movies just to try out the surround sound.  It was awesome.  I’d forgotten how nice it was to have that huge library on demand, too.

So I’m excited now to get things up and running.  It’s been a good little while.