planet larry migration

I wrote not too long ago that I was going to give up maintaining Planet Larry and look for a new owner.  I got a lot of offers from a lot of people who were willing to step up and help take care of the project for me — thanks guys!

Of all the submissions, one really stood out in my mind as someone who had some really great ideas for the site and was excited to take on the project — Daniel Robbins.  Daniel is the original founder of Gentoo, and currently maintains Funtoo.  He and I have worked together in the past, and despite the controversy his name may bring being attached to the project in the eyes of some, he has assured me that he will run things objectively.  I trust him, and believe there won’t be any issues.

We’re still working on the migration, which mostly means waiting for me to get some free time (crazy schedule), and part of that was announcing publicly that we have a new admin for the site.

Thanks, guys, for all your help in running the site.  It’s been good for me to work with so many users.  I always enjoy getting another perspective on how Gentoo projects are used in the wild, and this has helped a small bit to share that view.

new feeds

I’ve been having a slew of issues running Apache on my Linode VPS, which I’m still trying to pin down, so in an attempt to offload some of the usage, I’m now going to use Feedburner to provide the RSS feed for Planet Larry.

I know I’ve played with Feedburner in the past, and kind of flip-flopped on whether to use it or not, but this time I’m sure I’m gonna stick with it.  It’s better for users, since they will always have a feed available (whether I have issues or not), and it’s better for me since I can offload that part of the network traffic, which is actually quite a lot.

I’ve already updated the feeds and my apache config to do a permanent redirect, but if you want the feed URLs directly, here they are:

Sorry for the inconvenience.  It seems like everytime I post about Planet it’s bad news or maintenance.  Believe me when I say that it aggravates me far more than it does you.

Specifically, the issues I’m having is that Apache is sucking up all the available RAM, of which I only have 360 megs on my account.  It’s then rolling over to using all the swap space as well, which only slows things down even more.  I’ve just started playing with tweaking the MPM configuration a bit, and I’m still trying to find a reasonable solution for my configuration.

In the past, the Linode had been seizing up occassionally, and I’d normally just reboot it and get on with my life.  Recently, I installed monit (an awesome app), and pinpointed that the issue seems to always be with apache.  Now, I’m just trying to narrow it down even  more from there, but offloading the RSS feeds seems like a good step to take anyway … I get gigabytes of traffic per month just on that, believe it or not.

I’m toying with the idea of setting up lighthttpd instead, but I really prefer apache, and would rather set it up to behave in a low memory environment instead.  So, for any downtime in the near future, chances are it’s just me tweaking something.  At least now, thanks to monit, I have a much better idea of when something goes wrong.

Oh, one other tweak I’ve made is that the planet script itself is more robust as well.  That thing used to run out of control, but I’ve made some changes that will ensure that if it runs away, at least it won’t bring down the system.  I also started playing around with the idea of writing my own feed parser to replace the Planet software completely, and it looks like it’s going to be much simpler than I imagine.  I haven’t actually started down that path yet, since I have bigger projects to complete, but I’m actually enthusiastic that it’d be far, far simpler than I imagined.

planet feeds

I’m actually starting to like it when any planet problems pop up, because I’ve been systematically weeding them out one by one. The frequency of one user “spamming” the feed has been going down quite a bit. I think I found the major culprit for most every case though: Planet combined with WordPress’ Atom feeds.

I don’t know if this is to spec or not, but some version of WordPress sometimes will set the updated date to all posts, regardless of their original publish date. The default planet configuration is to display the posts that fall within the recent date range, using updated over published when pulling the Atom XML files.

What I have been doing is just tweaking each user’s config whenever the problem arises and then flushing the cache and repulling everything, and that has worked out well. Today, I finally went on the warpath a bit for Planet Larry, and just changed everyone’s config who was using an Atom feed to use their RSS2 one instead. Plus, I globally set the config to ignore the “updated” tag in feeds (in case any Atom ones do creep in).

I guess I’ll see what happens for now. Hopefully, less issues.

Edit: And, of course, after clearing the cache … one gets through. Rawr.

Edit: That won’t work either, it screws up LiveJournal posts.  Bleh.  Well, I think it’s all fixed now, though Google Reader might be whacked for a bit … but they’re doing some caching on their end, too, not much I can do about that.

which rss reader do you use?

So, I’m going to try and take a poll here, what program do you use to read RSS / Atom / news feeds? I would have an *actual* poll, but I honestly don’t even know what’s out there, much less what the popular ones are. Please leave a comment if you can.

The reason I ask is because I’m trying to debug some issues with planet — both Gentoo’s and Larry’s.

Oh, and apologies to everyone for breaking Planet Gentoo today. I spent a lot of time cleaning up a *lot* of cruft that should have been fixed for some time now. Chances are, too, that things will only get a bit more misarranged over the next few days as I tweak and test things some more, so bear with me.

Update: jsled had a good comment — why not just look at the access logs? Well, duh, why didn’t I think of that. -_- I don’t have access to Planet Gentoo’s, but Larry breaks it down well enough. It looks like, as far as programs go that are leaving a user agent, akregator is first with 19%, liferea is next with 6%, then Gregarious, RSSOwl and NetNewsWire with all less than 1% each.  The rest are browsers or Unknown.

planet admin stuff

I’m working on cleaning up some planet stuff again, and for Planet Larry I made a small policy shift — if your feed is spitting out partial feeds, I’m not gonna accept it anymore.  Currently it only affected like two people, I think (unless I missed someone), so if you wanna get back on … fix it. :)

I realize it’s a small case of me shoving my own preferences down the pipe, which is something I avoid as much as possible.

If I’m going to add on one more asinine policy requirement, I figure it’s fair that I take one away as well, though most people probably don’t notice or care about this one. I prefer to have the blogger’s name show up as the blog title, and have in just about every case (minus one) forced them to be that way.  But, I’ve changed my mind there as well, so if you want to customize the name on the feed, I can do that for you.

An example would be that my blog (wonkablog) shows up in the list as Steve Dibb (my name).  So, if you wanna change it from displaying your name to something else, just drop me an email.

And, as always, more bloggers are welcome to sign up. :)

planet larry google search

Planet Larry has kept archives of posts for about five months now, but I just barely added a Google search form to the side nav so you can … uh, search.

It would be nice to have the archived pages put the dates in the title tag, but looking through the Venus code I couldn’t really see a quick way to do that, so meh.  I could probably just write a quick sed line to fix it when its archived, I suppose.  I’ll look at it later.  Or let someone else figure it out for me. :)

Oh, also, just a reminder, Planet Gentoo stores archives as well, though there’s no index page or search index yet.  Another item on my long to do list.

I don’t know about anyone else, but everytime I want to go Planet Larry, I still type in, even though I ditched the domain a few months ago.

Well, I got tired of it not working, so I re-registered it, and it redirects once again as normal.

Also, we can always use more bloggers — if you have a Gentoo blog, lemme know about it, and we’ll get you added.  It’s a very informal process, just send me an email with your blog URL and stuff.  Now that I think about it, I really need to catch up with all the new Gentoo devs and get them on Planet Gentoo as well. Slack…

Finally, I decided I’m going to create a feed specifically for ex-developers, but since I’m too lazy to go out and find their blogs (and I don’t think I still have an old copy), if you guys could send me your info, that would greatly help to speed things along. Update: It’s too much work to create a separate feed, so I just put them back in the main feeds. Now, behave. :)

And here’s an image just because this blog post is so boring, it needs one.

planet larry: maintenance cleanup

I’m still getting overdue stuff taking care of, this time in Planet Larry. I’ve finally cleaned up the scripts and the configs to make it much, much simpler to debug and run the update process. Not only that, now it runs about 15 times faster (under one minute, actually).

Something interesting I realized the other day is that Universe gets about 5% of the traffic that Planet does. As a result, I changed it from updating twice an hour to once every 2 hours. Planet, on the other hand, I changed it from twice an hour to every fifteen minutes, and I might make it update even more frequently than that.

Anyway, all that’s changed is a bunch of boring backend stuff, which was some much needed cleanup that I’ve been putting off for a very, very long time. The fact that planet has been a maintenance nightmare is what has mostly kept me from dragging my feet on getting anything done, but now that that headache is gone, I feel hugely relieved.

There’s still more stuff to do, but I think I’m over the worst of it. What I’d like most is to work on next is getting the actual frontend cleaned up, and get a new design. That’s not really my thing, so if someone can help out, that’d be much appreciated.

planet, packages: small bugfixes

I took a few minutes today and cleaned up a few small bugs on Planet Larry and friends.

One thing I get asked for every now and then is if I have archives of past posts. Well, I do now. I just copy the HTML file of the last post to $date one each run. A simple and unelegant solution. I’ll be doing the same thing for Planet Gentoo soon.

Also, fixed the FeedBurner link on the main page — I didn’t even realize it was broken. While I was at it, I created one for Universe as well.

On the packages website, I finally fixed it so you can search against just packages again. That’s been annoying me for a while. By default the search is way too wide, I think. It will search the full atom, the package description and the package name. I have to do the package name twice because of regular expressions (starting with, ending with, exact matches, etc.). And there’s still no simple way to search for packages containing multiple words, which is also an annoying little bug. Advanced searches for GPNL and Packages has been something I’ve wanted to do for a long time, but have been putting off since I started the projects. Sheesh. Every time I sit down and start to poke at it, though, I realize just how big a beast it is, based on what I’d like to accomplish. I really need something for the interim, though.

Anyway, I better quit before this post gets any more boring. One last thing — we can use more users who are Gentoo users and have a blog on Planet Larry. Just drop me an e-mail and I’ll get you setup.