Gentoo Love: Intro to Portage Sets

A few quick gentoo tips before we get to the instructional material.

1. Emerge world as often as you can. Once a week is probably a good frequency.
2. Unless you’ve got a rock solid track record emerging world, don’t do it when you are really tired.
3. In the best interest of the above, and your sanity, minimize the number of tilde keywords you have unmasked. I got all adventurous with one of my servers some years ago and there’s no good exit strategy. Sometimes it’s a pride point, other times, just pain.
4. Don’t unmerge zlib. Just don’t, even if you are planning to put it right back. Pretty much nothing can run without it. Not portage, and not make, so you can’t get it back really. If you did, just copy libz.so (and symlinks) from another machine (probably of the same architecture).
5. Also, don’t run eclean. It will break a lot of ebuilds and can’t even do that particularly well.

Getting setsy with portage

One of the main disadvantages of portage is the generally poor grouping of related packages. There are:

• Package Categories (media-libs, dev-python, etc.) – These are pretty great, except when a package’s category changes and some ebuilds don’t pick up on that change. There is some way for portage people to redirect old ebuilds to new packages, but it has failed me more than once. Also, package categories don’t speak much to dependencies (not that they should). Portage/ebuild people decide on these, though the project maintainers might have some say.
• Meta Packages (kde-meta) – Not many of these, but they seem to mostly be a dependency container. They are fine for installing, if a bit opaque, but they can be terrible for uninstalling. I’ve uninstalled old metas that left their obsolete and orphaned packages strewn about (kde 4.2 stuff). These are made by the project maintainers, I think.
• “Profiles”(world, system) – Not a great name (maybe the wrong name?), but these aren’t as helpful and commonly used as they should be. Emerging world is definitely useful, but there could be a more granular operation between individual packages and ALL PACKAGES. These are generally automatically constructed for the user, though the user can do some manual editing.

Oh KDE, you slay me.

Enter sets

Sets are basically like profiles, but the user gets a lot more control. They are groups of packages that can be reference like:

emerge --ask --update @my-set

You’ve got to admit that’s nicer than doing something like this.

Now when I said “Enter sets,” I meant… almost. Sets are only available in portage 2.2+. So the first step is to get that. Before that, I want to mention that it is technically still in alpha and a broken portage can make it hard to revert to a working portage. Nevertheless, so far it works fine for me and a lot of others have been using it since early 2009. In /etc/portage/package.unmask, add:

>=sys-apps/portage-2.2.0_alpha8

And in /etc/portage/package.keywords add:

sys-apps/portage ~*

Now, just emerge --ask portage and make sure it’s gonna pull in 2.2.

Your first set

Long story short, the format of the most basic user sets is just like the world file under /var/lib/portage/world. Just make a file with a list of packages, one per line, and put that file under /etc/portage/sets. For example, a set of scripting languages:


dev-lang/nodejs
dev-lang/php
=dev-lang/python-3.1.3
>=dev-lang/ruby-1.8.7_p249-r2
dev-lang/tcl


Now you can refer to that list of package like:

emerge --ask --update --deep @my-scripting-set

It may be prudent to prefix your sets so they don’t conflict with any other packages.

How to really clean up your system

By now you’ve probably seen the light, but I’m going to share one of my favorite uses so far to drive the point home. Say you’ve let your system go for a while, and you’ve accumulated some packages. Maybe you’ve switched from Gnome to KDE or maybe all the way to xmonad; regardless there is cruft to be removed. Here’s how you clean that stuff the Right Way:

1. Update your gentoolkit and portage
2. equery for some packages to remove and save the list:
   

   equery list kde-*/* > ~/kde_installed_packages_12122010

3. Review the result and format properly. equery gives specific versions by defaults, so we’re just gonna throw ‘=’ in front of every line to make them valid package atoms. We’ll use sed:
   

   sed ':a;N;$!ba;s/\n/\n=/g' ~/my-kde-set

   You’ll have to manually add one more equal sign for the first line, but that should work.

4. Review the result and move the file into place:
   

   mv ~/my-kde-set /etc/portage/sets/

5. Depclean and unmerge:
   

   emerge --depclean --ask --verbose @my-kde-set

Ah, so fresh and so clean. You should move the set file out of the /etc/portage/sets directory now.

Conclusion

Sets fill a much-lamented (by me) gap in portage. They add organizational power without removing fine-grained control, without which Gentoo would not be Gentoo. My only concern is
Why did this take so long? Given that this is basically how ‘world’ has always worked and we’re well into version 2, we should have had this ages ago. Also, sets are really not that complex or tailored to package management, I wonder if archlinux or some other distro has solved this better. Of course, I’ve only scratched the surface, and there’s a lot more you can do with sets, it seems. For a good starting point, you can do a search on sets.conf.

---

P.S. The screenshot above was brought to you by ImageMagick:

import -window root ~/screenshot.png

---

Advertisement

LESS and incron: CSS at its finest

less is, once again, more.

Web development is full of challenges. That’s my nice way of saying writing CSS blows. CSS is powerful, but at the cost of being too fine grained and low level for easy development. It’s like the assembly of web design. Other developers are all-too-aware of the situation and have come up with a few solutions, including CSS frameworks, which reduce the amount of from-scratch code and provide a system (e.g., Blueprint or the 960 grid system), versus the freeform mess of raw CSS, and CSS extensions, like LESS, which is the topic of the day.

Act One: L-E-S-S spells bliss

If you aren’t using something like this yet, you might as well be punching yourself in the crotch every time you code.

I have also messed with Sass, which was not as “Syntactically Awesome” as LESS, and xCSS, which was overkill (but I might revisit it later). LESS is good because:

• Any standard CSS file is a valid LESS file – easy to only use the features you need
• Just as powerful as Sass, offering variables, functions, nesting, CSS-specialized math operations
• Aptana CSS highlighting works great:
1. Go: Window->Preferences
2. General->Editors->File Associations
3. Add file type: *.less
4. Add editor: Aptana CSS Editor (right at the top)

I mean, look at this syntax:

@left_column_width: 300px;
@column_margin_width: 15px;

/** Palette **/
@light_blue: #d0dae3;
@pastel_blue: #7492ac;
@med_blue: #1e5d97;
@dark_blue: #0b3c68;

@light_orange: #f6b860;
@pastel_orange: #dfab62;
@med_orange: #e9951f;
@dark_orange: #a1630c;

/** Colors **/
@top_nav_color: @light_blue;
@top_nav_hover_color: @light_orange;
@header_color: @pastel_blue;

#header {
	clear:both;
	float:left;
	width:100%;
	border-bottom:1px solid @dark_blue - #111;
	background: @header_color;
	padding-bottom: 12px;
	margin-bottom: 7px;
	
	.page-title {
		float:left;
		clear:none;
		display:inline;
	}
}

/***** Two column layout a la   *********
****** http://matthewjamestaylor.com/blog/ultimate-2-column-left-menu-pixels.htm  *****/

/* column container */
.colmask {
	position:relative;
    clear:both;
    float:left;
    width:100%;			
	/*overflow:hidden;*/	
}

/* 2 column left menu settings */
.leftmenu {
    background:#fff;
	overflow:hidden;
	.colright {
	    float:left;
	    width:200%;
		position:relative;
		left: @left_column_width + (2 * @column_margin_width);
	    background:#fff;
	}
	
	.col1wrap {
	    float:right;
	    width:50%;
	    position:relative;
	    right: @left_column_width + (3 * @column_margin_width);
	    padding-bottom:1em;
	}
	
	.col1 {
	    margin:0 @column_margin_width 0 (@left_column_width + (4 * @column_margin_width));
	    position:relative;
	    right:100%;
	    /*overflow:hidden;*/
	}
	
	.col2 {
	    float:left;
	    width: @left_column_width ;
	    position:relative;
	    right: @left_column_width + @column_margin_width;
	}
}

Now you have an easily customizable two column layout and color scheme. Change your values in one place and they gracefully propagate. If you wanted to change the colors or column width before, you would have to change dozens of values. Really, stop what you’re doing, change your CSS file’s extension to .less, and become a happier person.

Act Two: Incron

So the only problem that I ran into is the bump in the workflow: compiling from LESS to CSS. This is where incron comes into play. Incron monitors files for changes and can trigger actions as specified by you with a cron-like syntax we can all love. I have a little Gentoo development box I use; I put incron on it and set it to run lessc whenever my LESS file changed. Just install/emerge incron, run incrontab -e and add one line:

	/path/to/less/files/mystyles.less IN_MODIFY lessc $@

Now you’ll have a file called my_styles.css and everything will be hunky dory. Of course this could easily be extended to do more powerful things with your styles, like move them into the right place or give them fancy names or version them or whatever. The potential here is also not limited to LESS, so consider it an investment. If this intrigues you, I think this tutorial should be all you need for now.

Well, web development is a big basket of ugly and this only addresses one aspect. There’s still cross-browser grossness and JavaScript debugging horror. But LESS helps. LESS helps.

Hit and Run: Google Android

Google is awesome and Android is a great concept: a really lofty ideal that pretty much only a giant like Google could have a hope of realizing. When mobile Linux didn’t take off on its own and it started to become clear that there wasn’t going to be a large scale open-source mobile platform, they acted with insight and decisiveness, announcing Android in 2007, and first releasing it in 2008. Now here’s why I think they’re going to blow it.

I recently sat down with the 'droid for some hard-hitting back-and-forth (forever)

Android is an operating system, but not generally used by manufacturers in unmodified form. That’s because manufacturers want to differentiate their hardware and first-party software offerings. If they want to compete, they pretty much have to. I would say this is akin to Linux splintering off into a million flavors; each flavor has its own App Store equivalent and its own community for making sure that App Store works. Gentoo doesn’t readily work with apt or yum, but the Gentoo community keeps Portage working. Same goes for Ubuntu or whatever. Developers don’t worry about their package working on bizarre distros because there’s a community for that.

Manufacturers mess it all up by making their own flavor and expecting developers to support it. They’ll tweak or leave out functionality and still call it an Android phone. Google wants Android to get popular like Linux (moreso, probably), but it won’t happen because everyone keeps passing the buck on keeping software running seamlessly across hardware. I don’t see how Google can regulate, or how developers can keep up, so it comes back to the manufacturers. Remember manufacturers are the ones who couldn’t afford to develop a decent operating system of their own in the first place; they’re inherently greedy and they suck at software. I foresee a rough road for our boy Android.

SILT: Stuff I Learned Today

This is my dad wearing a pair of shades I found.

This is the first in an indefinite series of catch-all blogposts. Now that school is up and my schedule is more regular I think I can make a few more minutes a day to log some findings and post some links. Also, I’m thinking that by presenting a more innocuous task (a short, general post, as opposed to a long, targeted one), I might find myself sowing the seeds of expanded posts.

Also, sometimes I feel silly calling up friends and telling them each individually about cool junk when I know they’re all subscribed to this baby right here. Content will range from factoids to news to mini-rants to Linux and beyond. So:

• Peep Show Season 6 has started. I might need to cache up the whole season before I start, it’s such dark goodness. 9.5/10 on imdb with 5,200+ votes? Intense.
• I found out a way for a Linux machine to dynamically get a hostname on a Microsoft Active Directory network that isn’t set up to update hostnames via DHCP. This was ridiculously hard and I strongly suggest you contact me if you ever run into this issue. It involves kerberos and this one-off script.
• I’m watching Kate Humble’s 4-part series of Middle Eastern travel, The Frankincense Trail. It’s from the BBC and it’s not too bad. It probably is a bit too frankincense-centric to be honest, what with her hauling her own little load of frankincense everywhere and asking everyone whether their people use or used frankincense. The most interesting tidbit I picked up is that apparently some parts if not all of Israel enforce some crazy Shabbat rules. For instance, you can’t use (technically be directly involved in the use of) anything electrical. You can ride an elevator, but you can’t press the buttons (it stops on all floors). You can open a fridge and grab something, but you have to tape the light sensor down beforehand. All this because it’s the ‘day of rest’. I’m wondering how universally this stuff is implemented.
• Last, but certainly not least, Oyama is back up at the makuro.org address. My $30 Time Warner lets me upload at like 300kb/s so please be courteous and only max out my connection at night or during the workday. Also, I’ll know who my true friends are based on who logs in first (only my true friends use RSS and FTP).

Well, here’s hoping that wasn’t too painful, because I’m planning on learning a lot of stuff and posting about it. And then, this series of blog posts, like its namesake (silt), will provide me and my offspring a fertile farmland on which to raise agricultural goods.

Is Gentoo dying?

I started using Gentoo linux for a few reasons:

1. To learn the inner workings of linux, as opposed to glossing over them with ubuntu or some other “user-friendly” distro.
2. To learn more about the software I use. Gentoo really lets you configure all the various features of the software that I use. Hooray configurability!
3. Because it had great documentation, like the Gentoo wiki. I was using OpenSuSE at the time and I kept finding myself reading HOWTOs about Gentoo and applying them to OpenSuSE .

Anyway, now there’s all sorts of ruckus about the Gentoo Foundation being leaderless and maybe I’m just not paying attention, but I haven’t heard anything promising about that situation in a while. Also, the Gentoo wiki had a massive data loss due to an unexpected data center closure, and it hasn’t recovered since (it’s been a while now).

As for the other benefits of Gentoo, well, there are other power-user distros, even some that give you good package management that does its best to stay close to the source. I’m looking at Arch Linux at the moment. Arch Linux also does a ton of bleeding-edge software stuff, which I find myself doing a lot on Gentoo nowadays.

On that note I should mention very subjectively that recently in order to upgrade I’ve had to blindly work around a lot more issues than when I started out. One could argue that it’s something about KDE 4.2, but there’s something else going on here as well. There’s way too much unmasking and way too much manual dependency resolving for the type of basic stuff I’m trying to do, stuff that I’ve seen portage do the equivalent of in the past. And I’m doing it during a major documentation shortage, one that doesn’t seem to be coming to an end.

Well, if I switch, you can be sure I’ll post my whinings about my next distro right here.

mplayer-fu

I am a big, corpulent fan of thesauruses and mplayer. Mplayer has about a billion command line options, as any respectable linux staple should. Unfortunately, I’ve had some difficulty tracking down some of the more useful options. For instance, I was looking for a way to trim a sound file from the command line.

With mplayer you can specify output devices, so the command to play a song through your speakers and the command to write a new file often differ only by the output specification. Where this comes in is that the command I was looking for is also useful for playing a segment of a sound file.

I googled around for “mplayer trim audio file”, “mplayer crop audio file”, “mplayer start end command line options” (to see if the ones I wanted were in the main list), and “mplayer start end positions”. I think I ended up going through my own shell history until I found the commands I used for creating GIFs from movies (like I said mplayer/mencoder is great). There I found the elusive -ss and -endpos command line arguments. Why ss is ss and not startpos, I don’t know.

Next time you’re looking to trim anything or play anything, use the following commands as a template.

mplayer -ss 16 -endpos 1:21:33 [additional options] [input files]

Note that ss and endpos have the same input specification. So you can go by number of seconds, or mm:ss, or hh:mm:ss. endpos is the number of seconds you want to capture/play, so ss=5 and endpos=10 will play the 10 seconds starting at the 5th second.

(Note here that I had some strange results on one file, but that could have just been mplayer.)

Anyway, I hope that was helpful. As a postscript, here are some other examples for you.

Create a trimmed wavefile from an input file, including ripping audio from video files:

mplayer -ss [time] -endpos [time] -ao pcm:file=output.wav [input_file]

If you want to trim a video, you’ll have to switch to mencoder. Don’t worry, ss and endpos still work:

mencoder -ss [time] -endpos [time] -ovc copy -oac copy -o mytrimmed.avi [input_file]

(There might be a way to do it with mplayer, but I don’t know how. This works fine though 🙂 )

Digikam is great

I recently spent 3 days going through back photos, culling and organizing, in an attempt to tidy up my file structure and generally feel more caught up. After all that, I felt like the job was somewhat incomplete, since folders arranged by date, location, and sometimes subject don’t really amount to much in the way of findability.

What I need is tagging and search functionality in a fast package. I’m not big into posting photos online, even on more professional services like smugmug. It takes too long to upload and the proportion of photos I want to share is very low. Because all of this stuff is on my server’s RAID5, and my server runs kde 3.5 on gentoo, there’s basically only one legitimate choice, digikam. It tags, it searches, and it’s fast. It handles RAW files (definitely don’t want to upload those). It can even embed the tags in the comments in the JPEG files (though not by default). All metadata is stored in a sqlite database that it keeps in sync with the files.

It does make a copy of whatever pictures you point it at though. But, the filestructure is preserved and everything looks just like it used to, so I can’t think of a reason to keep the untracked files around.

It’s served me and my 20GB or so of pictures well. There’s a screenshot attached.

a screenshot of my install of digikam on kde 3.5

Addendum: I’ll be upgrading from digikam 0.9 to 0.10 when I upgrade from kde 3.5 to 4.1 when it’s released (later this month I think). KDE 4.1 uses qt4, which is gonna be dandy for amarok 2.0. I’ll probably post about that, toowoowoowoo.