I Sansa no more

Well, I did it. My Sansa e280 is now once again the property of the Bellevue Circuit City. The gadget had pretty sweet hardware (holding my entire music collection in the palm of my hand was a cool thing), but I have no real need for it -- certainly nothing that would be worth putting up with Sandisk's attitude regarding non-Windows computers.

I'm currently trying to decide whether to remain portable-less for now (on the theory that portable media players are likely to advance ridiculously in the next few months), to buy a simple 4GB or 2GB Flash player, or to give up on Flash for the time being and get a hard disk-based player. I'm leaning towards waiting until the 4GB model of the Cowon D2 becomes available in the U.S., and then ordering it -- its feature set is pretty amazing, and hopefully 4GB will be enough storage capacity (especially considering that you can insert an SD flash card to get more storage space). In the meantime, I can cover most of what I was using the player for with a combination of podcasts, .oggs of my music, and borrowing Kate's portable FM radio. Less convenient, but entirely workable.

PR techniques

It was recently reported that the scientific publishing industry had hired a PR heavyweight to attack the movement for the free and unencumbered dissemination of scientific results. I was especially struck by the slogan he supposedly recommended to the publishers:

Open access equals government censorship.

Just read that one a few times. I've found it gets funnier every time you do.

The Internet generation's lament

For people who read my blog and not Planet Debian, I have to quote this comment on one of Biella's recent posts, by Luis Villa:

The world is producing interesting things for me to read faster than I can read them.

This is so, so, so true.

Sandisk doesn't want your money for a music player unless you run Windows

I bought a Sansa e280 last weekend to replace my aging and bulky Neuros (the last straw was when I lost the charger for the Neuros). It's nice to have such a small, light music player, and having 8GB of space is a big bonus. But when I went to the Sandisk web site to get a new firmware, I discovered that they only provide a .exe of a magic firmware download program. This is despite the fact that, as you can find by searching on the Web, the actual firmware upgrade procedure is just "copy the firmware file onto the device".

So, I wrote to support, mentioning that I don't have a running Windows installation and asking if they could provide an alternate means of updating the firmware. Here's the response:

Please be informed that your Sansa e280 is supported on Windows XP. If the operating system of your computer is not Windows XP, if you have just bought your Sansa e280, we do recommend that you contact the store where you purchased it and inquire about replacements that can be supported on your operating system or if that store honor refunds, please do ask for it. Please do provide the proof of purchase and packaging for verification purposes.

Now, I can work around this problem -- but after reading that brush-off I think I might just take their advice; as it happens, I saved the packaging just on the offchance something like this came up. Can anyone recommend a decent, small, flash-based media player that works well with Linux? Must-have features are:

• FM radio (this eliminates, e.g., the Archos).
  
• Some support for non-encumbered file formats.
  
• Support for installing firmware upgrades from Linux.
  
• Support for syncing music from Linux.
  
• Either .ogg support or a lively (even if unfinished) Rockbox port. (Rockbox is actually preferred, since I'll probably just switch over to it if possible; this was one of the deciding factors in my purchasing an e280)
  

I still haven't made up my mind: I might hang onto the e280 out of inertia and wait for the Rockbox guys to figure it out (it looks like they're making pretty good progress). But if there's a better vendor out there that actually wants my money, I'd love to hear about them.

Package management usability

[UPDATE: I'm closing comments on this post, because the spammers have discovered it -- dnb]

Lars says:

The uncomfortability of Synaptic may be partly due to the fact that it is
a different model from Windows: you get all your programs from one place,
instead of anywhere on the Internet, partly due to just not using Synaptic
much at all, and partly due to Synaptic not being perhaps as easy to
understand as possible. I personally find Synaptic to be very nice, and
use it regularly, but I am biased by having used Debian package managers
for over a decade.

Now, synaptic is a lot nicer for users than scary console frontends like aptitude. But it's really a tool for "power users". Here's a quick list just of what I see when I start it (bearing in mind that my UI design skills suck):

• The "quick introduction" screen that pops up is way too long. The fact that it needs to be so long is already bad.
  
• There's a huge, and I mean HUGE list of sections down the left. What are these? Which one do I look in? For that matter, what is "gnome", and are "shells" programs for the seashore?
  If I want to find software for processing video, do I look in "graphics" or "multimedia"? (and what are these funny "(contrib)" markers?) Which of the huge number of packages in each section do I want? From the point of view of a new user (especially), this categorization is useless, not to mention often wrong or misleading.
  
• The four buttons in the lower-left, which change the display, are a good idea, but they don't say what they do. A label saying something like "Group By" would go a long way here.
  
• I think the version numbers in the main display probably provide too much information for a novice user. They won't really know what to make of them, and if they need the information it can be provided on demand; as it is, it's just one more distraction to sort through while looking for what they really want. The extra space could, e.g., be used to provide more room for the status column (currently labeled "S").
  

So much for superficial observations. There are a few more things that I see that are worth more discussion.

Mechanics of selecting packages

To change the state of a package, you click on the unlabeled little box to its left, then select the correct option from a list containing "Unmark", "Mark for Installation", "Mark for Upgrade", "Mark for Reinstall"...you get the idea. Even if only one option is valid, the menu still appears with all but one option grayed out, and you have to select the one you want.

This is complicated, cumbersome, and surprising. It also introduces terminological confusion, since "Unmark" isn't really what you think of when you go to cancel an installation or whatever.

The best option I can think of here is a pulldown menu (which I believe the GTK+ list widget (god, I almost wrote "control") supports). The menu should default to the current state, e.g., "not installed", and should have options that describe what will be done with the package, e.g., "install", and so on. Only valid actions should appear on the menu. The Package menu should probably adapt in the same way as you select packages. I don't recall offhand how GTK+ menus work, but if possible commonly used commands like "remove" should be closer to the mouse when the menu appears than uncommon commands like "reinstall".

Another choice I can see, if you want to avoid menus, would be something like this: have two checkboxes, one for "install" and one for "upgrade". "install" would be checked for installed packages and not checked for not-installed packages. Checking or clearing it would mark packages for installation or deletion. "upgrade" would mark or unmark packages that were installed for upgrade. Of course, this introduces a redundant column for not-installed packages, which I don't like. It also seems to me that it would be less obvious what's going on for new users. But it would be more efficient for power users (who should all be using aptitude anyway ;-) ).

Progress bar madness

I haven't extensively used synaptic, but several core misfeatures of apt and our archive pretty much guarantee that if it displays progress bars for key operations like loading the cache or updating the package lists, the progress bars either run multiple times (i.e., don't actually tell you how close to finishing you are) or actually go BACKWARDS -- yes, BACKWARDS.

The great package hunt

Maybe the biggest problem is that unless you know what package you're looking for, basically all our package interfaces are useless. There are lots of fancy searching features you could add, but actually a quick test in synaptic suggests to me that this would be going in the wrong direction. Unlike aptitude, synaptic includes descriptions, and searches for stuff users might actually want (like, say, "burn CD") seem to turn up relevant results (i.e., software that burns CDs).

I'm sure the search algorithm could be improved (with inexact matching, better indexing (a package description suffix tree?), etc), but the big problem I see is that you tend to get back several dozen results, many of which are libraries, development packages, command-line software or software that doesn't work; some of the other software is software that can burn CDs but whose purpose is not to burn CDs (e.g., backup software).

How can we solve this? The first thing that springs to mind for me is to use popularity-contest data to weight the list. I know it's not terribly accurate, and it has some skewing problems (stuff in the default install will be favored), but I suspect it would greatly bias the results in terms of software that's useful to many people (using the "recently used" tallies might especially help here).

I don't have many other good ideas on the searching problem; I've seen vague suggestions of counting reverse dependencies, but this doesn't seem like a good metric of useful software to me; trying to extract more information from the short and sometimes inaccurate package descriptions seems like a really hard problem to me. (Google can do cool statistical analysis partly because it has so much data to work with AIUI; even a single Web page typically has orders of magnitude more information than a package description)

Another issue is the problem of finding related software once you've installed one piece. We have "suggests" and "enhances", but they aren't exactly displayed in a prominent or easy-to-understand way by any frontend. I think it might be good to display a clickable list of suggested packages somewhere visible (e.g., at the top of the description box) when the user has selected a package for installation. We could even display a little bar, like on (eg) Amazon: "Debian suggests installing these packages..." ;-)

It might also be interesting (if this is considered acceptable; since the data is anonymous I don't see a problem) to analyze the popcon data to find out what packages are being installed in tandem. I don't know if we have enough information to do it, but it could turn out to be really useful to be able to click on a package and see other packages that are popular with users who installed that package. Going back to online stores again, those "other people who bought 'Types and Programming Languages' also bought 'Introduction to Category Theory'" links are amazingly useful, and something similar would probably be a big help in Debian.

How to Breathe, Functionally (part 2)

In a previous post, I pointed out some ways that you can acheive the "compute this, now save it and compute this, ..." style of coding in a functional language such as Haskell without going mad. This is a continuation of that post.

There's one other purely syntactic feature of Haskell that I rather like for readability: Haskell lets you define variables after you use them.

GoToTheNextTrickyBit someVal
   where someVal = DoThisAndThisAndThat...

Although it isn't always appropriate to do this, there are times when I find it extremely intuitive to define my variables after they're used. For instance, to take a "practical" example, compare

mailTo customer =
   let customerName = getCustomerName customer
       customerAddr = getCustomerAddr customer
   in
       makeMailingTo customerName customerAddr

with

mailTo customer = makeMailingTo customerName customerAddr
   where
      customerName = getCustomerName customer
      customerAddr = getCustomerAddr customer

I don't know about anyone else, but when I read the first definition I mentally go "huh. Why are we extracting the customer name?", then skip down to where the name gets used, and finally re-parse the name in that context. The "where" form lets me write the code in the same order that I would read it, giving the code a natural flow. It's very important, of course, to keep the number of auxiliary definitions small; otherwise you leave too many "open questions" for the reader of your code to carry around as she works her way down to the definitions.

But it isn't all roses at the low syntactic level. IMO, the biggest practical problem with writing Haskell code is the syntactically significant indentation.

Python, of course, also has significant whitespace -- but it works mainly because it's extremely simple. Any line ending in a ":" introduces a new block, and statements within a block have the same amount of indentation. In contrast, Haskell's rule...well, just read that link above. You can write code that has the same meaning under any indentation, but it's not idiomatic to do so -- and worse, since the statement terminators are not obligatory, a missed terminator might work fine, might change the meaning of your code, or might fail to compile in an obscure way.

A side effect of the complicated indentation rules in Haskell is that the Emacs Haskell mode doesn't work very well at indenting my code. It usually chooses an indentation that I don't want, and often one that doesn't even compile. I'm sure Emacs could do better, but I would be willing to live with a little more punctuation if it meant that the code was easier for programming tools to process correctly. Functional coding advocates assert, correctly IMO, that side-effects are problematic because even smart programmers will write more bugs in their presence. I think the same logic should apply to syntax: making the language harder for humans and machines to parse has negative overall effects, even if it is possible with sufficient effort to parse it correctly.

I think that conceptually, Haskell is a great language -- the problems I see with it are issues of syntax, tool support, libraries and speed (laziness has a high price). Oh, and also the fact that, as with most programming languages that are pleasant to use, you can't get a real paying job writing in it unless you have a PhD. :-(

How to Breathe, Functionally (part 1)

David Welton writes that the problem with functional languages is that you don't get to mentally "catch your breath" between computations; piling function invocations on function invocations makes it hard to understand what's actually being calculated. He has an amusing summary of how functional code can become unreadable:

Do this with the result of this with what this produces when given this input that is derived from this function which takes the result of this other thing after calculating the result of something that is derived from the output of a function that ...

Indeed, I've seen functional code like this (much of it written by myself). Here's a particular gem:

f splitPoints = map fst $ tail $ scanl (flip ($) . snd) ([], toSplit) $
          map splitAt splitPoints

I wrote this code a year ago, and I don't know what it does! It's hard to read and hard to understand. So, however, is imperative code that says

Add 5 to A, now if B is not zero we jump to where we subtract one from B and if C is less than 3 we add 4 to C otherwise we divide C by 2 and then add it to A, then we ...

The reason both pieces of pseudocode are hard to understand is that they don't use the abstraction features built into the language. Good imperative code uses meaningful variable names, uses the structured-programming primitives the language provides, and abstracts complex operations into functions.

My rule of thumb (which is, heh, not always religiously followed) is that a piece of program structure that does more than about 3-6 things is probably too complicated; it seems like that's about as much local context as I can hold in my head comfortably (and the number is more 3 than 6 these days, if you get my drift). As David puts it,

If you'll bear with a poor comparison, sometimes programming is like a song. You do the tricky section, then you go back to an easier section and catch your breath for a bit.

The problem seems to be that people trained on imperative languages, when they're tossed into the ocean of functional coding, can't find the guideposts and landmarks that would help them impose some structure on the water. Should I put this complicated expression inside a "let", or break it out into another function? When do I use a higher-order function to unify two dissimilar pieces of code? Should I write this code in terms of higher-order library functions or directly? It's easy to wander off down a chain of nested invocations of anonymous higher-order functions, or composed partially applied library functions, just "because you can" and end up forgetting what you were trying to do in the first place.

Luckily, some of the basic rules of keeping your code readable remain more or less the same: assign intermediate results to variables with meaningful names, and abstract complicated operations into separate functions. To tackle David's example:

You get sections with DoThisAndThisAndThatWithTheResultsOfSelectThisFromTableXYZ that you have to slow down to think through.

...

GoToTheNextTrickyBit where ManyThingsHappen.

All in all, you wind up with a sort of ebb and flow that works out pretty well if you're in tune with the code being written, going from intense calculations, to a line or two where you do no more than store things in a variable, take a breather, and prepare for the next important stanza.

The way to "take a breath" in a functional language is generally to assign an intermediate result to a variable, or to extract a complicated operation into a function and give it a meaningful name. One Haskell approximation to David's example is

do DoThis
   AndThis 
   results <- SelectThisFromTableXYZ
   someVal <- AndThat results
   GoToTheNextTrickyBit someval

If you're writing pure functional code, you might write

let
   results = SelectThisFromTableXYZ
   someVal = AndThat results
in
   GoToTheNextTrickyBit someVal

Of course, this will become unreadable if you write a huge complicated nested "let" expression in place of "AndThat". For a truly dreadful example of this kind of coding, check out the Gnucash Scheme code that generates the "Cash Flow" report (warning: read this in short bursts, or you WILL go blind). This is exactly analogous to writing too many nested loops and conditionals in imperative code, and the fix is the same: find logically distinct bits of code and extract them into separate functions.

I have a few more thoughts, but this post is getting too long, so I'll inflict them on the world in another post. The rule that short things are easier to understand applies to blog posts as well as code. :-)

Via C7: it goes fast AND slow!

The low-powered system I assembled recently is based on a Via Epia board and a C7 CPU. I spent quite a bit of time when I was setting it up trying to get the C7's speed stepping to work. I loaded a number of cpufreq modules without luck; crawling around on the Web only led me to discussions about how the C7 couldn't be speed-stepped in Linux and maybe the centrino cpufreq driver would work.

Well, yesterday I randomly searched again -- and poof! There was a discussion from early January about how maybe the ACPI cpufreq driver would work. I loaded it, set up powernowd, and was rewarded with

daniel@jeeves:~$ cat /proc/cpuinfo 
processor       : 0
vendor_id       : CentaurHauls
cpu family      : 6
model           : 10
model name      : VIA Esther processor 1200MHz
stepping        : 9
cpu MHz         : 400.000

Sweet. Now all I need is for the Debian kernel folks to package a version with a vt1211 lmsensors driver so I can shut down or slow the fans, and I'll be all set. (I could probably hack it in myself, but according to what I've read on the Web, the next kernel release will have a working driver)

[UPDATE: I disabled cpufreq after only a day or so, because Linux got horribly crashy with CPU scaling turned on. Hopefully the kernel will be fixed sometime in the future :-/ ]

Cuckoo!

The most unique Christmas present I got this year was the cuckoo clock from my father's parents. There's something very cool about mechanical clocks that work by means of weights and gears instead of electronics: when the cuckoo goes off, you can actually watch energy being transferred from the weight to the cuckoo device, as the weight that powers it drops precipitously!

Unfortunately, the clock seems to lose about half an hour every twelve hours. Does the Lazyweb know any techniques for getting mechanical cuckoo clocks to keep better time? I don't mind resetting it once a week or so, but twice a day is a little excessive.

Mini-Review: mythtv for non-TV viewers

Several months ago, I set up a low-powered box to act as a fileserver and media center. I don't get cable TV and I have no interest in doing so, so I don't need a PVR. However, I do have a rather large music collection (and new speakers to play it on); I want to be able to put Internet radio on in the background (since we can't get NPR over the air), and a quick-and-dirty slideshow program for my digital camera photos could be nice too. So, I decided to try out the free media center, mythtv. I've summarized the notes I took here for the benefit of other people who don't watch TV.

Setup was fairly straightforward -- I installed the packages from debian-multimedia.org and configured them. I had a bit of trouble getting myth to actually start, but looking over the documentation apparently resolved it (my notes contain no details).

Mythtv has some very well-thought-out aspects. By all accounts, it makes an excellent PVR; since I don't watch TV, this is the last time I'll mention PVRs in this post. :) The interface looks exactly the way you want a TV-based system to look, with bright colors, big fonts and big interface elements. There's even a cute OpenGL mode with fancy screen transitions (I left them disabled because my video driver is unstable even without 3d effects running -- thank Via).

However, it became painfully obvious after a bit of use that MythTV is really meant as a TV program, with everything else being an afterthought. Here are just a few of the problems I noted:

• MythTV can't play internet radio; the functionality simply doesn't exist.
  
• You can do only one thing at once. Perhaps this is fine for TV, but given that it is not uncommon that I want to, say, play music while reading the news, or while ripping another CD, this is a gaping hole in functionality for me.
  
• MythTV can display news (RSS) feeds. However, the news feeds aren't in alphabetical order in either the viewer or the setup screen (where you pick what to see). This makes it rather painful to pick the feeds you want out of the dozens of presets.
  
• The built-in browser crashes with SIGABRT on startup, making the news feeds useless anyway.
  
• The built-in support for ripping flac files is confusing to activate (you have to pretend you're making an Ogg and choose "Perfect" quality!) and doesn't give you control over the process (for instance, replaygain can't be added to the ripped files).
  

That's just a small sampling of the problems I ran into. I ended up just setting up a normal desktop system with a wireless keyboard/mouse, and I'd recommend the same thing for anyone else who doesn't need a PVR. The non-TV features in Myth have a strong smell of being extras for people who occasionally switch away from the TV, rather than a well-integrated media system in their own right.

The US isn't the only country with crazy politicians

[UPDATE 2007-10-01 22:43: It appears that Erich wrote an expanded and better-translated post while I was at work]

Erich Schubert writes that

Nach aktuellen Plänen der CSU wäre der Vertrieb von Debian GNU/Linux illegal.

Auch zahlreiche andere Distributionen währen wohl davon betroffen: Alle, die sogenannte "Killerspiele" vertreiben...

For non-German speakers, I provide the following approximate translation:

CSU wants to make Debian illegal

The current plans of the CSU [Christian Social Union] will make the distribution of Debian GNU/Linux illegal.

In addition, many other distributions will be affected: specifically, those that distribute so-called "killer games"...

For instance, Overkill (Debian also includes, e.g., Tremulous and OpenArena, which are based on the Quake3 engine and have 3d graphics...)

(hopefully I got the gist of Erich's post without mangling it too badly!)

What I don't know is how serious this is. The CDU (Christian Democratic Union) is IIRC the current ruling party in Germany; however, according to Wikipedia, the CSU is some sort of local affiliate in Bavaria. Does it have the clout to get this proposal made into law?