The basic setup was quite simple: I had no problems setting up the N1 to use my home Wi-Fi network, and GrooVe IP was able to use my existing Google Voice account to make outgoing calls without a problem. Freedom from telcos at last!, I thought.

Well, not quite. Almost immediately I encountered a problem: there is apparently a bug in the drivers or firmware for the wireless adapter that causes it to shut down and disconnect from the wireless router when the phone’s screen shuts off, regardless of the phone’s setting for the Wi-Fi Sleep Policy. This means that I can’t receive incoming calls unless the screen is on.

Ok, no big deal, I thought. This phone is running a relatively old build of Android (2.2); it’s the official developer phone; so I ought to be able to find an update or workaround that fixes this issue. I’ve been winding my way down a troubleshooting rabbit hole ever since.

As a Debian user, I’m not used to this kind of experience. Here are my thoughts about the Android platform and ecosystem, after 24 hours of trying to fix this issue:

The Android “community” is fragmented. There is (or was) a lot of discussion fairly recently about fragmentation in Android software. A related and, I think, bigger problem is the fragmentation in the user community. There does not seem to be a central place to look for information about software problems, or to look for help. Instead, there’s a mish-mash of ad-supported Web-based forum sites, generally with a very low signal-to-noise ratio. Reports from frustrated users are many; but follow-ups from technically capable people are few and far between. If a solution is posted at all, it’s difficult to tell whether it worked, and whether it would be worth trying yourself. Sometimes, someone posts link to yet another third-party site with instructions to load a .zip file they’ve posted onto your device — not exactly confidence inspiring, to say the least.

Compared to the Debian world, this is like being back on Windows: authoritative information is almost impossible to find in the user community, and even when information looks authoritative, the fact that it’s posted on some random Web site instead of an official wiki or mailing list causes me to think twice.

There are a few things that Google and other stewards of the Android ecosystem could do to fix this community fragmentation:

1. Create a single centralized mailing list for Android users. It should be a mailing list, or series of mailing lists, along the lines of those found at lists.debian.org. Having a mailing list, as opposed to a Web-based forum, allows proper threading and quoting (and doesn’t require clicking through artificial pagination) which makes it easier both to post and to find high-quality information. Technically-adept users will be more likely to provide good help to others if there is a single place to do so, and that place is accessible via e-mail and/or NNTP. (Of course, a Web-based interface is still useful, especially for searching the list archives, but shouldn’t be the primary means of access.)
2. Create a centralized wiki containing articles about common problems, HOWTOs for addressing them, and attachments of files that users may need.
3. Use these mailing lists and wiki for communication with the community. Of course, a mailing list and wiki won’t solve the signal-to-noise ratio unless the people who actually maintain Android have some involvement in it. Representatives from Google and from the handset manufacturers should regularly answer questions on the list, and they should enforce a certain amount of netiquette: users should be encouraged to search the archives first, ask detailed questions, refrain from flaming, etc.

This last point brings up another important issue: there seems to be a huge disconnect between “users” and “developers” in the Android world. Contact between users needing technical assistance and the people who can actually provide that assistance is few and far between; and often, the message amounts to, “We’re looking into it. Sit tight until the next release!”

To their credit, Google appears to want some of this interaction to happen. They have “community managers” who occasionally respond to user issues in the Google-hosted forums. Unfortunately, questions from users far outweigh messages from community managers, and few users are in any position to help one another. The contrast with the Debian users list is stark: in the Debian world, there is no artificial boundary between (enabled, enlightened) developers and (disabled, helpless) users; there are just users with varying degrees of knowledge, who contribute what they can when they can.

Google also has a publicly-accessible issue tracker, including a feature that allows additional users to upvote an issue. This is good for what it’s worth, but it doesn’t seem to have much influence on the direction of Android development. One gets the sense that Google uses a different, internal issue tracker for Android bugs and features. Android releases, for example, do not seem to list the specific issues that they close or address. So though some of the information I’ve seen indicates that my particular problem has been solved in Android 2.3.4, I can’t find any official confirmation of this.

Again, my point of comparison here is the Debian bug tracking system, where one sees regular contact between maintainers and users, and the relationship between issues and releases is made public and obvious.

I would much prefer to see a single issue-tracking system for Android that’s used by both users and developers, and that makes it obvious to anyone who cares to look exactly which issues are being worked on, when there’s a patch available, and when a fix will be available through an update.

Finally, there is the issue of the opacity of the update process. Android updates are rolled out to users a few at a time, without any explanation of why one group gets them before another. It’s possible to find instructions for installing these updates manually, but again, the information comes from third parties, not the Android developers, and it feels far from officially sanctioned. I have no idea why my phone thinks it’s up-to-date at version 2.2 (remember, it’s an unlocked, developer phone) when others have gotten 2.2.x or 2.3.x updates over the air.

I understand that for many users, updates come via their cell carrier, which has its own set of headaches. But for those users who are running an unmodified, unmediated Android on officially-supported hardware, I can’t understand why there is no equivalent of apt-get dist-upgrade the moment a new stable release is available.

All of this leaves me feeling a little helpless in the face of what I thought was going to be a minor problem. I want to like Android, and I hope the project succeeds in building a stronger community and a stronger relationship between users and developers. From where I sit, though, it has a long way to go.

I’m still undecided about buying a device, but I wanted to catalog some of the programs and hacks that I’ve (re-)discovered as I looked into the issue a second time.

1. Calibre looks like a great piece of desktop software for managing e-books. It knows how to talk to multiple e-readers, and can inter-convert between popular formats, allowing (e.g.) Kindle readers to read ePub books by first converting them to Mobipocket format.
2. Savory extracts some of the code used by Calibre to allow Kindle users to download and convert ePub books directly on their reader, without having to go through desktop software.
3. epubjs is a nice ePub reader written entirely in JavaScript, so if the Kindle and other non-ePub devices ever acquire a JS-enabled browser, or even just a local JS interpreter/engine, this will be another option for reading ePubs there. There is also a post at Ajaxian about some other JS-based ePub readers here.

The main issue I have, though, is in dealing with PDFs. It doesn’t look like I’m going to be able to justify the expense of a large-format e-ink reader in the near future (the Kindle DX is the cheapest, I think, at $489!). I’m still looking for a comprehensive set of tools for manipulating PDFs so that I could read them easily on a smaller screen. Specifically, I need tools for:

• extracting text from text-based PDFs, or at least being able to reflow them and trim their margins
• converting scanned images of book pages in PDFs into text via OCR software

I haven’t found a complete solution for either task, but I have come across various programs that do some of these things:

1. PDFMunge is a Python program that can help with the task of trimming margins and reflowing text in text-based PDFs.
2. pdftk is a comprehensive Java library and command line tool for manipulating PDF files.
3. Google Docs now has the option to use OCR to convert PDFs to text. It doesn’t work perfectly, especially for more technical material, but it’s easy to use. I believe it is based on Ocropus and/or tesseract-ocr, both of which are Free software and can be built and run locally (if you can figure out how to do so…the dependencies are pretty significant).
4. Briss looks like a nice way to crop scanned PDFs using a GUI interface.

Sadly, none of the e-readers available today seem to have the full set of features I would want:

1. e-Ink Display: I can’t read for long periods on an LCD, so that rules out something like a smartphone, netbook, or tablet PC.
2. Expandable storage: one of the big downsides of the current Kindle is that its storage is limited to the 1.4GB available to you when the device ships. I especially can’t understand why Amazon removed the SD card slot that the Kindle 1 had.
3. Physical keyboard and note-taking abilities: this tells in favor of the Kindle, but against the Barnes and Noble Nook, as well as against a lot of the other e-readers I have seen. I want tactile feedback when I’m typing; I can’t stand typing on touchscreens. Some readers appear to offer no textual input at all, which isn’t much use to me; I need to mark text as I read.
4. Wi-Fi: another black mark against the Kindle 2. It’s nice that Amazon, Barnes and Noble, and others want to offer me free 3G service as a way to ensure that I can impulse-buy books from anywhere. But without Wi-Fi as a fallback, I am scared off by clauses like this one in Amazon’s License Agreement and Terms of Use:
   

   Amazon reserves the right to discontinue wireless connectivity at any time or to otherwise change the terms for wireless connectivity at any time, including, but not limited to (a) limiting the number and size of data files that may be transferred using wireless connectivity and (b) changing the amount and terms applicable for wireless connectivity charges.

   Having a fully Internet-ready e-reader would make it much more useful to me, but it’s clear that Amazon wants nothing of the kind. They want to control the kind of information I can get, which they couldn’t do if they included a network interface that — horrors! — didn’t route all traffic through their servers. I don’t really want Amazon, or any other company, knowing everything I read online. And I don’t trust them not to “discontinue…or otherwise change the terms” of my Internet access through their blessed portal.

5. Support for open formats, including PDF and ePub: most of the non-Kindle readers win here again, though it’s not clear how much of the PDF standard any of them supports. A lot of the PDFs I read are scanned images from actual books, and I would like to have simple tools for cropping pages, or splitting one page into two, to better fit a reader’s screen.
6. Extensible platform: I’d like to be able to write my own programs, or download others’ from the Internet, if the built-in software doesn’t cut it — preferably without having to root the device. For doing academic reading, programs like a multi-lingual dictionary or a citation database would be helpful. Amazon has a Kindle Development Kit in the works, which is nice, except that it’s Java-based; I would much prefer, say, a combination of Python and C. (I’m not sure yet if JVM implementations of Python, Ruby, Scheme, etc. will work on the Kindle…but that would be great!) The Nook has nothing so far, but the fact that it’s running Android points to hackability in the future, with or without Barnes and Noble’s support. Other readers have more explicitly open software platforms, but without a large number of users, they probably won’t see much development.
7. Low price: the $259 that Amazon and Barnes and Noble are currently both charging is about as high as I would be willing to go. I simply can’t afford to sink $300 or more into a highly specialized device. This unfortunately rules out a lot of the lesser-known readers for me, because they don’t have the agreements with publishers that would allow them to subsidize their hardware with e-book sales.

So what am I to do? I’d love to be proven wrong, and find a reader out there that has all these features and more. But until then, I think I’m stuck with paper.

My own experience tells me that part of the reason papers are struggling, either in their print or online versions, is that they are locked into a terrible revenue model based on advertising. Ads take up greater and greater space in the physical paper, and seek more and more attention on a digital page. They are typically not relevant to my interests, and range from nuisances (an extra page turn to get past the spread for expensive perfume) to actual intrusions (graphics that cover the news content in my browser until I explicitly close or disable them). The papers are apparently in a tight spot: fewer people are subscribing, so they must rely on more advertising revenue to operate; but since readership is declining, individual ads are worth less. Papers must therefore be more aggressive about the number, size, and flash of ads they print or display; this furthers annoys, and drives away, readers. Of course, they can’t be as good at the advertising game as online media can — they can’t personalize print ads, they can’t print anything animated or with sound, and they probably can’t afford to keep separate advertising departments and print different editions for every city they sell papers in — so their advertisements will never be as targeted or effective. Their value will decrease accordingly, leading to further reduced income.

If left unchecked, this cycle will result in papers with more and more advertising, and fewer and fewer readers, until they can no longer afford to operate. That’s the end of a paper, which, for many reasons, can be a damn shame.

The problem here comes from the assumption that the papers must maintain a certain revenue level to remain profitable. They turn to advertisers to fill in the gap in revenue left by subscribers, which slowly drives more subscribers away. One way to save the newspapers, therefore, is to do away with that assumption. Instead of looking to maintain revenues, they should cut costs.

They shouldn’t cut costs in the ways they have been, however: closing bureaus, laying off writers and editors, putting more content online with less editorial review. These are cuts to the things which make papers valuable. The value of a newspaper is in the professional training and the contacts of its reporters and editors. Despite what proponents of “New Media” might say, trained journalists still have an edge in their access to world events and the people involved, their ability to report those events in a straightforward way, and their knowledge of how to interpret and draw connections between those events and others.

So where should the papers cut? There is one major area that, apparently, they haven’t considered: getting out of the printing business. This doesn’t mean that papers shouldn’t be printed; it just means that someone else should print them. Specifically, newspapers (especially national newspapers) should make their content available, free of ads, to anyone who wants to print and sell it — for a fee. This fee might be subscription based, or it might be on a per-paper basis, but it would allow anyone who thinks they can efficiently print (and possibly deliver) a newspaper to do so. The printers would then turn around and sell the papers to individual customers: newsstands, coffee shops, delivery services, even individuals.

Ideally, newspapers should allow printers broad freedom to modify the content of the newspaper, so that printers can seek to make a profit by printing different formats, placing local ads, and so on. The only way printers would be likely to sign up for this arrangement, after all, would be if they thought they could somehow do a better job printing and distributing papers than the newspaper companies themselves could, so they would need to have an arrangement that allowed them to apply their own expertise. The newspapers might dictate some aspects of the printing — for example, they might not let their title be printed if certain severe content modifications were made — but the fewer restrictions on the printers, the better. If a printer wants to use a different font because it uses less ink or looks better, or a different paper quality because it’s cheaper in his area, or employ any other strategy for making money selling copies of the news, he should be able to do so.

I have no numbers to back it up, but I think this strategy has prima facie plausibility because it puts both aspects of the newspaper business in the hands of people who know it best. Newspaper companies would be paid for producing the content which they are experts at producing: writing about the news, in all its many forms. Printers would likewise be paid for the fruits of their expertise: physical copies of that content, produced cheaply and sold in a relatively small local area. Physical newspapers, like many information resources, suffer from a “last mile” problem that small, local businesses are better equipped to solve than a central printing authority. By shifting the cost of producing physical papers to the people best equipped to solve that problem, newspaper companies would be free to focus on the news, instead of on how to sell and print advertising. With their revenue source back in alignment with the actual value of their product, they might just be efficient enough to stick around.

In practice, it is difficult for individual users to exercise this freedom. This is because:

• users may have no idea who to ask to fix a problem or add a new feature; given the nature of most Free software projects, finding the right programmer might be difficult
• even if a user knows who to ask, the programmer or development community might be unwilling to fix the problem because they don’t see it as worth their time
• if the programmer or development community offers to fix the problem or add the feature for a fee, the user may not be able to afford it herself, and may have no idea how to find other people who are also willing to pay for the feature or bugfix

So, although users of Free software have an important freedom in theory, they are still often unable to reap some of the benefits that Free software promises, such as avoiding dependence on a particular developer, or pooling their resources to support development.

This may be one reason why users of popular desktop systems like Ubuntu are often perfectly happy to install proprietary software: even if they know the advantages of using a Free program, without a way to exercise their freedoms, they might as well use a proprietary program that works better or has commercial support.

Meanwhile, because programmers don’t often have a way of getting paid by users to write or adapt Free code, there is a common perception that the only way to earn a living writing software for the masses is to make it proprietary, or to be hired by a large company that will pay you to write software they need. It is difficult for programmers to earn money just by contributing to the Free software projects they are most interested in; sometimes, they ask for donations, but I doubt that many of them expect to earn much from them. We continue to see proprietary programs offered for Free systems as a result. Ubuntu’s Software Center is the latest incarnation of a solution to this perceived problem: Canonical plans, eventually, to sell proprietary applications alongside Free programs as a way of earning money to support their organization. They also think it will attract more programmers to GNU/Linux generally. I hope to be corrected, but I don’t see any part of their proposal that will alleviate the bad choice developers currently face between writing Free code and earning money. They aren’t proposing a payment system for Free software development.

This problem seems eminently solvable. I think we can and should fix the problem of allowing users to exercise their freedoms, and helping developers get paid to write Free software, at the same time. I have a proposal for how to do so: let users pledge contributions toward bugfixes and feature implementations. The basic idea is that the issue trackers used by many Free software projects could easily be extended to allow users to pledge contributions toward particular programming projects; when those projects are completed, each user would pay their pledge to the programmer or team who submitted the patch.

Imagine the following scenario, for example. Janet notices a bug in the way her wireless card works in Ubuntu. She searches on Launchpad and discovers that others have the same problem; there is an open bug report for her issue. Though she has no programming knowledge herself, she is willing to pay $50 toward getting this bug fixed, because it is a major inconvenience and she has no workaround. She clicks a “Pledge now” button in the Web interface and enters this amount. Meanwhile, Katherine, a savvy kernel hacker, notices that a lot of people are having the same problem, and spends some time improving the wireless driver for Janet’s card. Her patch fixes the issue, and the bug is closed. Janet (and anyone else who pledged) are notified that a patch is available, and that they must forward their payments to Katherine. Katherine earns a modest sum for her work, allowing her to spend more time fixing wireless card issues than she otherwise would be able to.

Obviously, there are a lot of logistical problems to be solved here, such as:

• What happens when multiple patches are required to fix a bug, or multiple programmers work on the fix? How should the payment be distributed among patch writers, package maintainers, and others who contribute to improving the quality of the software?
• How do users get a hold of the new version? Do they wait for the patch to move through the normal packaging process before sending their payments? (What if a fix is already available upstream at the time a pledge is made?)
• Who collects the payments, and who ensures that they are made? (Would an arbitration system be needed?)
• What happens to pledges if a bug is marked invalid, or made a duplicate of another?
• What if the patch fixes the problem for some users, but not others?

I think these problems can be addressed by a little engineering and/or “social” innovations. They make a case for experimenting with different informal payment mechanisms to see what works best; they don’t constitute an argument against pledge-based payment systems in general.

The real hurdles presented by this approach are more abstract. They crop up whenever money is introduced where it was previously absent. Payment systems could significantly re-structure Free software communities, and possibly even threaten their existence. Although a payment system could help users exercise their freedom and help programmers to earn a living writing Free code, it might do so at the expense of the volunteerism and collaborative spirit that have made the Free software movement so successful. Programmers who currently work on a project as much as they are able, for their own satisfaction, might become resentful if others start earning money to do the same kind of work. Project leaders might find themselves having to manage payments, in addition to the difficult technical and social problems they already work to solve. And while some users would be able to exercise their freedom more effectively than they can now, the extent to which they can do so would essentially be a matter of their ability to pay, a difficult moral problem in its own right.

For these reasons, we should proceed with caution. I think organizations like Canonical can and should help design these payment systems; after all, they have an interest in making money by developing and distributing Free software. But it should be up to the teams behind individual projects to participate in any payment system, and to decide how they will distribute payments among their members.

Thinking about this problem again, I realized that there is a much better answer. Today, my favorite Unix tool is: the pipe.

Gnus treats mail like news, meaning you “subscribe” to various mailboxes, and once read, email is simply hidden unless you explicitly ask to see it. The idea is that the activities of reading mail and news are very similar — you want to see what’s new without having the old stuff around, sort or filter incoming messages, write responses, then get on with your day — so the protocol used to access them (NNTP vs. POP, for example) shouldn’t make a difference in how you do them. Gnus therefore unifies these activities in a common interface, and deals with different protocols and storage methods with different “virtual servers,” i.e., back-ends which find mail or news wherever they may be. To use Gnus, you must learn how to tell it the details of how to access your messages via one or more back-ends.

So, I went to try to find the right back-end for an IMAP mail account. This raises a further complication: unlike most of the mail back-ends, which assume mail is stored locally, IMAP stores messages on the server. This makes the IMAP back-end more like a news server back-end than a mail spool back-end.

After reading the manual section on IMAP, I had tried a variety of settings that didn’t quite seem to work: although I could connect to the server, and read my mail from the Server Buffer, I couldn’t seem to “subscribe” to any “groups”, or split my IMAP mail into these groups. Fortunately, this excellent article provided a complete, working IMAP example. I’m hoping to be a Gnus addict by the end of the summer.

UPDATE: I’ve been successfully using Gnus for a few months now. Here are some relevant bits from my .gnus file to get any new users started:

;; GMANE is about the only free news server I've seen.
;; I set it to my primary server so I can read a few Free software mailing lists.
(setq gnus-select-method
     '(nntp "news.gmane.org"))
;; Mostly, though, I just want to read my mail.
;; This setup uses a standard SSL-based connection to read the mail for the accounts I have through
;; UC Berkeley:
(setq gnus-secondary-select-methods
      '((nnimap "calmail" ; primary email
		(nnimap-address "calmail.berkeley.edu")
		(nnimap-server-port 993)
		(nnimap-authenticator login)
		(nnimap-expunge-on-close 'never)
		(nnimap-stream ssl))
	(nnimap "ocf" ; secondary account
		(nnimap-address "mail.ocf.berkeley.edu")
		(nnimap-server-port 993)
		(nnimap-authenticator login)
		(nnimap-expunge-on-close 'never)
		(nnimap-stream ssl))))

Groaning about CSS bugs in Internet Explorer is a favorite topic on forums across the Web, but most people seem resigned to their existence and (begrudgingly) implement their own hacks and fixes, including reverting to table-based layouts. I have never seen or heard a cogent discussion of the reasons why Internet Explorer is the way that it is, or what can be done about it.

This problem is not unique to web browsers and web design. It is an inherent tension present whenever proprietary software is used to read and write data that’s supposed to adhere to an open standard. Open standards tends to evolve; proprietary software does not tend to evolve to meet them. This is partly due, I assume, to a resource allocation problem: it’s possible that even huge proprietary software companies like Microsoft can lack the resources (or willpower) to stay abreast of changes in standards over which they have little control. A bigger contributor is psychology: a culture of software ownership does not tend to foster the values of deferring to standards for the good of a group, sharing code and patches, or contributing in any other way to a community which does not recognize software property “rights.”

Ignoring standards is evil?

Contrary to popular belief, I tend not to think that Microsoft and other proprietary software outfits ignore or imperfectly implement data standards because they’re “just evil” or anything of the sort. Believing that might bring you some emotional comfort, but it’s an uncritical and shallow explanation that does little to remedy the problem. Instead, I think this behavior is rooted in a variety of cultural attitudes historical events, and conventional wisdom and platitudes. In the rest of this post, I want to focus on one such bit of conventional wisdom, which I think is a major contributor to the way that both proprietary software companies view open data standards, and users’ acceptance of those views: the belief that software is “just a tool.”

This is a belief that Microsoft in particular has promoted for some time. Microsoft makes most of its money from fairly boring, functional products: Windows and Office. The software in these products is basically designed to facilitate users’ accomplishing a series of well-defined tasks: writing a document, calculating a budget, sending an email, starting up and shutting down. Microsoft has a concerted interest in making sure that their users don’t think of software as anything other than a means to these pre-defined ends, because if users did think that, they might start to want things like well-designed graphical interfaces, open APIs and data formats that would allow them to extend the software for other purposes, and a general hackability that would make defining and accomplishing one’s own tasks easier.

Tools

There’s no doubt that computer programs — at least some computer programs — are tools for accomplishing tasks. On the most basic level, programs are a series of instructions that turn inputs into outputs. Inputs come from users, and outputs are meaningful to users. We design and run programs with the express intention of getting from one end of this chain to the other, but what happens in between may or may not matter to us. When you think of software as just a tool, you’re taking the latter perspective: you don’t care how it does the job, as long as the job gets done.

The problem is that, in reality, we all care at least a little bit about how a program does its job. Programs don’t generally run in an isolated environment, and can have side effects that make the system less usable as a whole. If a word processor erased all the documents you had previously written in order to create space in which to save the one you were currently writing, you wouldn’t likely be very happy. The program is still acting as a tool (it helps you write and save documents), but it has a nasty side effect (erasing other files) which prevents you from doing other things, like revising work you’ve already done. This might seem like an extreme example, but it isn’t so far off from what some of the other “applications” out there will do to a system in the name of just getting a job done. For example, resource-intensive programs, like anti-virus software or Windows Vista’s content protection pipeline, can use up so much memory, processor time, and I/O that the system as a whole slows to a crawl, even though its processing power and storage are thousands or millions of times faster and larger than systems available 20 years ago. Yes, these programs are tools for accomplishing specific ends, which they attain (with varying degrees of success). But they impose such significant constraints on what other programs, and thus users, can do that they fail to be useful in an more global sense.

In the tool analogy, these programs are like a hammer that automatically pulls out a nail for every one you put in. You could argue that the defect in the hammer isn’t really the hammer designer’s responsibility, since the hammer is just a tool to aid in the task of pounding nails into a board, and it still does that very well. This would plainly be ridiculous. For a tool to be useful, it can’t just “blindly” aid its user in completing one task. Rather, part of what it means for something to be a tool is that it helps its user accomplish a variety of goals, often in tandem or as means to some greater end — say, building a house. (This is one reason we often speak of tool kits.) It doesn’t make sense to refer to software as “just a tool” if all you mean by that is that it makes one kind of task easier, perhaps at the expense of doing other kinds of work.

The UNIX world is much more sensitive to this broader concept of tool-as-part-of-a-kit. Many UNIX programs are designed to do exactly one thing, and to do it very well, without trying to be all things to all people. A huge part of what makes these programs so useful, though, is not their specialization to one task, but their ability to be combined with other programs. By combining the tools of the UNIX environment, users can easily complete new tasks that the system designers could never have dreamed of. It is the UNIX philosophy of providing both a set of useful, efficient, task-specific tools and an environment that facilitates combining those tools that makes UNIX the system of choice for a wide variety of computing tasks.

This brings me to another point where the “just a tool” mindset breaks down. Software is a very unique sort of tool: one that can produce other tools whose power are limited only by the user’s imagination. People who think software is only good for doing one pre-defined task at a time have had the wool pulled over their eyes. What makes computers so great for doing so many things is that they are programmable. You can balance your checkbook with a pocket calculator, but if you use a spreadsheet instead, you’ll be able to write programs that can answer new questions when you think of them (e.g., “how much did I spend on groceries between December and February?”) without too much more effort. If you write your notes using a text editor instead of a pen and paper, you will be able to search them, reorganize them, and reformat and print them in ways that pen and paper will never allow. Information-processing tasks in general are easier to do with a computer because, once you have the input data in a well-defined format, writing programs to manipulate that data in new and interesting ways is much simpler than performing those manipulations by hand. For this reason, software at its best is less like a tool and more like a whole machine shop: not only does it allow you to do certain kinds of things to start with; it enables you to build your own custom tools when the need arises.

In the end, it may be the case that some people prefer to operate their computers like drones, working through one pre-defined task after another, never stopping to think if they could do them better or more easily by thinking of their software in a new way. For such people, software truly is just a tool, and a rather blunt one at that. But the rest of us need not suffer this fate.

Mnemonic is a Python library for analyzing experimental data about human memory. It contains functions for performing several specialized analyses and a library which allows users to readily build their own analyses.

If you want to check it out, and you have the Bazaar version control system, you can get it this way:


$ bzr branch lp:mnemonic


Mnemonic is in the earliest stages of development, but I would welcome contributions from anyone who finds it useful. My hope is that it will one day replace the MATLAB codebase at the Computational Memory Lab, where I work as a Research Coordinator. Please let me know if you use it!

I am currently running gNewSense DeltaH, which is based on Ubuntu’s Hardy Heron release. I have found the out-of-the-box hardware support to be surprisingly good: I didn’t need to install any special packages to get the screen resolution right, to get the mouse working, or to get the keyboard working before boot (all of which were issues with Ubuntu when I tried it a year ago). The gNewSense installer worked perfectly fine, and gave me a happily-booting system without any additional configuration on my part.

The rest of this post will be dedicated to the things I had to do to get the system working to my satisfaction after the initial installation. Even though the hardware support is impressive, especially for a distribution which makes a point of removing any and all binary-only “blobs” from the kernel, there are a few things which I have found necessary to get a system which will eventually allow me to fully replace OS X. I’m documenting them here for my own benefit, and for the sake of anyone else who might be considering running gNewSense or Ubuntu on a MacBook.

1. Getting the wireless card working. Prior to doing any system updates or installing any packages I might want, I had to get a working Internet connection. We share a wireless connection here, so plugging in direct was not really an option for any length of time. I used the madwifi drivers, which are not available as a gNewSense package (though they are now available in Ubuntu’s linux-restricted-modules package, according to the madwifi wiki). I don’t understand the licensing issues involved, because the COPYRIGHT file in the madwifi distribution says its source code is dual-licensed under the BSD and GPL licenses, which is good enough for me.

There was one snag trying to get the wireless drivers to compile. gNewSense DeltaH installs both the linux-image-2.6.24 and linux-image-2.6.24-generic kernels, but only the generic version properly uses both CPU cores, so that is naturally the one I was using. However, the default installation does not install the kernel headers for the generic kernel build, so madwifi would not compile against it. Therefore, it was necessary for me to plug into the wired connection and run


$ sudo apt-get install linux-headers-2.6.24-generic


before I could compile madwifi. Once the Linux headers were installed, though, running


$ make
$ sudo make install


in the madwifi directory worked flawlessly, and the kernel loads the driver at boot.

Update: after installing the linux-libre kernel, version 2.6.29-1, from Uncle Gnufs’ World Famous Home Baked Free Kernel Shoppe, wireless works without the Madwifi drivers. The built-in ath9k driver is used instead, and I have experienced no problems with it so far.

2. Getting the mouse working. The MacBook touchpad mostly works after a default installation, but a couple of features are missing that I really need and enjoy:

• right-clicking
• two-finger scrolling

Right-clicking is a must-have in any modern desktop environment, and it was actually the more difficult problem to solve. I am still not sure if I like the solution, but for what it’s worth, this will work:

1. Install the xkbset package:
   
$ sudo apt-get install xkbset

2. Create a file ~/.Xmodmap, and add the line:
   
keycode 115 = Pointer_Button3

   Key code 115 is for the left Apple key. You can use a different key if you wish; use the “xev” program to determine what the key codes for different control keys are.
3. On the System menu, go to Preferences -> Keyboard. On the “Mouse Keys” tab, check “Allow to control the pointer using the keyboard.”
4. On the System menu, go to Preferences -> Sessions. On the “Startup Programs” tab, click “Add.” Enter what you like for the name and comments; the command should be:
   
xmodmap /home/username/.Xmodmap && xkbset m

   (replacing “username” with your actual username).

5. Restart X with Ctrl-Alt-Backspace.
6. The LEFT Apple (Command) key will now simulate a right-click wherever the mouse pointer is.

At the moment, this solution is only working for me some of the time, and it doesn’t have the “native” feel of OS X’s Control-click for rightclick. (Indeed, you don’t have to click the mouse button at all to get a right click.) This feels uncoordinated to me, because the key is not a modifier for a normal mouse click (the way it is normally a modifier for a keypress); instead, it is a mouse click. This means that whatever key you choose must be solely dedicated to right-clicking. If anyone comes up with a better way to make a key/mouse-click combination function as a right click while still preserving the normal function of that key elsewhere, I would love to hear about it.

Two finger scrolling is easier to set up, though I don’t have it working as smoothly as in OS X yet either. Simply add these lines to your /etc/X11/xorg.conf file in the section which has the “Synaptics Touchpad” identifier:


Option "VertTwoFingerScroll" "1"
Option "HorizTwoFingerScroll" "1"
Option "TapButton1" "0"


(If you don’t want horizontal two finger scrolling, set that to “0″ instead of “1″. The third line turns off tapping on the touchpad as a click; leave it out if you like being able to click with a tap.) There are a couple of issues with this so far. First of all, the touchpad seems too sensitive to me: two-finger scrolling goes very fast, and accidental brushes of my palm or thumbs on the touchpad can have drastic effects on whatever I’m doing in a way they never did in OS X. Second, the right side of the touchpad also appears set up to be a scroll zone, which is redundant and unhelpful for me; I fixed this via the GUI (System -> Preferences -> Mouse -> Touchpad -> uncheck “Enable vertical scrolling”) but I don’t know how to set this permanently in the Xorg configuration, which I would like to do. Again, suggestions are welcome.

3. Installing GNU IceCat. IceCat is the GNU Project’s version of the Firefox browser. I much prefer it to Epiphany (which was mysteriously eating my downloads), and it adds some nice security features that Firefox doesn’t have. Downloading the .deb from ftp://ftp.gnu.org/gnu/gnuzilla/3.0.1-g1/ and installing it with dpkg worked fine.

Update: I’m having some difficulties getting Gnash, the GNU Flash player, to work with IceCat. I tried installing mozilla-plugin-gnash via apt-get and creating a link to the installed flashplugin-alternative.so from the IceCat plugins directory…and IceCat promptly crashed when I tried to open a Flashy site. I am going to assume that this is due to some kind of version conflict between the mozilla-plugin-gnash package and IceCat, and try compiling from source next.

Update 2: I compiled Gnash from source, and it works at last! I still can’t play videos on Hulu or YouTube, because they use Flash 9, and Gnash doesn’t have support for them yet, but oh well. Here are the steps to compile Gnash and the plugin for IceWeasel:

1. Install dependencies:
   
$ sudo apt-get install libboost-dev libboost-thread-dev libagg-dev libsdl1.2-dev libgstreamer0.10-dev libatk1.0-dev libatk1.0-dev libglib2.0-dev libgtk2.0-dev libungif-dev libjpeg-dev libboost-dev libboost-thread-dev libagg-dev libsdl1.2-dev libgstreamer0.10-dev libatk1.0-dev libatk1.0-dev libglib2.0-dev libgtk2.0-dev libgif-dev libjpeg-dev libxml2-dev libcurl3-dev libboost-date-time-dev libboost-serialization-dev

2. Obtain the Gnash 0.8.4 source .tar.bz2 from ftp://ftp.gnu.org/pub/gnu/gnash/.
3. Unzip the sources:
   
$ tar xjvf gnash-0.8.4.tar.bz2
$ cd gnash-0.8.4

4. Configure the build for GTK (not KDE) and to install the IceCat plugin:
   
$ ./configure --with-plugindir=/usr/local/lib/icecat-3.0.1-g1/plugins --enable-gui=gtk

5. Make and install:
   
$ make
$ sudo make install
$ sudo make install-plugins


4. Fixing font rendering. This solution comes from Deep Thought. Add this XML to ~/.fonts.conf:


<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
<match target="font">
<edit name="autohint" mode="assign">
<bool>true</bool>
</edit>
</match>
</fontconfig>


Other outstanding issues:

• Sound
• Sleeping/hibernating: the system does sleep (suspend to RAM) and resume, but after doing so, I can’t switch from GNOME to one of the text-based virtual terminals (e.g., via Ctrl+Alt+F2) — they’re all black.

That’s all for now. I will update this post as I discover solutions for other issues.