I’m a big fan of measurement and data so my motivation and goal for the Kindle log files was to see if I could extract some useful information about my Kindle use and reading patterns. In particular, I’m interested in tracking my pace of reading, and how much time I spend reading over time.

You’ll recall from the previous post that the Kindle keeps a fairly detailed syslog containing many events, including power state changes, and changes in the “Booklet” software system including opening and closing books and position information. You can eyeball any one of those logfiles and understand what is going on fairly quickly, so the analysis scripts are at the core just a set of regexps to extract the relevant lines and a small bit of logic to link them together and calculate time spent in each state/book.

You can find the scripts on Github: https://github.com/mattbnz/kindle-utils

Of course, they’re not quite that simple. The Kindle doesn’t seem to have a proper hardware clock (or mine has a broken hardware clock). My Kindle comes back from every reboot thinking it’s either at the epoch or somewhere in the middle of 2010, the time doesn’t get corrected until it can find a network connection and ping an Amazon server for an update, so if you have the network disabled it might be many days or weeks of reading before the system time is updated to reality. Once it has a network connection it uses the MCC reported by the 3G modem to infer what timezone it should be in, and switches the system clock to local time. Unfortunately the log entries all look like this:


110703:193542 cvm[7908]: I TimezoneService:MCCChanged:mcc=310,old=GB,new=US:
110703:193542 cvm[7908]: I TimezoneService:TimeZoneChange:offset=-25200,zone=America/Los_Angeles,country=US:
110703:193542 cvm[7908]: I LipcService:EventArrived:source=com.lab126.wan,name=localTimeOffsetChanged,arg0=-25200,arg1=1309689302:
110703:193542 cvm[7908]: I TimezoneService:LTOChanged:time=1309689302000,lto=-25200000:
110703:183542 system: I wancontrol:pc:processing "pppstart"
110703:193542 cvm[7908]: I LipcService:EventArrived:source=com.lab126.wan,name=dataStateChanged,arg0=2,arg1=:
110703:183542 cvm[7908]: I ConnectionService:LipcEventArrived:source=com.lab126.cmd,name=intfPropertiesChanged,arg0=,arg1=wan:
110703:183542 cvm[7908]: W ConnectionService:UnhandledLipcEvent:event=intfPropertiesChanged:
110703:193542 wifid[2486]: I wmgr:event:handleWpasupNotify(<2>CTRL-EVENT-DISCONNECTED), state=Searching:
110703:113542 wifid[2486]: I spectator:conn-assoc-fail:t=374931.469106, bssid=00:00:00:00:00:00:
110703:113542 wifid[2486]: I sysev:dispatch:code=Conn failed:
110703:183542 cvm[7908]: I LipcService:EventArrived:source=com.lab126.wifid,name=cmConnectionFailed,arg0=Failed to connect to WiFi network,arg1=:


Notice how there is no timezone information associated with the date/time information on each line. Worse still the different daemons are logging in at least 3 different timezones/DST offsets all interspersed within the same logfile. Argh!!

So our simple script that just extracts a few regexps and links them together nearly doubles in size to handle the various time and date convolutions that the logs present. Really, the world should just use UTC everywhere. Life would be so much simpler.

The end result is a script that spits out information like:

B000FC1PJI: Quicksilver: Read 1 times. Last Finished: Fri Mar 16 18:30:57 2012
- Tue Feb 21 11:06:24 2012 => Fri Mar 16 18:30:57 2012. Reading time 19 hours, 29 mins (p9 => p914)

Read 51 books in total. 9 days, 2 hours, 29 mins of reading time


I haven’t got to the point of actually calculating reading pace yet, but the necessary data is all there and I find the overall reading time stats interesting enough for now.

If you have a jailbroken Kindle, I’d love for you to have a play and let me know what you think. You’ll probably find logs going back at least 2-3 weeks still on your Kindle to start with, and you can use the fetch-logs script to regularly pull them down to more permanent storage if you desire.

I’ve never understood why some people consider their voting choices a matter of national security, so when via Andrew McMillan, I saw a good rationale for why you should share your opinion I found my excuse to write this post.

Party Vote
I’ll be voting for National. I’m philosophically much closer to National than Labour, particularly on economic and personal responsibility issues, but even if I wasn’t the thought of having Phil Goff as Prime Minister would be enough to put me off voting Labour. His early career seems strong, but lately it’s been one misstep and half-truth after another, the remainder of the Labour caucus and their likely support partners don’t offer much reassurance either. If I was left-leaning and the mess that Labour is in wasn’t enough to push me over to National this year then I’d vote Greens and hope they saw the light and decided to partner with National.

Electorate Vote
I live in Dublin, but you stay registered in the last electorate where you resided, which for me is Tamaki. I have no idea who the candidates there are, so I’ll just be voting for the National candidate for the reasons above.

MMP Referendum
I have no real objections to MMP and I think it’s done a good job of increasing representation in our parliament. I like that parties can bring in some star players without them having to spend time in an electorate. I don’t like the tendency towards unstable coalitions that our past MMP results have sometimes provided.

Of the alternatives, STV is the only one that I think should be seriously considered, FPP and it’s close cousin SM don’t give the proportionality of MMP and PV just seems like a simplified version of STV with limited other benefit. If you’re going to do preferential voting, you might as well do it properly and use STV.

So, I’ll vote for a change to STV, not because I’m convinced that MMP is wrong, but because I think it doesn’t hurt for the country to spend a bit more time and energy confirming that we have the right electoral system. If the referendum succeeds and we get another referendum between MMP and something other than STV in 2014, I’ll vote to keep MMP. If we have a vote between MMP and STV in 2014 I’m not yet sure how I’d vote. STV is arguably an excellent system, but I worry that it’s too complex for most voters to understand.

PS. Just found this handy list of 10 positive reasons to vote for National, if you’re still undecided and need a further nudge. Kiwiblog: 10 positive reasons to vote National

I had the first opportunity to use their services today when I needed an SSL cert to secure the IMAP server I run for my parents and I was very pleased with the experience. The web interface is a bit weird and you have to jump through some strange hoops, but to save paying more money to the SSL certificate cartel it seemed more than worthwhile.

Like most CAs these days the certificate which signs your server certificate is not the actual root certificate included in your operating system or browser, but an intermediate CA certificate which is in turn signed by the root certificate. This means that you have to ensure that your server includes the intermediate CA certificate alongside the server certificate so the client can validate the entire path back to the root.

Unlike Apache which explicitly allows you to specify a certificate chain file, the openssl methods used by Cyrus 2.2 only seem to recognise a single CA certificate in the file pointed to by tls_ca_file. All as not lost however, as the openssl libraries are actually quite smart and will automagically determine which intermediate certs they need to bundle into the handshake if you install them appropriately under /etc/ssl/certs (at least on Debian).

The trick is that you have to install the intermediate CA cert into a file named after the hash of the certificate, like so:

# wget http://www.startssl.com/certs/sub.class1.server.ca.pem -O /etc/ssl/certs/startcom-class1-intermediate.pem
# hash=$(openssl x509 -hash -noout -in /etc/ssl/certs/startcom-class1-intermediate.pem)
# ln -s ./startcom-class1-intermediate.pem /etc/ssl/certs/${hash}.0
# ls -l /etc/ssl/certs/${hash}.0
lrwxrwxrwx 1 root root 34 2011-06-13 07:43 /etc/ssl/certs/ea59305e.0 -> ./startcom-class1-intermediate.pem


Then in imapd.conf:

tls_cert_file: /etc/ssl/certs/your-server-cert.pem
tls_key_file: /etc/ssl/private/your-server-key.key
tls_ca_file: /etc/ssl/certs/startcom-ca.pem


Voila. Works everywhere I’ve tried so far.

Start Com – Highly Recommended. I’ll be using them for any future SSL certificate purchases (e.g. EV certs) that I need to make.

By default Linux will ignore IPv6 RAs if the interface is configured to forward traffic. This is in line with RFC2462 which states that a device should be either a Host or a Router. If you’re forwarding packets you’re a router and you’re therefore expected to be sending RAs, not receiving them. This policy does make a certain amount of sense but there are obviously situations where it can be useful to accept RAs and still forward packets over the interface[0]. The confusing part is that the Linux IPv6 stack allows the accept_ra sysctl to be set to 1 (enabled) at the same time as the forwarding sysctl is set to 1, yet all incoming RAs are ignored with no hint as to why. If you’re not aware that the default behaviour is to ignore RAs when forwarding is enabled it looks very much like autoconfiguration has simply broken.

The key piece of information is that makes everything as clear as mud is realising that the forwarding and accept_ra sysctl’s are not simple boolean enabled/disabled flags like many of their brethren. There are instead three possible values for each, all clearly documented in sysctl.txt, when you take the time to read it. Ironically the documentation states the type of the values as “BOOLEAN” even though they’re not… at least it helped me to feel better about my hasty assumption that the sysctl’s were boolean values.

accept_ra – BOOLEAN
Accept Router Advertisements; autoconfigure using them.

Possible values are:
0 Do not accept Router Advertisements.
1 Accept Router Advertisements if forwarding is disabled.
2 Overrule forwarding behaviour. Accept Router Advertisements
even if forwarding is enabled.

Functional default: enabled if local forwarding is disabled.
disabled if local forwarding is enabled.

The documentation for forwarding is similar, but much longer, so you can refer to the link above to see it.

Conclusion: If you want to autoconfigure IPv6 addresses on an interface that you’re also forwarding IPv6 traffic over, you need to set accept_ra to 2.

No doubt there are more IPv6 quirks and defaults like this waiting to trap me in the future

[0] Arguably you really don’t want to be autoconfiguring addresses on your router ever, but that’s a philosophical debate that isn’t really relevant to this post.

Almost as much fun as reading on the device has been learning about how it works. The Mobile Read forums have lots of step by step posts on how to do specific tasks like replacing the screensaver image, but they don’t give much background detail on how the Kindle is actually operating which is what really interests me. Luckily among all the step by step posts I also found a “usbnetwork” package which also adds an SSH server to the Kindle, so after installing that and then SSHing in to my Kindle I’ve been poking around.

Under the cover the Kindle reveals a fairly standard Linux installation. While the hardware and IO devices are obviously unique, compared to something like an Android phone, the Kindle is refreshingly “normal”.

Hardware

• Freescale MX35 ARMv6 based CPU with Java specific instruction support.
• 256MB RAM
• 4GB of internal flash presented as an SDHC device with four partitions. A ~700MB root partition, a ~30MB /var/local partition, another roughly 30MB kernel partition and then the rest (~3.1G) as the writeable “user” partition where your books and other content are stored. The root and /var/local partitions are ext3! (not jffs or some other more traditional flash based file system) while the user partition is vfat for easy use with Windows, etc.
• The board is code-named ‘luigi’ and there are lots of references to ‘mario’ and ‘fiona’ scattered around the device, and even in some URLs on Amazon’s website. Someone was obviously a Super Mario fan.
• The wireless chipset is Atheros based using the ar6000 drivers.
• The WAN (3G) modem presents itself as a USB serial device and is controlled via a custom daemon named ‘wand’ which uses the standard Linux pppd package to establish IP connections over a private APN Amazon (provided by Vodafone here in Ireland).
• The EInk display shows up as some special files under /proc rather than as a device. With a bit of digging I found some simple constants that when written to the proc files cause the screen to display the standard boot/progress/upgrading images. I haven’t deciphered how to make more complex updates to the display yet.

Software

• The kernel is based on Linux 2.6.26, with a bunch of hardware specific patches and drivers from lab126.com, an Amazon subsidiary who appear to be responsible for much of the low-level driver and device development.
• Lots of familiar open source projects are present, e.g. syslog-ng, DBus, busybox, pppd, wpa_supplicant, gstreamer, pango, openssl and the list goes on. You can download all the sources from Amazon’s website. I haven’t spent any time to see what if anything has been modified.
• There were a few unexpected finds as well such as GDB and powertop! No doubt useful for the developers, but highly unlikely to actually be used on a shipping Kindle.
• Boot-up is controlled by a set of sysv style init scripts which setup the filesystems and then start a handful of daemons to look after the low-level subsystems (network, power, sound) as well as the standard syslog and cron daemons you’d expect to see on any Linux box.
• Once the basic system is up and running the init scripts kick off the “framework” which lives under /opt/amazon/ebook and consists of lots of Java classes. The system uses the cvm Java environment from Sun/Oracle which is specialised for embedded low-memory devices like this. The framework appears to take over most of the co-ordination, management and interaction tasks once it has started up.

The application/framework code is heavily obfuscated apparently using the Allatori Java Obfuscator. The jrename and jd-gui utilities have proven very handy in helping to untangle the puzzle, although they still only leave you with a pile of Java source code with mostly single letter alphabetic variable and class names! I’ve been using IntelliJ’s support for refactoring/renaming Java code to slowly work through it (thanks in large part to error/log messages and string constants found through the code which can’t be obfuscated easily and help to explain what is going on), and I’m slowly beginning to piece together how the book reading functionality works. I’ll maybe write more on this in a future post.

In one of my initial tweets about the Kindle I mentioned that it seemed to be regularly uploading syslog data to Amazon based on some sendlogs scripts I’d noticed and a few syslog lines containing GPS co-ordinates that had been pasted on the Mobile Read forums. I can’t find any trace of GPS co-ordinates in any syslog messages I’ve seen on my device, but there is definitely information about the cell sites that my Kindle can see, the books that I’m opening and where I’m up to in them:

101206:235431 wand[2515]: I dtp:diag: t=4cfd77b7,MCC MNC=272 01,Channel=10762,Band=WCDMA I IMT 2000,Cell ID=1362209,LAC=3021,RAC=1
,Network Time=0000/00/00 00.00.00,Local Time Offset=Not provided,Selection Mode=Automatic,Test Mode=0,Bars=4,Roaming=1,RSSI=-88,Tx
Power=6,System Mode=WCDMA,Data Service Mode=HSDPA,Service Status=Service,Reg Status=Success,Call Status=Conversation,MM Attach St
ate=Attach accept,MM LU State=LU update,GMM Attach State=Attach accept,GMM State=Registered,GMM RAU State=Not available,PDP State=
Active,Network Mode=CS PS separate attach mode,PMM Mode=Connected,SIM Status=Valid; PIN okay; R3,MM Attach Error=No error,MM LU Er
ror=No error,GMM Attach Error=No error,GMM RAU Error=Not available,PDP Rej Reason=No error,Active/Monitored Sets=0;39;-11 1;180;-1
5,RSCP=-111,DRX=64,HSDPA Status=Active,HSDPA Indication=HSDPA HSUPA unsupp,Neighbor Cells=,Best 6 Cells=,Pathloss=,MFRM=,EGPRS Ind
ication=,HPLMN=,RPLMN=272;01 ,FPLMN=234;33 234;30 234;20 272;05 ,n=1:

101206:233416 udhcpc[5639]: Offer from server xxx.xxx.2.254 received
101206:233416 udhcpc[5639]: Sending select for xxx.xxx.2.10...


Interestingly you can see from the last two lines, that Amazon has taken some care to preserve privacy by not including the full IP address given to the device by my local Wifi network, so in light of that I find it interesting that they decided not to obfuscate the Cell and Book IDs in those respective log messages too. Seems rather inconsistent.

As to how and when these logs are sent to Amazon, the picture is a little bit murky. Every 15 minutes tinyrot runs out of cron and rotates /var/log/messages if it is greater than 256k in size. Rotated logs are stored into /var/local/log under filenames like messages_00000044_20101207000006.gz and alongside the log files are a set of state files named nexttosendfile, messages_oldest, messages_youngest. Something regularly sweeps through this directory to update the state and remove the old logs (after sending them up to Amazon I assume). I suspect that something is buried in the Java application code mentioned above.

On the whole the Kindle is a fascinating piece of technology. It delivers a wonderful reading experience on top of a familiar Linux system and is going to provide me with many more hours of entertainment as I unpack all the tricks and techniques that have gone into this device. I would recommended it as a present for geeks everywhere.