Krzysztof Kowalczyk blog

SumatraPDF 0.9.3 released

2008-10-02T12:22:00Z

Yet another update to Sumatra. Changes in this version:

ability to disable auto-update check
improved text rendering – should fix problems with overlapping text
improved font substition for fonts not present in PDF file
can now open PDF files with non-ascii names
improvements to DDE (contributed by Danilo Roascio)
SyncTex improvements
improve persistance of state (contributed by Robert Liu)
fix crash when pressing ‘Cancel’ when entering a password
updated translations

SumatraPDF 0.9.1 released

2008-08-24T22:31:00Z

A smaller update to my PDF viewer for Windows. The biggest improvement is in handling more PDFs correctly (e.g. those generated with Crystal Reports)

Other changes:

improved rendering of some PDFs
support for links inside PDF file
added -restrict and -title cmd-line options (contributed by Matthew Wilcoxson)
enabled SyncTex support which mistakenly disabled in 0.9
misc fixes and translation updates

SumatraPDF 0.9 released

2008-08-11T19:31:00Z

SumatraPDF is a nimble PDF reader for Windows and it just received an update.

The biggest change in this version comes from completely removing poppler rendering engine and fully relying on mupdf. That makes the code a little bit smaller and a little bit simpler.

Other changes:

add Ctrl-P as print shortcut
add F11 as full-screen shortcut
password dialog no longer shows the password
support for AES-encrypted PDF files
updates to SyncTeX/PdfSync integration (contributed by William Blum)
add -nameddest command-line option and DDE commands for jumping to named destination(contributed by Alexander Klenin)
add -reuse-instance command-line option (contributed by William Blum)
add DDE command to open PDF file (contributed by William Blum)
removed poppler rendering engine resulting in smaller program and updated to latest mupdf sources
misc bugfixes and translation updates

realloc() on Windows vs. Linux

2008-07-27T13:31:00Z

Short version: realloc() on Windows can have much worse performance than on Linux/Mac.

As part of working on Sumatra, I sometimes work on mupdf, underlying PDF parsing/rendering engine. I’m not an expert on that code but I try to fix bugs and try to improve performance.

In testing I noticed that on some PDFs it’s pathetically slow, so I profiled the code and that told me that 99% of the time was spent in realloc(). Also, the same PDF was rendered almost instantaneously on Linux.

The code was reading a stream of unknown (a priori) length and the way it worked was by reading 4k chunks of data into a buffer that was grown with realloc() call.

If you’re lucky, realloc() will return the original block of memory and just fiddle with internal allocator structures to make it bigger.

If you’re unlucky, realloc() will have to allocate a new block of memory and copy the date from old memory.

If you’re reading 1MB file and growing file data buffer by 4k, you’ll have to call realloc() 256 times and each time the buffer will be bigger. If you’re unlucky and realloc() has to copy the data every time, that’s a lot of cpu time spent data copying.

From observed behavior I have to conclude that realloc() implementation on Linux and Mac is more likely to be able to reuse original memory than implementation in Windows’ CRT (which most likely just calls HeapRealloc()).

Once I knew what the problem was, the fix was simple.

My first implementation would read into separate 4k buffers and at the end allocate the result buffer and copy the data just once.

My second implementation was even simpler: it would grow the buffer size by doubling it, instead of just increasing by 4k. It was slower than first implementation (more copying) but simpler and fast enough.

A side note: a simple and incorrect conclusion would be that memory allocator implementation on Linux is much better than on Windows. Things aren’t that simple, though. It’s easy to write an allocator that has excellent performance for realloc() but is terrible for everything else. Engineering is an art of compromises and optimizing realloc() apparently wasn’t high on Microsoft’s list of things to shoot for when they were designing allocator in Windows, which is, btw, comparable in performance to allocator in Linux or Mac.

Minimalism is the new black

2008-07-18T18:46:00Z

It was a time for visual refresh and what is more refreshing than a dash of minialism? Minimalism is good, even when shamlessly stolen.

This also marks a switch from WordPress to a completely static site. Feels like going back in time but the upside is removal of a source of security vulnerabilities.

Another switch is sponsored by the department of unnecessary tinkering: Apache is out, nginx is in.

Another switch is for the environment and my pocket: ditching theplanet.com (which does way more that my needs are at the moment and is priced accordingly) and moving to more modest and cheaper sliecehost.

Announcing fofou - forum software for Google App Engine

2008-07-06T18:33:13Z

Elevator pitch: Fofou is forum software optimized for software product support/discussions. Free to use, easy to install and free to run, thanks to Google's App Engine.

I believe that every software product (be it open source or commercial) should have associated forums so that users can ask questions about the software, suggest improvements or simply tell you how much they love your software.

Unfortunately most forum software is bad. I call it phpBB-disease (although it's probably UBB that is the root of all evil in this case). The sad fact of life is that popular software gets cloned. That, by itself, isn't bad, except in cases where the popular software is bad. phpBB is bad but people who write their own forum software mindlessly copy its design, just because it's popular, with only minor variations. The end result is that we get more and more of the same awfulness.

The first reason why most forum software is bad for software support is that it requires registration. Software forums are mostly for casual interaction: ask one question, report one bug, suggest one feature. Asking someone to register just to do that is stupid and a great way to turn away a big percentage of people. I've been in that situation myself and it makes me angry when a software vendor makes me jump through hoops just to report a bug in their software.

Another reason, harder to describe succinctly, is that most forum software is way too complicated. They show way too much information, for example a date when user registered. On every page. First: who cares? Second: I always confuse that with the post date, because it's more visible.

They encourage creating multiple forums, which doesn't make sense in low-frequency forums. But the functionality is right there, the interface is optimized for showing multiple forums so hapless developer creates a separate forum for bug reports, another one for announcements, another one for feature requests, another for miscellaneous discussion about the product and yet another about miscellaneous discussion about everything else and then over months experiences what it's like to host dead forums with nary a post per month. One forum is enough for everything until the volume justifies creating more forums.

That's just two example out of many - most developers endlessly add more and more features and few are willing to Get Real 37signals-style.

But once in a while someone comes in and shows that there is a better way. In case of forum software it was Joel Spolsky who wrote forum software from scratch for the needs of his software company. Even more importantly, he described eloquently what kind of mistakes most forum implementations make and how his forum fix those mistakes.

The code for his implementation is not available but Wayne Venables wrote FruitShow, a re-implementation of Joel's ideas in PHP.

Which brings me to another reason why most forum software is bad: it's hard to install. Even when the code is free, it's still a hassle to run your own server, install Apache, PHP and MySQL and make sure it all runs fine.

This is where Fofou comes in: it re-implements Joel's forums on Google App Engine infrastructure, so not only is it easy to deploy your own copy but it's also free to run (it's very unlikely that you'll go over Google's free quota).

Fofou is also an open-source project, so you're free to mess with it, change it any way you want and contribute improvements.

I believe that thanks to ease of development, ease of deployment and generous free quotas, App Engine has a potential to revolutionize the landscape of low-end web hosting. App Engine is plenty powerful to host your website, blog, wiki, forums and what not, for free, with better quality guarantees than most low-end web hosts.

It doesn't make sense to pay even $5/month for low-end web hosting and mess with configuring web servers and databases and PHP software.

It doesn't make sense to pay $5/month for TypePad blog, if you could install TypePad-like software on App Engine and run it for free.

It's even attractive to use App Engine instead of free (read: ad supported) services like blogger or wordpress.org, because you get rid of ads, you can install any blog software you like and you get more control over the website.

It's still early in App Engine lifetime so we're not yet at a point where this is true. The biggest missing piece is availability of high-quality, free re-implementations of the most common types of software people use: blogs, wikis, forums etc. Fofou is here to give you one more choice when it comes to forum software.

Why Google should sponsor a faster Python VM

2008-07-05T23:47:11Z

It's not very secret, and it's not little or dirty, which is why I'm not calling it a dirty, little secret.

Python is slow, it's a fact.

Another fact: it doesn't have to be slow.

In 1994 a couple of guys who spent a lot of time investigating creating fast VMs for dynamic language in the context of Self project at Standford decided to create a super fast, commercial Smalltalk implementation. They got half-way there. Within two years they created Strongtalk, an almost working VM that beats in speed even today's commercial Smalltalk implementation. It was never finished because Sun bought the company in 1997 and their ideas and talent were applied to Java VM, which is why Java today is really, really fast.

The techniques used to make Java and Smalltalk go really, really fast aren't secret - there are papers, Strongtalk source is available under BSD license and Java VM source is also available for inspiration.

There's nothing about Python that would preclude using those techniques to make it way faster. And I'm not talking few percent faster, or even twice as fast: the speedup could be tenfold.

It's all possible if few smart people apply themselves to the problem. Unfortunately those few people are apparently busy doing other stuff other than contributing to Python.

This is where Google could come in and apply few dollars to hire few smart people to specifically work on writing super fast Python VM.

Why would Google care? Because they could save money. Lots of money. And lots of energy so that Sergey can enjoy his dollars a few yers more before global warming kills us all.

Lots of Google's software is written in Python, and it'll grow even more thanks to Python being a default language in Google App Engine. One of Google's big expenses is processor power and energy needed to power their servers. If an application becomes faster, it directly translates into using less servers which translates into paying less money for hardware and power and cooling and data center space. At Google scale making Python twice as fast surely must translate into saving megabucks. Hiring few smart people will cost way less than megabucks.

What are they waiting for

Google App Engine tip

2008-07-05T21:21:52Z

Don't put template files in directories marked as static dirs in app.yaml file. For whatever reason those files are not available to template.render() function.

What's worse: this works locally, in dev environment and only breaks when you upload the app. I've learned that the hard way.

I've opened a bug for this issue (so go there and star it) but I wonder if Google even looks at them anymore, after all the requests to add Pascal support.

SumatraPDF 0.8.1

2008-05-29T05:28:23Z

Shipping feels good, I just shipped 0.8.1 version of SumatraPDF hence I feel good. Changes in new version:

automatic reloading of changed PDFs (contributed by William Blum)

tex integration (contributed by William Blum)

updated icon for case-sensitivity selection in find (contributed by Sonke Tesch)

language change is now a separate dialog instead of a menu

remember more settings (like default view)

automatic checks for new versions

add command-line option -lang $lang

add command-line option -print-dialog (contributed by Peter Astrand)

ESC or single mouse click hides selection

fix showing boxes in table of contents tree

translation updates (contributed by many people)

Go get it.

Extreme (size) optimization in C and C++

2008-05-20T05:43:58Z

Following the sage advice of releasing early and often, I'm releasing my magnum opus Extreme (size) optimization in C and C++.

It's meant to be a book about techniques to make your C, C++ and even Objective-C (it being a superset of C) code as small as possible and use as little memory as possible. All based on real-life code.
It's certainly early (only one chapter is complete) and hopefully I'll be updating it often.

So give it a look and enjoy the extremisms.

_NT_SYMBOL_PATH considered harmful

2008-04-18T02:31:59Z

When you debug your software on Windows, it’s sometimes useful to also have access to symbols for Windows DLL. Microsoft is kind enough to provide them for download and the easiest way to make them available locally is to set _NT_SYMBOL_PATH environment variable so that it downloads symbol files locally from Microsoft servers when needed. The right value is srv*c:\symbols*http://msdl.microsoft.com/download/symbols (you can change c:\symbols to any directory on your hard-drive).

I was running such setup for some time but observed that starting up a debugging session too a long time. It was annoying and after a while it hit me that I’ve caused this: even the simplest Windows program implicitly links a lot of DLLs. Both Visual Studio and WinDBG feel the need to load up symbols for all those system DLLs and it turns out it takes a lot of time.

I unset _NT_SYMBOL_PATH and I’m a happier person for it. Most of the time I don’t need system symbols since the bugs are in my code, but sometimes having them is useful (e.g. when the crash happens inside Windows). Fortunately you can still have access them. When needed you can use .sympath to set the path (e.g. “.sympath srv*c:\symbols*http://msdl.microsoft.com/download/symbols”) and then do .reload to force loading of symbols.

Remapping Page Up and Page Down on Mac to move a cursor

2008-04-17T02:51:51Z

It’s probably just a matter of being used to certain ways, but being a recent Windows->Mac switcher, the fact that Page Up and Page Down in a text editor moves the text without moving the cursor was driving me nuts.

I’m not the only and this article describes how to fix it. And, as an additional bonus, how to remap Home and End keys to move to beginning/end of a line, instead of beginning/end of a file.

Thinkpad x60 - a retrospective

2008-04-17T00:49:00Z

The best time to review a gadget is after using it for months. Unfortunately given the churn in PC electronics, it usually means that the gadget will be obsolete.

Nevertheless, I’ll summarize my feelings towards x60 after almost year of use despite the fact that you can’t buy it any more.

As a programmer I like my compilations go fast so I have rather high demands on my laptops but those days it’s almost a non-issue. x60 doesn’t have the fastest processor, a weak graphics card and a hard-drive that is far from being a speed demon, but as long as I stuff it with 2 GB of RAM, I have no problems using it with Visual Studio or even hosting a Linux inside VMWare. Yes, things could always be faster but they’re fast enough for me.

I bought x60 (as opposed to some other Thinkpad laptop) mostly because of its tablet features. I thought that I’m compromising on the size and that 12” isn’t enough work comfortable work (which was not a big deal because I still had my 14” t61 as a backup). Turns out I used it as a tablet only a couple of times. I’m not complaining because I also discovered that 12” laptop is good enough for day-to-day work (as long as it has 1400x1050 screen) and I ended up shelving my t61 and using primarily using x60.

I count myself as disgruntled Vista users. It really is worse in many ways than XP, sometimes in infuriating ways (like taking forever to show files in a folder on the hard-drive or annoying with UAC dialogs to the point that the only sensible solution was to disable it). Kind of pathetic, given that it was promised to be second coming of Windows OS and that every new release of Mac OS X is praised to heavens (and not without reason).

It’s even more infuriating that I can’t upgrade to SP1 because, get this, SP1 won’t install if it detects an known incompatible driver. Turns out my video driver is on that list. Lenovo doesn’t seem very motivated to provide an updated driver even after weeks since SP1 release (seriously, they should have upgraded the driver before SP1 official release - it’s not like it was a tightly kept secret). Driver from Intel site doesn’t install claiming my computer is not authorized. Thank you Intel and Lenovo for exemplary team work - I imagine there’s nothing as rewarding as pissing off people who spend thousands of dollars on your stuff.

I phantasize about downgrading to XP and Lenovo even has a program where they offer to send an XP downgrade CD for a certain price for selected devices (mine apparently included). That’s what one of their web pages claim. However, they were clever enough to make it impossible to actually apply for that. I spent (literally) hours trying to find a way to order that CD and failed. Their web pages just point back to each other, possibly as way to showcase an infinite loop.

Crapware - another annoyance courtesy of Lenovo. There are countless Lenovo branded programs with cryptic names. They all want to load on startup and show ugly icons in the start bar. I can’t understand from their names what they’re supposed to do but apparently not much since I see no functionality loss after uninstalling most of them. Lenovo might not be as bad as Sony or Dell, but it’s bad.

Despite the annoyances, it’s a good laptop and will do until I can buy a laptop with at least 128 GB of SSD much faster than current best for a reasonable price.

I would entertain switching to Mac but Apple doesn’t seem inclined to offer a resolution higher than 1280x1024 on their 13” laptop and I’m spoiled by 1400x1050. Once you taste it, there’s no going back.

Software worth buying - SftpDrive and ExpanDrive

2008-04-16T01:47:17Z

SftpDrive and ExpanDrive is really the same program, except named differently for different OSes. SftpDrive is for Windows and ExpanDrive is for Mac.

It does one thing very well: it allows mounting any ssh accessible directory as a hard-drive/volume so that all programs running on your Mac/PC can access it as if it was a local drive.

One use for that is as a replacement for sftp GUI client.

But what is really great is that it greatly improves interaction with remote servers, be it an Amazon EC2 instance or a co-located server. When I had to edit files on remote Linux servers, I used to ssh and use Emacs for editing. Despite investing many hours in learning Emacs and elisp (to the point that some of the keystrokes are hard-wired in my brain) but I’m definitely disgruntled Emacs user and in my love-and-hate relation with it I have much more hate than love. Not to mention that for editing C/C++ code nothing comes even close to Source Insight.
Now I can just mount a directory and transparently edit remote files using my favorite, most familiar editor.

It’s a simple idea but the brilliance of SftpDrive is that it works really, really well and on sufficiently fast connection I can comfortably edit big projects. This might seem like an obvious property of such program but my experience with Samba (either when accessing my NAS over wireless Wi-Fi N network or accessing a filesystem exported from Linux running on Windows under VMWare) shows that this obvious property is lacking and trying to edit large code base in Source Insight on a samba-exported files is an exercise in restrain from chewing your own hand when Source Insight locks up for painful seconds when you’re in the middle of typing code.

Two thumbs up for SftpDrive/ExpanDrive. If only they also supported mounting S3 as a filesystem.

Software worth using: andLinux

2008-04-14T05:50:31Z

I live in a multi-OS world and it’s nice to be able to do multi-platform work on just one physical computer. Since my main OS is Windows, when I need to do Linux work I resort to VMWare. While being great piece of software, it annoys me that the Linux world lives in its own sandbox. Sharing the filesystem doesn’t work very well (regardless of whether I use VMWare’s built-in folder sharing or use samba in either direction) and while I’m in VMware, Alt-tab no longer does the expected thing.

andLinux solves some of those problems and, as a bonus, is free. andLinux is an easy to install Ubuntu distribution based on coLinux.

While VMWare emulates the whole hardware, so that you can install any OS and it won’t see any difference, coLinux is a modified Linux kernel that runs as a process on Windows. Based on that it’s possible to build a whole Linux distributions and andLinux is a pre-packaged Ubuntu with Windows installer.

After installation you get ability to run a console inside Linux and the advantage over VMWare approach is that a console is just another application on Windows. Open five consoles - you just get 5 windows that behave like any other Windows application.

By also running an X-server on Windows side, it’s possible to launch graphical applications.

There are some downsides to coLinux, the biggest one being the fact that it’s slower than VMWare. Latest betas are supposedly faster than what andLinux uses currently, so there’s hope it’ll get better.

The Linux “hard-drive” is just a file on your Windows filesystem. The default size of that file is only 2GB so you’ll likely need to increase it quickly, which isn’t straightforward. The method I used involved expanding the file on Windows with dd command and then running a “resize ext filesystem” program on the Linux side. It’s not the safest way to do it, but it worked for me.

File sharing between Linux and Windows side can be done either by cofs driver which allows mounting any directory on Windows under /mnt, or samba.

On the whole andLinux is an attractive alternative to VMWare. I wouldn’t use for tasks that are heavy on CPU and disk-drive (e.g. compiling a large code base) since it can be twice as slow as VMWare on the same hardware, but when that is not a factor, a better integration with Windows is really helpful.

Pet peeves of mine - files

2008-04-12T18:22:06Z

Files are fine, I just don’t like that programs expose me to them for no reason at all.

Take TextMate for example. It allows you to create projects. Project is just a bunch of files so it’s really a tiny amount of information. Yet TextMate forces me manually manage projects by saving them to files.

Instead of just remembering all projects I have ever created and giving me some simple open/create new/modify/delete interface for projects, TextMate forces me to fish them out myself from filesystem.

This disease afflicts many, many programs. The only editor that I know that does projects right is Source Insight.

Someone concurred

2008-04-09T03:32:58Z

Most people missed it, but not Dave Winer:

What you're going to see here that you've never seen before is shrinkwrap net apps that scale that can be deployed by civillians.

Exactly. I would gloat that I said it earlier, but he dreamed it in advance:

The next step after that will be packaged applications that deploy through Amazon that you can buy for shrinkwrap prices.

Speaking of predictions: one of the Google example apps was HuddleChat, written in a couple of days by 3 people and bearing uncanny resemblance to 37Signals' paid product Campfire. Bruhaha ensued, the app went offline, 37Signals is "disappointed with Google".

Like I wrote yesterday, they (and a host of other companies betting their business on providing hosted, specialized app for a fee) will be even more disappointed when apps written to Google's App Engine copying functionality of those hosted applications will become available (either for a one-time fee or open-sourced and free) and deploying such apps on GAE will be no harder than signing up for the service.

It's all technically possible. It's not going to be too hard for Google to build a packaging system and a catalogue of applications and provide one-click deployment of any GAE app.

Google App Engine - the first Internet operating system

2008-04-08T08:59:26Z

I've read all of the commentary on Google App Engine and most people agree: it's a competition for Amazon's AWS. Or a copy-cat.

They might be right but that's not why Google's service is disruptive. Google App Engine really is the first Internet operating system and given generous free plan it spells death for low-end web hosting and some of the specialized application hosters.

Currently I pay The Planet $150/month to lease a decent server. It's all mine but all I really use it for is my WordPress-based blog and 2 minor websites. Yes, I overpay, I could move to a low-end web host for tens of dollars per month.

But now I have another choice: I could move my blog to Google's data center. Hard to beat reliability, no overselling tricks that Dreamhost (and many others) use and their free plan is more than generous to host a website (on a good day I have 3 thousand visitors, which adds up to 90 thousand per month - plenty of room to grow until I reach Google's estimated 5 million cap).

The free plan is more than generous for many, many people. Need a website for your small company? Host it on GAE for free. Need a blog? Host it on GAE for free (and you don't even need to show ads). Need a website for your free software project? Host it on GAE for free.

I don't think that's exactly what Google had in mind but after skimming terms of service I don't see anything that says "you shall not host your personal websites".

Pretty much anything you can do on a low-end webhost you can also do on Google App Engine. I'm pretty sure we'll quickly see a number of applications like blogs, wikis etc. written to GAE and people will be able to run their own copy of the app in Google's cloud.

That's why it is the first, true Internet operating system. It provides a set of services for running an application just as Windows provides set of services for running applications.

An application written to GAE is like a software package and individuals can run their own copies of those software packages on Google's machines.

Other companies that should be afraid, very afraid of GAE are hosters of specific applications that don't have network effects. Companies like SixApart, PBWiki, WordPress.com, github, 37signals, FogCreek etc., etc.

They have a very comfortable business: those days hosting a blog, a wiki, a chatroom costs pennies per user per month and they charge several dollars. Sign up few thousands paying customers and you have steady revenue flow and plenty of money to subsidize basic free accounts and work only 4 days a week. But why would I pay 37signals $12/month for Campfire if I could install a Campfire-like application on GAE OS, for free and with full control over my data (as opposed to believing that 37signals will not abuse my data)?

Yes, those companies also benefit from GAE if they use it for hosting, but at their scale it won't be free and they can only stay competitive if they improve their apps faster than people writing apps for GAE. Or if their apps have built-in network effects (Facebook does, MySpace does, but a blog engine or a wiki doesn't).

Google even competes with their own ad-supported businesses like Blogspot or even Gmail. If someone writes Blogspot or Gmail clone that could be run on GAE, there's nothing stopping people to ditch Blogspot (along with forced ads) and run blogspot-like app on GAE. Yay for unintended consequences.

Since GAE is really an operating system, how can people make money? Shrink-wrapped software. I could, for example, write MBE (the Mother of all Blog Engines) and sell it for a fixed price to people so that they can run their own copy on Google's machines.
Granted, this probably won't happen because open source is the norm in web apps and the norm has a way of perpetuating itself.

In a way it's all possible today: I can download a copy of WordPress and run it on a host of my choice, but, coming back to PC analogy, it is as if I had to assemble my own computer from parts and install an OS customized to that application in order to run.
With current approach the language of the application is given (e.g. PHP + MySQL or Python + SQLite or Ruby + PostgreSQL) and I have to assemble environment to run the app. That is a major pain in the ass.

GAE inverts this: the environment is given and application writers need to use available language and APIs. It shifts the burden from users of the application to developers, which is how it should be, because there are much more users than developers.

GAE is revolutionary because it provides a standardized environment for running applications and standardized OS leads to explosion of available applications. Compare number of apps on Windows with number of apps on fragmented Unix variants.

GAE covers some of the same ground that AWS but even though it's less powerful, it's also much more ground breaking. By providing an Internet OS it enables shrink-wrapped software model for Internet apps: write the app once, run it many times.

You can expect renewed interest in improving on old ideas. There will be multiple blog engines competing for your attention, all with the same ease of installation. Today there's little point in writing a new blog engine becuase it's a lot of work and few big aggregators took the market.

GAE levels the playing field. Once people learn GAE some aspects of developement will be much easier e.g. you'll no longer need to be accomplished DBA, unix admin, Apache and MySQL deployment master combined into one. Being a Python programmer will be enough.

When each app will be incredibly easy to install, more people will do it and apps will be competing on their looks. Barrier to experimentation and creating new variations will be dramatically lowered.

GAE will create a market for deployable apps and where's the demand, there'll be supply.
Everybody is focused on the startup angle of GAE - how it will enable easy scalability for new web services but I think they're all missing the point (and therefore I'm smarter than everybody else). I say that GAE will be much bigger as a service for running personal applications like blogs, websites, wikis etc. (assuming that Google will remain as generous as they are now with their free plan).

I'm sure that Microsoft works on a similar platform - they used to be the kings of the platform so they should understand what the right model is, but Google not only is first to market but their willingness to be a loss leader might throw a wrench into Microsoft's plans. I doubt they factored hosting small fish for free into their plans.

Those are indeed exciting times and I almost wish I lived in the future, when I can get my hands on non-beta version of GAE.

gflags - a debugging story

2008-04-07T07:28:39Z

I hate crashes that disappear when run under the debugger and I had one when porting mupdf to Windows.

It helps to know that there's at least one reason for a changed behavior under the debugger: it automatically triggers using debugging heap. While debugging heap usually helps find problems, sometimes it does the opposite by changing the details of memory allocation.

One helpful tool when debugging memory problems on Windows is gflags which can enable page heap instrumentation for a given program. It works by putting each allocation into a separate region of memory and putting a non-readable page right after that. Also, upon freeing it makes the memory unreadable. That way an overwrite of memory block while it's still being used or accessing the memory after it was freed will cause immediate crash.

The downside is that using gflags uses much more memory. But in those days of cheap gigabytes it's not a problem that can't be solved with a couple hundred bucks.

Basic usage of gflags.exe is simple: gflags /p /full /enable foo.exe

From now on foo.exe will always be run with this instrumentation turned on. To disable, do gflags /p /disable foo.exe

To see which programs have page heap enabled, do gflags /p. gflags offers many other option and you can learn about them via gflags /?. If you run gflags without any options, you'll get a (very confusing) GUI.

It worked like a charm. I got a crash on accessing freed memory and all I had to do was to backtrack to where this memory was allocated to figure out the problem.

picoLisp - Arc before Arc

2008-02-02T23:40:49Z

Arc has generated an amount of web buzzing way out of proportions to what it has to offer (a small lisp core implemented as an interpreter in MzScheme). At this point it has almost nothing interesting that hasn't been done elsewhere and by any reasonable standards it's just a toy for exploring ideas and not a language implementation that can be used for writing software.

I think it illustrates some basic truth (or maybe just a fact) about advertising, human nature, lack of fairness except I'm no-where near being able to state coherently what that basic truth is.

I can, however, point to a picoLisp - an Arc-like lisp implementation that has been around for many years (picoLisp 2.0 was released in December 2002 while Arc was announced in November 2001).

Well, if justice was to be served, Arc should be called picoLisp-like lisp implementation because picoLisp was a working implementation before Arc was even announced. But Arc seems to occupy the "succinct lisp" niche in minds of people so let's just go with the flow.

picoLisp already implements many things that Paul Graham was thinking about and what is most important is that it exists, is mature and can be used for writing real software, today.

A few nice things about picoLisp:

it's tiny
it's very succinct
built-in support for web applications
integrated object-oriented database, with direct bindings between (web) UI widgets and database objects
actively maintained and improved (around 4 new releases per year, latest release is from Dec 30 2007)
mostly cross-platform (Linux, Mac OS X, FreeBSD, Cygwin on Windows). Unfortunately it doesn't have native Windows version.
open-source (GPL)

The only thing it lacks is a huge fan base willing to wax poetic about how great it is. This is where you can help and show the world that we don't need Arc because we already have picoLisp.

Great Android talk

2008-01-12T07:31:41Z

Stanford has a great series of weekly talks related to computer science and of the recent talks was a talk given by Google engineers about Android platform.

It's a great intro to not only of Android is but also why it was created and sprinkles bits of history.

Too much oo

2008-01-11T05:41:59Z

Just found in an otherwise good code:

class DocDecompressor {

public:
    DocDecompressor() {}
    ~DocDecompressor() {}

    size_t decompress(ZLInputStream &stream, char *buffer, size_t compressedSize, size_t maxUncompressedSize);
};

Why is it a class and not just a function?

Rhetorical question, I know. But it illustrates what I've seen more than once: people get so enamored with some technique that they blindly use it even when it doesn't make any sense.

Rebol vs. Shoes

2008-01-09T08:09:28Z

There's no justice in the world.

Rebol 3 Alpha: 664 kB
Shoes (Ruby + UI framework): 25 MB
Capabilities: pretty similar

Ok, so Ruby probably does a little bit more, so it's a bit apples to oranges comparison, but it's one huge orange and one tiny apple.

Was there ever a better example of a gigantic difference that a thoughtful design can produce.

One data point: libcairo (which is a 2D graphic rendering library and is part of Shoes) comes in at 536 kB which is nearly the size of the whole Rebol system (which is a full blown language interpreter AND a 2D graphic rendering library with capabilities similar to libcairo AND a full blown UI framework with buttons, scrollbars and what not).

Thinkpad x60 - a year later

2008-01-08T07:19:07Z

Reviewing consumer electronics few days after buying it isn't very useful because we're in the hopeful mode and look at it through the lenses of potential and not real life experience.

I've had my Thinkpad x60 Tablet for 8 months so I'm past the thrill of novelty and can offer opinion that also incorporates annoyances that show up in real life usage.

The bottom line is: I highly recommend it (well, you can't get x60 anymore, but there's x61, an evolutionary upgrade).

You can google for all the boring tech specs, so I won't repeat them here. Suffice to say that with 2GB of RAM the laptop is plenty fast for real work (I'm a programmer so "real work" involves running Visual Studio, VMWare etc.) and Thinkpads are my favorite laptops (I've also used Dells, Fujitsu Lifebook and Sony Vaio).

I bought tablet version because I was excited about tablet features. Turns out I'm not using them much (pretty much only when reading comic books because flipped screen has better aspect ratio for that).

I'm still happy I did it, though, despite higher price over non-tablet version. The reason is that only tablet version has an option of sxga+ (1400x1050) screen resolution.

In notebooks I value small size but I also value high resolution screen and performance. I was afraid that 12" is too small for everyday work (my previous sweet spot for notebook size was 14" Thinkpad t42 with 1600x1200 screen). Turns out I was wrong: 12" with 1400x1050 screen is comfortable enough and I do most of my working on x60. I shelved my (still perfectly good) t42.

Rant: I hate all websites (including Lenovo's order pages) that use those SXGA, WXGA etc. acronyms and don't bother to include pixel count, something that I actually understand without googling.

Digression: being spoiled by those resolutions I would have a hard time buying Apple laptop. Their 13" MacBook has low screen resolution and their 15" MacBook Pro is too big for my taste.

At some point the LCD broke (half of the display was garbage) but Lenovo fixed it quickly (and at no charge since I was still covered by warranty).

After 6 months the battery pretty much died. It used to last 6 hours, now it can barely get to 2. Doesn't bother me that much, because where I use it I have power. Besides I think 6 months is standard for batteries so can't blame Lenovo for that. I could fix the problem by buying new batter.

Semi-rant: I hate all the Lenovo-provided pre-installed software. It's usually awfully designed, slowing down boot and sucking resources. I uninstalled most of it and saw no difference. Some of that crap I didn't uninstall (afraid of loosing some crucial functionality) and from time to time Lenovo's tvsched.exe goes for a wild ride and uses 100% of one of the CPU (which I notice because the computer slows down).

Vista is huge disappointment. I had an option to use XP but I chose Vista because it looked nice on the paper and I thought it has better tablet functionality. Today I would choose XP.

The most annoying aspect of Vista is that it's slow at some common basic tasks, like browsing folders which is such a fundamental problem that it begs the question: how could they fuck this up? Other annoyances are increased resource usage (they seem to have doubled number of processes running after boot) for no discernible value.

Security improvements are a good thing but not when they lead to breaking significant software (like Visual Studio 2005 or Visual Studio 6), lead to UI aggravations like UAC and try to kill third-party software ecosystem (the "are you sure you want to run this very untrusted application" message box).

At the same time some of the things that I care about and have annoyed me forever are not improved (like a dialog for editing environment variables or the fact that notepad doesn't understand Unix newlines).

The only improvement over XP I can see is a marginally nicer title bar. I still have XP on my desktop so I have a good comparison. XP wins over Vista by a large margin thanks to better performance and being less fussy and annoying.

A tip: RAM prices at Lenovo (and Apple and most other companies) are exorbitant. I bought my laptop with 1GB in 1 SIMM (important to leave the other slot free) and bought the other 1 GB at crucial.com. It's a good saving.

Logging in WinDBG

2008-01-07T01:57:52Z

Logging is often useful when debugging software to help understand its behavior.

Consider a problem I was having: Sumatra was crashing because of invalid refcounting of fz_shade object. Refcounting took place only in two functions: fz_keepshade() and fz_dropshade() so my plan of attack was to see who calls those functions and from that hopefully figure out why there is a mismatch.

If I had a logging system with ability to dump callstacks already built into the program, I could modify the program to add logging to those two functions, re-run and look at the logs.

But I didn't so I used WinDBG as an ad-hoc logger.

The trick is to know the following:

.logopen $filename WinDBG command

.logclose WinDBG command

conditional breakpoints

Under WinDBG I set the following two breakpoints:

bp SumatraPDF!fz_dropshade "kb; g"

bp SumatraPDF!fz_keepshade "kb; g"

bp creates a new breakpoint on entry to a given function. A string "kb; g" is composed of WinDBG commands executed when a breakpoint is hit. In this case I just dump the callstack with kb and continue execution with g.

Then I used .logopen $filename so that everything WinDBG prints in output window also gets written to a file and voila - an ad-hoc logging without the need to modify and recompile the program.

And yes, after some staring at resulting log I fixed the crash.