I released first version of
SumatraPDF in 2006. That’s 15 years ago which seems like a good time for a retrospective.
The app
SumatraPDF is a multi-format (PDF, ePub, Mobi, comic book, DjVu, XPS, CHM) viewer for Windows and currently looks like this:
The code
SumatraPDF is an open-source document reader for Windows. It started as a PDF reader, hence the name. Over time I’ve added for e-book formats (epub, mobi), comic books (cbz, cbr), DjVu, XPS, image formats etc.
It’s about 127k lines of C++ (not counting libraries written by others).
It’s written against Win32 API, not using GUI abstraction libraries like Qt. This contributes to making it as small and fast as possible.
Almost all of it was written by 2 people, with occasional contributions from others.
The amount of code written is actually higher. It is the nature of long running code bases that the code gets written and re-written. We delete, add, change.
It’s a side project, done after hours, not a full time effort. How does a daily grind of working on an app looks like?
It looks like this:
You can also take a peek at
my dev log. I’ve only started it a year ago so only covers 1 year out of 15.
Why I created SumatraPDF
SumatraPDF is what I call an accidental success.
I never wanted to write a PDF reader for Windows.
In 2006 I was working at Palm and one of my job duties was writing a PDF reader for
Foleo, an ARM and Linux powered mini laptop. You never heard of Foleo because it was cancelled weeks before launch for reasons I’m not privy to.
At the time I didn’t know that PDF is popular but Palm management did which is why they decided that PDF reader is a must have application. I ended up being the (sole) dev on the project.
Writing a PDF rendering library is a multi-year effort. We didn’t have years so I used
Poppler open-source library.
My job was to write a basic PDF viewer that used Poppler to render PDF pages into a bitmap in memory and blit those bitmap on screen.
PDF is a complex format and rendering of some PDFs is slow. I wanted to improve the speed because Jeff Bezos told me that speed is something that customers will always care about.
Accidental app
The way to improve speed is to profile the code and look at the result.
Unfortunately, the toolchain for unreleased ARM hardware wasn’t very good. Forget about a profiler, kid, be grateful you have a C++ compiler and don’t have to enter assembly by typing hex, like Steve Wozniak.
Windows had decent profilers, so I compiled Poppler for Windows.
Once I had the library working on Windows, I wrote simplest GUI app that would show the pages and allow navigating between pages.
What do you know: I had a simple PDF reader for Windows.
I released it on my website. It couldn’t do much so I tagged it as version 0.1.
If you’re not embarrassed by your app then you’ve waited too long to release it
I didn’t come up with this nugget of wisdom but I agree with it.
Getting early users, learning what features they want the most beats toiling for months or years and implementing lots of features before you know anyone even cares.
Back to profiling: my plan worked.
I profiled the documents that took the longest to render and made a few surprisingly simple and surprisingly effective optimizations.
If memory servers, 2 optimizations had the biggest effect:
- optimizing string class to use what is know as “small string optimization” i.e. adding a small buffer inside string class to hold small strings inline (as opposed to always allocating memory for the string). Strings were used frequently and most of them were small
- fixing byte-at-a-time i/o by converting it to bulk reads. The way the code was structured in some code-paths it would do a virtual C++ call and a call to C read() function for each byte. Those are extremely cheap but not when you do it 5 million times
As a good boy I did submit my changes to Poppler.
As is my experience with contributing to open source projects, it was more of a miss than a hit.
Yes, I got 13 commits in but the project wasn’t very active and the maintainers weren’t eager to accept anything beyond small changes. Forget any major refactors.
I’m not one to voluntarily bash my head against the wall so I stopped trying.
(As you can see, I’m a fantastic team player).
Code quality
I want it and you should want it to.
How to maintain high code quality while working mostly solo, with no-one doing code reviews, no dedicated QA team?
Here’s how:
- test the code yourself. Step through newly added code in the debugger, verify the newly added functionality works as expected and in general use the app a lot
- automated crash reporting. Unfortunately it’s a pain to build but this is single most important thing you can do to improve quality of your software. Briefly: setup exception handlers to catch crashes in the app, in crash handler download symbols from the server to get readable callstack, create a crash report that includes callstacks of all thread, program and os information, log and submit that to a server. On the server, process those files and generate web pages for easy viewing of the crashes. Like I said: it’s a pain to build. Once you have crashes, look at them occasionally and try to figure out what went wrong and fix it
assert(). asserts are well established practice in C++ code: an additional code only executed in debug builds that verifies some conditions are true. If they’re not, something went wrong and you should investigate. I wrote wrote my own assert-like function which I enable in non-debug pre-release builds so that I automatically get bug reports from people hitting those conditions. Trust me: there’s no amount of testing you can do yourself that would match all the different things that a thousand people will do just by using the app.- logging. When investigating issues it helps to know what sequence of events led to a crash. My tiny logging module logs to a block of memory. That gets sent along with crash report. I also have an option to log to a file and I’ve recently added logging to a separate logging app via named pipe. This is perfect because most of the time I don’t care about the logs but when I do, I don’t want to restart the app to enable logging. With separate logging app, SumatraPDF is logging all the time and when it detects that logging app is running, it’ll also log to it. Implementation was trivial: logging app creates a named pipe, logger opens the pipe (like a file) and if open succeeds, it means the logger app is running and it reads the logs we write to the pipe
- static code analysis: max level of warnings in C++ compiler, make warnings into errors, Visual Studio’s `/analyze’ option, cppcheck, clang-tidy, GitHub’s CodeQL. Run those occasionally and fix the errors and warnings
- ASAN (Address Sanitizer), is fantastic. Was added in some point release of Visual Studio 2019. At a very small performance cost it can detect if you over-write memory or try to read uninitialized memory. I have a configuration with ASAN enabled. It’s fast enough to be used as a regular build.
- stress testing. Sumatra’s job is mostly to render complex document format. There often are crashes in specific files due to complexity of the formats. To ensure lack of crashes I wrote a stress test code that reads and renders all files in a directory. I typically run it before a release on a large collection of test files I amassed over the years
- unit testing. I don’t have a lot of them, they’re mostly for testing edge cases for low-level functionality like string formatting. They occasionally find bugs.
- memory leaks. It’s surprisingly hard to find an easy to use memory leak detection tool. I’m working on a very simple built-in leak detector. In the meantime I’m using Dr. Memory. It works but it’s super slow.
Frequent releases
When you don’t have many features, improving the app is fast and easy. It doesn’t take much effort to implement “Go to” dialog (implemented in v 0.2).
On one hand I don’t want to release too often but I also do want the users to get new features as quickly as possible.
My policy of new releases is: release when there’s at least one notable, user-visible improvement.
Web apps take it to the extreme (some companies deploy to production multiple times a day).
In desktop software it’s a bit more involved and I had to build functionality to make it easy i.e. add a check for new releases, write an installer that can update the program.
BTW: I mean “frequent in proportion to amount of new code written”. SumatraPDF releases are not frequent in absolute terms but frequent if you consider that it’s a part-time, after hours project.
Treat open source projects like commercial software
Majority of open source projects probably don’t fall into this category, but if you want your open source to be as successful as possible, act as if it was a commercial product from a software company.
What does it mean in practice?
From day one I created a website for the app. It had screenshots, it had documentation, it was easy to download and install. Granted, a kind soul on Reddit called it “a website made by a 6-year old”. The lesson here is two-fold:
- ignore haters and assholes
- a website built by a 6-year old is better than no website. It doesn’t have to be pretty, it has to be functional
I did basic SEO. Nothing beyond Google’s “SEO 101” docs: just pay attention to URLs, put the right meta-data, use the right keywords.
I had a forum for users to ask questions, submit feature requests and occasionally support each other.
I made the installation process as easy as possible.
Everything that is a good idea for promoting commercial software is also a good idea for open source project.
Switching the engine while the car is running
At some point I decided to switch from Poppler to
mupdf because mupdf was better and actively maintained.
Changing the app to use completely different library is not something you can do in an afternoon.
It’s demoralizing to work long time on code that doesn’t even compile.
To keep things compiling while also working towards supporting alternative rendering engine I developed an abstraction for the rendering engine.
The engine would provide the functionality the UI needed: getting number of pages in the document, sizes of each page (to calculate layout), rendering a page as a bitmap etc.
I’m much less enthusiastic about abstractions than most programmers (at least those who like to opine on Hacker News) but in this case it served me well.
I was able to incrementally convert program form using Poppler API to using Poppler via engine abstraction to using mupdf via Engine abstraction.
For a while I supported both engines at the same time but eventually I switched to just mupdf, to keep the app small.
This opened the door for supporting other formats via the same abstraction.
Simplicity vs. customizability
Simplicity sells.
I learned that from the history of Mozilla Firefox.
Before Firefox there was Netscape Navigator. It was a beast of an app, combining web browser with e-mail client.
Netscape couldn’t help themselves and was adding features upon features, leading to very complex UI.
A small group of renegades within Mozilla forked the code and focused on simple UI.
Simple Firefox was much more popular than the complex Navigator and eventually ate it completely.
From the beginning my goal was to keep the UI of SumatraPDF as simple as possible. An 80⁄20 app: 80% of functionality with 20% of the UI.
This requires resolve. I constantly get requests to add more icons to the toolbar and I constantly have to say “no” because adding 2 more icons to the toolbar to satisfy 10% of users makes the app slightly worse for 100% of the users.
Another trap is a siren song of additional settings. Sometimes people suggest that instead of doing X, the program should do Y. Not willing to remove X, they suggest adding a new UI setting “[ ] Do Y instead of X”.
Having settings dialog with 100 settings is not a good solution. It makes the app worse for everyone due to overwhelming them with choices and hiding important options in a sea of non-important options.
Not to mention that every conditional behavior requires more code, more potential bugs and more testing.
That being said, I also believe customizability is important. I believe that a big reason for Winamp being such a dominant music player (at the time) was its ability to skin the whole UI.
Some advanced features might only be used by 20% of users but those users are most likely power users that will evangelize the app more than the other 80% of the users.
My solution to UI simplicity vs. customizability: advanced settings file.
I designed a simple, human readable (and human writeable) textual format for
advanced settings. Think JSON, but better.
I didn’t bother to write UI for changing those advanced settings. I just launch notepad.exe with the file. When user changes the settings and saves the file, I reload it and apply the changes.
Be water, my friend
Change is the only constant. We must adapt to the changes in the world.
I can’t believe how many popular projects still use craptastic Sourceforge for source repository or mailing list.
Actually, I can believe: changing things takes effort and the path of least resistance is to do nothing.
I started with Sourceforge, switched to code.google.com and then to github.com.
I switched forum software three times.
I’ve added a browser plugin and then removed it when browsers stopped supporting such plugins.
I changed the format for storing preferences from binary to human readable text.
Windows XP went from being the OS used by majority of users to no longer being supported (long after Microsoft stopped supporting it).
At first I only had 32-bit build and now I have both but emphasize 64-bit builds.
Think outside of the box
Thinking outside of the box is hard because the box is invisible.
SumatraPDF wasn’t the first PDF reader application ever written.
But most PDF readers do not become multi-format readers.
In hindsight it’s an obvious idea to support as many document formats as possible but it took me 5 years to realize it.
Most readers are still single format and I do believe being multi-format helped SumatraPDF become popular.
I can’t say it’s totally unique idea. There were multi-format image viewers long before SumatraPDF and I probably was inspired by them.
Small and fast - pick both
By today’s standards SumatraPDF is tiny (installer smaller than 10 MB) and starts up instantly.
I believe being small and seemingly fast was a big reason for adoption.
This comes back to Jeff Bezos’ wisdom: there will never be a time when users want bloated and slow apps so being small and fast is a permanent advantage.
How do I keep SumatraPDF small?
I avoid unnecessary abstractions. Window’s system of controls is a giant pain in the ass to program against. I could use wrappers like Qt, WxWindows or Gtk. They are easier to use but cause instant, giant bloat.
I’m not afraid to write my own implementation of things. I have my own JSON, HTML / XML parsers that are a fraction of size of the popular libraries for those tasks.
I aggressively take advantage of rich functionality included in Windows.
Let’s say I need to do a network request. I could include a monster library like curl or I could write 300 lines of code using win32 APIs. I wrote 300 lines of code.
An absence of bloat is hard to notice because it isn’t there.
My pet peeve is over-using XML for storing data.
When I worked at Palm I was at a design meeting for auto-update system for a phone. Part of it was storing information about the current version in the image, downloading information about the latest version and comparing them.
The developer decided to use XML for storing that information. That seemed like a lot of bloat for storing simple information like a version number. An compliant XML parser alone is a lot of code. Surely a simple binary format would be easier to implement, I suggested and was ignored.
If you don’t have the power to fire someone, your ideas will be ignored.
(As you can see, I’m a great team player.)
For storing advanced settings I designed and implemented a file format that is smaller than XML, readable and writeable by humans and can be implemented in few hundred lines of code. It’s as powerful as JSON and even more readable.
It’s so simple that after implementing it I had the time to implement a serialization system for C++ objects and a Go code generator. To add more settings I don’t have to write more C++ code. I just add data definition to Go generator, re-run it and get data-driven C++ parsing auto-generated.
It’s my project and I act like it
When someone pays you to write code you have to do it the way they like it.
A big attraction of working on code you’re not paid for is that there is no one who can tell you what to do or how to do it.
My code would not pass a code review at Google and not because it’s bad but because it’s often unorthodox. Outside of accepted dogma.
(As you can see, I’m a great team player.)
I always used SumatraPDF as my playground for testing crazy ideas.
Minimize the code size by not using STL? That’s crazy but I did it. Granted, in 2006 STL wasn’t very good.
I learned about how Plan 9 C code had non-traditional scheme of #include files where they don’t put #ifdef wrappers in each .h file to allow multiple inclusion and .h files don’t include other .h files. As a result .c files have to include every .h file they need and in the right order. It’s a bit of a pain and no other modern C++ codebase I know of maintains such discipline.
But it’s my project so I did it and I keep doing it. It prevents circular dependencies between .h files and doesn’t inflate C++ build times because of careless including the same files over and over again.
I implemented a CSS inspired UI system. Not great, but mine. And I plan to replace with a different one.
Because I can.
Because no one can tell me not to.
Supporting other platforms (Linux, Mac, Android) is one of the most frequent requests. A request that I have to decline.
First, there is a pragmatic reason: I just don’t have the bandwidth to write code for 3 platforms.
Second, I believe an excellent app for one platform can become more popular than a mediocre app for 3 platforms.
Coming back to the first reason: I don’t have the bandwidth to write 3 excellent apps. Part of the reason SumatraPDF is small is my use of win32 APIs for the UI.
The only way for one person to even attempt cross-platform app is to use a UI abstraction layer like Qt, WxWidgets or Gtk.
The problem is that Gtk is ugly, Qt is extremely bloated and WxWidgets barely works.
Tests are not necessary, neither are code reviews
I’m not saying tests are bad or that you shouldn’t write test or do code reviews.
I’m saying that they are not necessary.
Dogma is powerful. Sometimes in my corporate life I felt like writing tests was just going through motion. Maybe we should spend more time writing code instead, I though?
But try to make a nuanced point about more tests vs. more code to your fellow developers and you’ll be burned at stake and your smoldering carcass will be thrown to wild dogs. Village children will use your severed head to play soccer.
(As you can see, I’m a great team player.)
And yet I do know that you can write complex, relatively bug free code without tests, because I did it.
I do know that you can write complex, relatively bug free code without anyone looking over your code, because I did it.
If no one uses your app then who cares if it crashes.
If many people use your app and it crashes, they’ll tell you and then you’ll fix it.
Overnight success takes a decade
SumatraPDF is relatively popular. Not Facebook popular or DOOM popular, but more popular than most apps. A respectable level of popular.
It all started with v 0.1 and a trickle of downloads. It remained a trickle for many, many months.
I’m not sure there’s a lesson here.
Success often takes a long time.
Unfortunately, at that stage it’s undistinguishable from (eventual) failure so this wisdom doesn’t help you if you’re working on a not-yet-successful project and debating if you should continue or abandon
The money
Open source is not a good business model.
If you want to make money do literally anything else: try to sell software, do consulting, build a SAAS and charge monthly for it, rob a bank.
I did experiment with making money and made some.
There was a time AdSense would pay decent CPM so I put AdSense ads on the website and it made some money. I no longer do because the rates did plummet and it isn’t worth annoying people. My soul has a price and AdSense can no longer afford it.
Now I’m experimenting with Patreon and Paypal donations. It makes more than $100 a month but not much more than that.
Like I said: don’t start open source project with intent to make money.
Rarely you can have both: freedom to do whatever you want and a good pay so pick what is more important to you. Open source gives you freedom but not money.
On to the future
I need to continue being like water.
For years I resisted adding editing features. “It’s just a reader” I said. But why not add editing? If people want it, give it to them.
The future of all software is as a web app. Why not bring the spirit of SumatraPDF to the web?
Those are just a few ideas I have today.
Being like water means that in 5 years I’ll have other ideas, informed by what’s happening at that time.
