This is a dev diary of implementing Filerion, a web-based file manager for online storage (Dropbox, OneDrive, Google Drive, s3 and more).
Today:
implemented upload to S3. This is a second attempt. The first one was to use pre-signed URLs generated in the backend and upload directly from the browser. That didn’t work because of CORS restrictions. This time I’m just routing the upload through my server.
note for the future: maybe it’ll be possible to use Cloudflare workers to do those uploads more efficiently and more cheaply. More cheaply because in Cloudflare one doesn’t pay for the bandwidth. Faster because the request might get handled by a Cloudflare server closer to the user.
lesson: it’s important to consider future optimizations / improvements to the code, but it’s also important to make progress. The current upload code works. Optimization can wait until we know that a product is successful and used by many people
implemented pre-caching of directory listings. I want Filerion to be really fast when navigating files. For most filesystems, reading a directory requires a network call, which is slow. Solution: read file entries in advance and cache them for really fast navigation. I’ve put a limit of 256 directories. Maybe should also put a limit on number of files (to prevent issues with very, very, very large directories.
implemented deleting of directories in s3
s3 doesn’t really have a notion of directory. It’s just key / value store. By convention, if key has “/” in the name, we think of it as creating a directory. s3 allows listing of files by prefix, so to delete a directory foo/, we first need to get all keys that start with foo/ and delete them
design question: deleting directories is dangerous. A mistake will delete lots of files the user didn’t mean to delete. Should I try to prevent mistakes? If yes, how? People become blind to confirmation dialogs. Maybe a more difficult confirmation dialog, like GitHub’s when deleting a repository i.e. ask user to enter a string to confirm deletion of directories. For now I’ll just add a warning in read when deleting directories:
implemented auto-sizing columns in list view. Before a long file name would make the table bigger than the panel and require scrolling. I’ve added code to take advantage of the fact that the size column is small so we resize name column to be
- . This will be more complicated in the future i.e. I’ll need to provide manual resizing of columns, there will be more columns etc. For now it’ll do
svelte technote: it’s implemented with an action that remember the size of each node as it’s mounted into DOM and in afterUpdate I calculate the size of name nodes and change the size by setting max-width property on each node
