Some bugs are more annoying than others. Some affect a small group of users and some others affect mostly everyone. But one of the worse bugs an app can have is those that involve data loss. And our latest version 2.9.4 contains one of those.
Here’s what happened:
For quite a long time now, we’ve been working on rebuilding the data storage foundations of Panels. We are aiming to open the app sandbox and unlock access to the files you added to your library. This requires quite a lot of changes. And for the last 9 months, we’ve been making those changes under the hood, using the most rigorous testing practices to ensure it doesn’t affect the normal operation of the app.
Last week we made a necessary change and we screwed up. By mistake, we moved the
system folder into the caches container.
iOS sandbox gives every app access to a small portion of the mass storage drive. To keep the explanation simple, iOS expects you to use one of 4 different folders inside the sandbox (there are actually more).
- The Temporary folder: meant to be used for temporary information.
- The Library folder: meant to be used to store information generated by the app.
- The Library/Caches folder: meant to be used to store information that can be re-generated.
- The Documents folder: meant to be used to store user documents and user-generated data.
The information stored in those folders has a different lifespan. iOS can delete the files in the temporary folder at any time, but never when the app is open. The content in the caches folder will much longer than the temporary folder, but the system can decide to delete it if the device is running low in disk space.
system folder that contains, amongst other things, the app database, was stored in the Documents folder. For no particular reason.
But now that we are planning to open the sandbox container and users will have access to it, we needed to move the
system folder to another place, to prevent users from accidentally deleting their database.
The plan was to move the
system folder from
Library/Application Support/system. But accidentally we moved it into
If you install 2.9.4 and you are running low on disk space, chances are that iOS will prune the folder and delete the database, containing meta-information like manual sorting, collections, and local copies of the reading sessions.
No. It depends on how frequent/infrequent they use their app or the amount of free space available on disk.
This morning we released 2.9.5 that solves this issue for new users and mitigates the data loss for affected users.
If your library appears empty after installing 2.9.4, I am afraid that your collections and manual sorting have been lost. If you have a Panels account, your reading session get automatically synchronized, so that won’t be lost.
2.9.5 restores the Library by scanning the comics folder and rebuilding the index. Unfortunately, the comics folder does not contain information about collections. The content will re-appear in the root of the library but, unfortunately, collections are impossible to be restored.
Yes. We’ve fixed the code but also added a set of tests to ensure the system folder is protected in the correct container.
We are also planning to add additional measures to back your database up periodically so, in case of an improbable regression, we can have a backup.
Also, the imminent switch to the open sandbox approach means that the collections are represented with folders, allowing us to re-build the whole library at any time.
Since some time ago, we make our releases using phased rollout. Meaning that a release is not made available to 100% of users at once but progressively. Right after we know something wrong was happening we paused the rollout to minimize the impact.
According to our metrics, 22% of our users have 2.9.4 installed at the moment of writing this. It might grow up a bit in the upcoming hours, but all new users will download 2.9.5, and users that haven’t upgraded yet will skip it and upgrade to 2.9.5 directly.
If you have 2.9.4 installed and you haven’t lost any data, please upgrade to 2.9.5 ASAP.
- Nov 17, 2021 at 12:18 PM - 2.9.4 is released on the AppStore
- Nov 17, 2021 at 8:51 PM - We receive the first report from a user
- Nov 18, 2021 at 2:23 PM - We acknowledge the report and pause the release and start investigating the issue.
- We keep testing and trying to reproduce the problem without any luck. The release is still paused.
- Nov 21, 2021 at 3:48 PM - We receive another report by email asking if it is possible that running short in disk space could cause it. This hint gave us the idea and pointed us in the right direction.
- We investigate the problem, found the error, and start working on a fix.
- Nov 22, 2021 at 1:06 AM - We release a new Testflight (2.9.5 - 202111220051) including the fix and start testing it.
- Nov 22, 2021 at 8:00 AM - In the morning I realized that the fix solves the problems for non-affected users but leaves the affected ones without a good solution. I start working on recovering the Library using the files in the
- Nov 22, 2021 at 10:45 AM - We release a new Testflight (2.9.5 - 202111221030) with the additional logic to try to restore as much information as possible.
- Nov 22, 2021 at 10:51 AM - We send 2.9.5 - 202111221030 for AppStore Review.
- Nov 22, 2021 at 11:49 AM - Apple Starts reviewing the app
- Nov 22, 2021 at 12:54 PM - 2.9.5 - 202111221030 Is approved
- Nov 22, 2021 at 12:55 PM - The app is released to the App Store.
We know this post does not solve anything for those who’ve lost their work organizing their Library. But
hopefully will explain what happened and what we’ve done to prevent this from happening again.
Our sincere apologies for the error.
If you have been affected and lost some information to this bug, please contact us. We’ll try to find a way to make up to you.