Raging Sloth Tech

Fixing Synology Photo Station 5


I must be honest, I suffer from a condition where minor imperfections drive me crazy. As such more major imperfections don't go over all that well either :) In general I really like my new Synology DS212j but even though I'd keep it around just for its file serving capabilities it drives me crazy that Photo Station 5 (very new as of typing this) which is part of DSM 4.0 just doesn't live up to my personal expectations. Don't get me wrong it does a lot of things right but there are things about it that could be a lot better and things about it that are absolutely horrible. To give you an idea of what I'm talking about I've made two lists.

Could be better
  • You should be able to set a per Album theme. Personally I'd like to have the background image for each Album automatically be the image I choose to be the album cover : Rating Minor
  • Theme options should include formatting options for the background image. I tried using a 10MP image from my DSLR and even though it caused my NAS to process for quite a long time it didn't end up a reasonable size and so it was terribly cropped after all that. I shouldn't have to mess around with resizing images to setup a background and css should be used to allow reasonable scaling: Rating Minor
  • There should be better control of default album permissions. Right now I can choose to have albums public or not public but I'd like to be able to set a default list of people who would have permissions to new albums. Rating Minor
  • I should be able to set access rights to full size images. As things stand you can turn access on or off to Photo Station but what if I don't want certain people to have access but I do for others? I can see this being a big deal for some people. I should be able to set this permission by user and album. Rating Medium
  • I should have way more control over what thumbnails exist and how they're used. Why is there a default 1280 thumbnail that doesn't seem to be used by the web version? I've read it exists for the iPad app, why not use the same one for web and iPads? Why can't I turn the iPad one off if I don't have one? Rating Medium

Absolutely horrible
  • Thumbnail generation using the NAS is horrifically mind numbingly slow Rating Terrible
  • Thumbnail generation using the Synology Assistant is way slower than it should be Rating Hair Pullingly Frustrating

I was told that thumbnails were this way by a friend who also owns a Synology NAS but I just didn't understand how bad it really was. Perhaps the worst thing is that people have been complaining about this for years… Or that it would be so easy for Synology to fix.

NAS Thumbnail generation


Situation


NAS Thumbnail generation is crazy crazy slow. I put up my current iPhoto library worth of jpegs and it was multiple day slow. Sure my NAS has the sort of processor you'd expect in a phone but things still don't have to be this slow. If you google around enough and log into your NAS (assuming you have a Synology NAS with DSM) using ssh you'll find out that in fact the thumbnail generation settings are quite intense even when quality is set to normal instead of High and there are hacks you can perform to make them generate a little faster. I'm personally not sure what happens if you delete the settings for one of the thumbnails entirely… I'll have to tinker with that when I have a chance. Also there is a method to delay thumbnail generation so your NAS isn't pegged at 100% CPU usage while you're trying to use it. This delay is not implemented well, however. You can only turn the delay on when thumbnails are being generated so you can't for example wake up in the morning and just set the delay until the night moving photos around and having it clean up after you when you sleep. Also you are only able to set a delay and everything goes back to 100% thumbnail generation once it is over.

Solution


Have a quality slider for thumbnails instead of High and Normal. It really wouldn't take much to implement would it? After all this time? Also instead of the click this button for a temporary delay how about a simple timeframe setting like for example only generate thumbnails from midnight until 5AM? or only on Mondays etc.? A little user interface change and some additional thumbnail profiles would go a really long way. I know the thumbnails that are generated are really high quality but in the lower res thumbnails who cares? Who looks to the 320x240 pixel thumbnail for high quality? It's just there to identify the photo. At the higher resolutions the unsharp mask that takes so long doesn't really make that much difference. After all we're used to seeing high resolution images shrunk down without unsharp mask on our monitors anyway… If someone wants to see the super high quality image they can view the full size as it came off the camera and potentially out of post-processing. They can also achieve here quality thumbnails with the Synology Assistant. When a video is uploaded to the NAS it is converted to flv instead of mp4 why can't the image thumbnails be lower quality too? Another good idea would be to have the 800x600 thumbnails generated for each image 1st. Then while the remainder of the thumbnails are being generated it can be used in the Photo Station gallery so the pictures are more quickly available on the site.

Synology Assistant Thumbnail generation


Situation


The Synology Assistant is much faster than just dropping files into your photo partition and having the NAS create thumbnails. That being said it is still crazy crazy crazy slow for what it is doing. There are a few reasons for this:
  • The conversion process is done using a separate executable, "convert" which means:
    • Every time a thumbnail is generated a new process is created and needs to load convert into memory (though it is probably cached)
    • Every time a thumbnail is generated the original file has to be read from disk again (it might be cached but still)
  • Convert is called for each thumbnail synchronously (i.e. it is not run in parallel) so if you have a dual or quad core machine (which most people probably do) all but one of your processors is not being utilized.
  • Unsharp mask is run on all thumbnails which is relatively intensive
  • Files are converted to thumbnails and then uploaded. The processor sits basically idle while files are being uploaded.
The first of these points however is an opportunity. Since a separate executable is used to actually process files it isn't all that difficult to simply replace it with our own. I experimented with this a bit. At first I would just intercept the conversion calls and run my own multi-processing to utilize all my CPU cores to generate 4 thumbnails whenever convert was called for the first thumbnail. This still suffered from being pretty slow. Then I changed the script to ignore unsharp mask. The truth of the matter is that if I want unsharp mask I'll run it on my original and I think people are used to having thumbnails that aren't super high quality (assuming that unsharp mask doesn't make a particular thumbnail look worse anyway) In doing so I switched the script over to using the Python Imaging Library which is quite fast and a lot easier to get installed on various OSes than ImageMagick. Still I didn't like how long things took. With the approach of intercepting convert calls I was still stuck with an idle CPU most of the time. Also since the calls were responding to calls to convert and multi-process (4 worker processes and a manager, in this case on a quad core) I was creating 5 identical processes every time convert was called. The solution to this was to break the script in two. I wrote a script that runs through a directory structure and generates thumbnails for all the files within. After that I have a separate convert program that simply copies these existing thumbnails into the location where Synology Assistant expects them to be (I plan on changing this to use hard links). This way my CPU runs maxed out until all the thumbnails are created and then Synology Assistant is responsible only for moving all the files to the NAS in a way that the NAS expects and doesn't cause it to dump the thumbnails and start over (in my experiments in moving the files over myself this constantly happened…) I really want to clean this up and have my own code move the files over (I could run two threads for each CPU and have items uploading at the same time they are being generated and abandon the need for temporary files) It looks like Phillips321 has found a way to do this (off the top of my head I think it might be that since he is generating the thumbnails on the NAS itself they probably have creation dates later than mine? I am concerned however that this actually only works on Linux as on my Mac I cannot cd into the thumbnail directories while I can in linux for whatever reason.) Anyway I've been really busy at work lately (and my amateur hockey team is in the playoffs and I'm trying to learn Stairway to Heaven on guitar…) so I still haven't tested my scripts on windows nor written up a good how-to on using them so please be patient :). For some users the Synology Assistant is probably sufficient but importing my initial library of >3500 files was going to take weeks and last hike I went on I took around 500 pictures… I'm going to e-mail Synology and ask them how the assistant moves thumbnails over without them being dumped by the NAS (I'd like it to work on Windows Mac and Linux) and if they respond make a nice cross-platform GUI app to make it simple for everyone. In the meantime I do plan on putting the scripts I have out there (with my scripts on my 2.4GHz core 2 quad the multi week generation took about 30 minutes to generate thumbnails and maybe 45-60 minutes to upload) but I need some time to test on windows and write up a good how-to first. Failing getting the info I need from Synology I might just write my own Photo Station competitor. What does everyone reading this think of Raging Photo Station? Play-offs are over in a week but work is going to be really rough till about June but please do check back. I might even find time to proofread this post :)