UMS very slow in handling playlists (.PLS)

[JDN165]

There is a folder on my network where several playlists (.PLS) exist, six to be precise.
These playlists are generated by a tool and each playlist contains 100 random songs from a music library on an external hard drive, each playlist contains songs from one music genre.

This network folder containing these playlists has been added to the shared content in UMS.
Well, so far so good.

The problem I have is that when I select UMS on my music player, and then select the playlists folder, it takes a very - very long time before I see the containing playlists.
I've examined the log file to see what happens during this period, and I noticed many of these log lines when the playlist folder is selected:

DEBUG PLS item added: [M:\Music\E\Erik Wøllo & Bernhard Wöstheinrich\Weltenuhr\05. Echoes of salons.flac,05. Echoes Of Parlors.flac]
DEBUG Final Verdict: "05. Echoes Of Parlors.flac" is streaming
DEBUG Add PLS entry: [M:\Music\T\Tangerine Dream\220 Volt Live\05 - Sundance Kid.flac,05 - Sundance Kid.flac]
DEBUG Final Verdict: “05 – Sundance Kid.flac” is streaming
DEBUG HikariPool-2 - Pool Stats (total=10, active=0, inactive=10, waiting=0)
.
.
etc etc.

This process continues for ALL entries in ALL playlist files and takes more than a minute. Each song in a playlist has 2 (or sometimes 3) log lines.
In between processing the playlist files, there are also many of these lines in the log:

DEBUG GlobalIdRepo: flag id 6627 out of range

and a lot of these lines:

DEBUG removes invalid ID 6605

My question: Can I skip this processing and evaluating whether a music file can be streamed or not? Since I know all my files can be streamed, this process is pretty useless and just slows down the way I can select the playlist on the player. Is there a setting or can I speed up this process in some other way?

[mik_s]

There is not enough information when only looking at DEBUG messages. If you put the logging in TRACE mode it will give a lot more info on what it is doing.

I don't know anything about how playlists work so can't be of much help but might be able to spot something looking at trace logs.

[JDN165]

Ok, here is my log.

I've created a small playlist for this test with only 5 entries (name: Test_playlist.pls) and I deleted all other playlists in the playlist folder. So, selecting the playlist folder in UMS only resulted in these 5 entries. With only these 5 entries there isn't hardly any delay of course, but when there are several big playlist files in this folder there's a serious delay time since UMS is checking all the files in all the playlists, obviously whether they can be streamed or not. Which for me is pretty useless because I know all my files can be streamed, so don't let me wait for more than a minute when I select the playlist folder in UMS

[mik_s]

It is because you have disabled the cache so UMS has to scan each file when you load a playlist instead of doing this once and storing it in the database.

This is what it is set as in your UMS.conf

Code: Select all

# Enable the cache
# ----------------
# WARNING: Disabling this will also disable the fully played status feature.
# When enabled, it makes all browsing significantly faster and enables features
# like played status tracking.
# Default: true
use_cache =false

There is an enable cache option in the settings that should be turned on by default.
As UMS relies on the database more and more these past few versions that option is going to be removed as there is no benefit to disabling it.

If you turn the cache back on you will se a massive speed boost when browsing your files.

[JDN165]

I indeed disabled the cache some time ago because I experienced a lot of trouble with an enabled cache.
Perhaps these troubles are over in the recent versions, so I could enable it. I will check the result...

Thanks for the suggestion.

[JDN165]

Well, I've done several tests now with and without cache enabled, and I can conclude that this setting unfortunately has no effect when using playlists.

With the cache enabled, I still have to wait almost a minute for my five playlists (and by that I mean just the name of the playlist and not even the content) to appear on my DLNA player.
It is just the selecting of the folder and the selecting of the playlist that is very slow, once the tracks are playing there's no problem.
In the logs I still see, when I select the playlist folder, that all the songs from each playlist are "evaluated" whether the track can be streamed or not, a very time consuming (and as I explained: for me useless) process, also with the cache enabled.

So unfortunately I'm afraid that I will have to accept that working with my playlist (selecting the folder and selecting a playlist) will be and remain a slow process...?

[mik_s]

It should only be slow the first time until the files are scanned when the cache is enabled.

As you have not had the cache on for a while there may have been changes made to the database structure over different versions of UMS and those changes have never been applied as it was disabled.

I had another look at your logs and did see the error

Code: Select all

com.zaxxer.hikari.pool.HikariPool HikariPool-1 - Exception during pool initialization.
org.h2.jdbc.JdbcSQLNonTransientException: General error: "org.h2.mvstore.MVStoreException: Unable to read the page at position 66520520726526, chunk 242, offset 1050719 [2.1.214/6]" [50000-214]

Hikari is something to do with the database, and this will be where the invalid ID messages are coming from

You mentioned you had problems with the cache before and that was why you turned it off. This may be related to that. It could be that the database was corrupted at some point.

You could try deleting the database to it will be recreated when UMS is restarted. It is located in "C:\ProgramData\UMS\database", then run a scan of all your files at startup. This might take a while for the first time but should take seconds on next restart.

If there is no improvement try doing some more logs and I'll see if I can see anything else.

[JDN165]

Ok, I will try this.
Should I delete all three files in C:\ProgramData\UMS\database ?

And I guess the rescan happens when I restart UMS (and "Scan shared folders on startup" is checked) ?
This rescan also is applicable to the contents of the playlist files ?

[mik_s]

Yes best to just delete the whole database folder.
You will need to recreate an admin login when you next open web settings, or you can turn off authentication if you are not bothered about it.

Any file in your shared folders will be scanned so it does not matter if it is in a playlist or not.
The scan will happen on startup (when that option is enabled) or you can do a manual scan by selection that option in the shared folders section.
The list in the PLS file just points to exiting files in your shared folders which should already be scanned. If it points to files not in shared folder then I don't think they will be included.

If you open the old GUI (right click on the tray icon and chose settings (old)) you should be able to see the scanning progress in the status bar.

[JDN165]

Ok, I've deleted the database files and now they have been recreated by UMS.
Seems to me browsing the shared folders is a bit faster. However, I don't observe any increase in speed with the playlist folders yet, but I will keep an eye on it.

Is this database also live updated when I delete and move files between the shared folders, because this happens a lot.
When I start UMS there's a rescan of the folders (in the settings dialog), but when I remove of relocate files while UMS is running I don't see anything happen.

Can these database files be read in some kind of viewer, so I can check the content of the database? Are they in a common database format?
Oh, found already an answer to this last question (viewtopic.php?t=14160)