On the Fly Database

Hi there,

Still investigating on mAirList and now I’m stuck with OTF Database data collection. :slight_smile:

I’m desperately trying to get the on-the-fly database working but nope, I cannot get past the splash screen.
I have reduced now the amount of files from 50.000 to some 25.000 but still the data collection does not finish.

Hmm, is there a limitation on this?
I have written my own program that collects all musicfile data such as path, artist, title and length into a delimited file and it uses recursive call of the data collection function to access subfolders. My program completes in more or less an hour to fetch all the data from these 50.000 files.

Q: Is there anybody in the community who uses OTF database?

I believe that such a function is crucial for the Liveshows, otherwise it will be impossible to find back a songtitle…
OK, i can still rely on my own program that collects the data and that also has a search function included. But pasting the data from the clipboard to mAirList is a p.i.t.a.

enjoy the music:
-Serge-

Do you have auto-cue enabled in mAirListConfig? This is what makes the import slow, because every file is opened and scanned for the cue points. Takes a second or so for each file.

I agree that the OTF database is no fun with that amount of files. Especially as the scanning takes place during startup while the GUI freezes. It’s not meant as a “real” database anyway, merely as a temporary solution until mAirListDB is ready (which will soon be the case).

How does your application store the data? Perhaps you could write a database script to access your application’s data from mAirList.

Hi Torben,

“auto-cue” ? please point me to the right direction. I can’t find that entry in the config. Sorry I’m too new to mAirList…

Well the approach I used was quite simple. I store the data simply in a delimited textfile.
The point was that we use categories for our rotationlists. Therefore it’ is possible that a file could reside under Rock, Pop, 80s, … or in more than one folder (diskspace is not a subject nowadays)
There are some rules to follow when using my program as when scanning on filename is requested, the split is done on " - " for artist and songtitle.
But all our files are renamed from the tags using this rule. There’s also a check-function included that scans the tags/filenames for invalid characters that can be specified in the ini-file. I consider invalid characters as those that cannot be displayed by the RDS decoders such as . , $ & ä ö ü etc…

Now that the mlp format is fixed, I will implement a mAirList playlist creator function to it.
The benefit is that only by distributing the delimited file (hosting on a periodic basis) all our DJs would be able to create playlists offline and without having to replicate the diskstructure of the production machine.
If someone wants a copy of it, no problem, just let me know. I’ll finish the playlst creation stuff probably by end of next week.

Here an example of the delimited filestructure
Path ; Artist ; Songtitle ; File-extension ; Time in ms ; Time in min:sec

D:\mp3s\Acid\Flow Dynamics - Tremendo Boogaloo.mp3 ; Flow Dynamics ; Tremendo Boogaloo ; mp3 ; 244314 ; 04:04
D:\mp3s\Acid\Flow Dynamics - Up in the Party.mp3 ; Flow Dynamics ; Up in the Party ; mp3 ; 230416 ; 03:50
D:\mp3s\Acid\Freak Power - Rush Radio Version.mp3 ; Freak Power ; Rush Radio Version ; mp3 ; 229137 ; 03:49
D:\mp3s\Acid\Galliano - Prince of Peace Edit.mp3 ; Galliano ; Prince of Peace Edit ; mp3 ; 238907 ; 03:59

Do you have VS available? The program is written in vb6. If you wish I could send you the source.

Attached a screenshot…

regards:
-Serge-

Auto Cue means that mAirList is setting Cue In, Fade Out and Cue Out automatically according some volume thresholds. It’s in mAirListConfig on the Misc -> File Import page. Checkboxes at the top to enable it, and volume sliders below.

Sorry, I don’t have VS (I do own a license for it, which I got free from the MSDN AA program my university was involved in, but I never had a reason to use it). But I get the idea.

If you want to make it simple, you could start by creating M3U files. One file name per line. Much easier to create than mAirList’s native XML files.

OK, I’ll check on Auto Cue tonight, although I’m quite sure they are unchecked.
I have the strange feeling that there’s something wrong with OTF data collection as the splash screen still got displayed this morning and I reduced the amount of files further down to some 12.000
I’m not sure if you have the possibility to test OTF with a large amount of files…?

Hmm, I wonder why the data collection of OTF is done at program start rather than having it triggered on request by the user.
It would prevent long loading times and the user can decide when to update. At least a progress bar, counter or wheel on the splash screen would be at least an indication that the collection is still working/alive.

The approach I use in my Audiofind program is that the program checks on start-up if there’s an index file available. If so, the creation date of the index is displayed on the main screen. So the user might notice (or not) that the file he’s using might be outdated…
Not that I want to impose something, but a similar structure would well fit into mAirList.

Well, creating the XML file is just a piece of cake. (once the structure is known)

regards:
-Serge-

As metioned above, the OTF database is only meant as a temporary replacement until a “real” database is available. The goal was to make a folder available for searching and browsing without any further user interaction. Thus “on the fly”. The way it is “hacked” inside the program initialization process does not allow any user interaction anyway.

It is not suitable for large libraries. I’m aware of that. I haven’t tried to import so many files at once myself, but I know that it’s slow.

I won’t spend any more work on the OTF database which will be obsolete soon anyway. I’d rather put my efforts in finally finishing mAirListDB. Which will have a nice GUI of course, see below :slight_smile:


mAirListDB.png

OK Torben, I agree with you.

So please tell me some words about mAirListDB. I have no clue what it should be or not be.
The word database triggers a red light in me ;D
Please understand that I (and nobody else probably) can afford to spend a lifetime on entering some 50.000 records into a database.
Does this mAirListDB offer some automation to setup records based on ID3 tag etc…
All I need is a search interface finding a file based on artist or songtitle.

regards:
-Serge-

mAirListDB is the missing piece of the puzzle. An application where you manage all of your files, playlists, etc.

It will be fully integrated into mAirList. It will use the same file import mechanisms as mAirList itself does, so all of your existing ID3 tags, MMD files etc. will be imported. There will be two versions, a “Light/Personal” version which stores all data in a single file on the playout PC, and a “Pro” version which uses a PostgreSQL server and is suitable for being run in a network with multiple playout sytems and planning stations.

I have been working on it for quite a while, but it’s making progress and will soon be released. However, I cannot promise that it will be available in the free version of mAirList. Depends on a few things which will be determined over the next weeks.

Ok, thanks for the info.

One last question regarding this topic: “Light/Personal” version is meant for the DJ license I suppose… ?

Don’t get me wrong on my next statement please, we did not discuss financial matters yet and I’m not sure about which license would apply to our Radiostation. Also, this is not the right place to discuss this…

Something I want to avoid absolutely is setting up an extra server for just handling “some” music files. Our studio becomes more and more a datacenter than a radiostudio. So the last thing I want to have is another server to manage.
We have a total of some 8 machines, Linuxservers, Streamers, RDS, Webrelated boxes (Webcam, etc…) all maintained solely by myself. :frowning:
Besides running the radiostation, I have a family and a job… ;D

regards:
-Serge-

There is no final decision about the v3 licensing model yet, but it could be something like this:

  • mAirList Community Edition (for personal use, no database, free)
  • mAirList Personal Edition (for personal use, includes “lite” database and a few more extra features for the semi-pro home user, low price)
  • mAirList Professional Edition (for business use and non-profit organizations, includes “pro” database, prices vary)

If you’re a non-profit organization, I’m going to offer you the Pro edition at a very low price you will be able to afford. But this should be discussed privately.

Regarding the database server, it does not need to be on a separate machine. The files can be on any network drive, and you just need to install PostgreSQL somewhere to store the metadata. PosgreSQL is just a piece of software, availabe for both Windows and Linux/BSD/Solaris/OSX. Just put it on any of your machines. Perhaps you’re familiar with MySQL. PostgreSQL is just the same thing, but it’s more powerful and entirely free, even when used together with closed source software like mAirList. (If you wanted to use a recent MySQL server with mAirList, you’d have to pay like $1000 for a MySQL license.)