Connecting to iTunes db

Hi Torben, came across a problem yesterday when trying to connect to an iTunes database.

Using mAirlist build 2.2.1.538
iTunes 8.0.2.20
XP, the very last release that is almost Vista (in appearence anyway).

Having created the database connection, mAirList generates an error message ‘2843833152’ not a valid integer.

Any ideas?

Kind regards tony

Can you please open your iTunes XML file in a text editor and look where exactly that number appears?

The number is too large to fit into a 32-bit integer. I will have to switch to int64, but I need to know whether it is used as an ID, duration, etc. If you could paste the part from the XML file, that would be just perfect.

Hi Torben, this may take some time as the station concerned is in wales about 130 miles away.

I will wait until their local techie is back on site and get him to create a temp database connection.

Kind Regards tony

You can just ask him to send you the XML file and set up a fake iTunes connection on your PC using that file. mAirList won’t try to access any of the audio files unless you drop one into the playlist. The browser should work alright, and you might be able to reproduce the problem.

Hi Torben, the lines from the iTunes xml file:

187

Track ID187
Name11 Track 11
AlbumShe’s the One [UK CD #1]
GenreRock
KindAAC audio file
Size51240
Total Time2843833152
Track Number11
Year1999
Date Modified2005-11-17T18:32:09Z
Date Added2005-11-17T17:32:09Z
Bit Rate128
Sample Rate44100
Persistent ID8CC4F6A974934AD0
Track TypeFile
Locationfile://localhost/C:/Documents%20and%20Settings/Mal/My%20Documents/My%20Music/iTunes/iTunes%20Music/Unknown%20Artist/She’s%20the%20One%20%5BUK%20CD%20%231%5D/11%2011%20Track%2011.m4a
File Folder Count4
Library Folder Count1

Kind regards tony

Alright, it’s the duration field.

I have uploaded a new snapshot (Build 546) which resolves this issue.

Cheers Torben, I will get the local techie to try the latest snapshot.

Kind regards tony

Hi Torben, how can we improve the startup time when mAirList uses iTunes as its database?

It takes an age!

Kind Regards tony

Tony, the best way to improve the db reading time is: store fewer files! ::slight_smile:

Seriously though, whatever ‘non-internal’ database is used, mAirList will have to read all of its contents. That is necessarily going to take a certain amount of time per entry, and (with my ‘profeshunul programmer’ hat on ;D) no matter what ‘tricks’ you employ, there is always a finite limit to that. I don’t know how fast or slow an iTunes database is to read, but I bet it’s still faster than an on-the-fly ‘database!’ :stuck_out_tongue:

I’m sure Torben has done (and continues to do) his best to reduce the amount of processing time the mAirList code takes to read an iTunes database, or any of the external data-storages which mAirList effectively imports every time it starts, but he is limited by how fast the underlying file-reading processes of the OS are working; or possibly the speed of whichever library (DLL) which his code uses to read the files; and of course the speed of Delphi itself. All code is NOT equally fast, unfortunately; if there’s a ‘bottleneck’ somewhere which is out of your control, that’s just tough luck: either way, quite often there is only one option you can use to do certain tasks, and it’s literally impossible to code around it.

You can mitigate against this problem to a certain extent by using a PC with a faster processor, and sometimes by using a faster HD, or a faster HD controller, or by adding extra RAM on the motherboard; but not necessarily! Even if you try to run multiple processing threads (sorry to get heavily techie here!) within your program, that won’t always necessarily help either: some DLLs just won’t allow you to run multiple instances of themselves, though there are mercifully few of THOSE left nowadays.

The reality for programmers is that no matter how efficient your own code is, at some level you are dependent on the internal speed and efficiency (or lack of it :o) of code within other people’s DLLs, or basic Windows system calls, and there isn’t anything that even an exceptionally talented developer like Torben can do about that.

But to return to the original question, the only real solution to the ‘import on startup’ issue is to have an internal database built into mAirList, and as I’m sure you’re aware, we will all have to wait for V3.0 of mAirList for that.

(opt diatribe n q, for anyone else in the UK old enough to remember the CIX system from the 1980s! ;))

BFN
CAD

Cheers Cad, I will be dropping down to Wales to visit Able, the station concerned, in about 3 weeks so the local techie may reduce some of the 7,000 audio files by then. Perhaps best to leave the PC running all the time as most of the playout is from PC.

Startup is around 5 minutes so will be something to live with.

Although more than old enough to remember CIX, before my time :wink:

Kind regards Tony

Also worth mentioning that iTunes misuses the XML format in very disgusting way, which makes the processing very inefficient. For example, take a look at the code above and how e.g. the track and album name are stored:

<key>Name</key><string>11 Track 11</string>
<key>Album</key><string>She's the One [UK CD #1]</string>

Any experienced XML developer would agree that this should rather be

<string key="Name">11 Track 11</string>
<string key="Album">She's the One [UK CD #1]</string>

or even

<Name>11 Track 11</Name>
<Album>She's the One [UK CD #1]</Album>

(the latter is much like mAirList’s MMD files etc.)

With the way iTunes stores the data, you cannot simply use an XPath query if you look for a specific piece of data, but you first have to find the “key” node and then look for the next sibling node. Bah.

Yowch! And though I’m hardly ‘an experienced XML developer,’ I totally agree with you, Torben: that’s a pretty outrageous misuse of XML.

Though I’m hardly surprised: the bigger the company, the more likely they are to ‘bend’ standards.

Tony: This slightly abstruse bit of information would definitely be part of the reason why iTunes is slower to import than it needs to be (cf. my long post above).

BFN
CAD

True. In particular, this is the reason why mAirList needs to parse the complete file at startup and load it into memory. If it was nicely structured XML, I could just access the file directly on disk when performing a search.

Let’s see - perhaps we can write a batch converter from iTunes to mAirListDB (Lite) later. Then you can just convert your iTunes database periodically and use the generated mAirListDB all of the time.

Hmm … a batch converter …

OK, since I would never allow iTunes anywhere near any of my PCs (on principle: I’m an equal-opportunity OS bigot who hates all Apple stuff just as much as Vista or anything to do with L—x! ;D), is the iTunes DB stored as a single ‘XML’ text file on disk, or in some other format? Or is the info stored within each M4A file?

If it’s basically a single text file (or group of text files), here’s my offer.

Creating a ‘converter’ from iTunes format to a real XML file (Torben could specify the format for me, as above) sounds like exactly the kind of ‘utility’ program I enjoy writing. Like my MMD import/export programs, it would seem sensible to write this in VBA (>SIGH< … oh all right then, and in OpenOffice BASIC as well >grumble< ;)).

Why? Well, it could become the iTunes equivalent of my existing Import MMD Files, and you’d end up with a spreadsheet containing the data as well as a GENUINE XML-format output file. You could then use the spreadsheet as an audit report to easily check what was written to the XML file. The spreadsheet would be directly compatible with the MMD Import/Export spreadsheets, even though most fields would be blank. That would give you the option of ‘directly’ creating MMDs from it as well, if you wish to do so.

Do the team think¹ this would be a good idea?
(¹UK radio joke)

PS: No, I haven’t forgotten about making an OO version of Import MMD Files, either!

BFN
CAD