I do not think it’s the formula, maybe the fault is located in the retrieving of the time values.
What are the parentheses after GetFixTime for? GetFixTime is a function which returns some value, no brackets needed there. You can try to put in a GetValue function afterwards. Maybe the machine doesn’t like the recursive acquiring of the time values either. You might want to try:
var
FixTime: TDateTime;
begin
FixTime := CurrentPlaylist.GetItem(iStart).GetFixTime.GetValue;
FixTime := round((FixTime * 86400) - 60) / 86400);
CurrentPlaylist.GetItem(iStart).SetFixTime(FixTime);
end.
Maybe you can skip the round()-function, too. Or try to use frac(), which returns the fractional part (i. e. the figures after the decimal point) of a number. (This, for TDateTime-variables, is the time portion, the date portion is retained in the integer numbers.)
Lastly there is a TimeToSeconds()-function which might work for you.
What I am doing in these cases is involving a button or text element to show these values. If it says 00:00, I know, there’s something wrong, for instance.
ExecuteCommand('BUTTON.4 TEXT ' + FormatDateTime('hh:nn:ss', FixTime));
All of these gimmicks I have encorporated in my backtiming script here (in German, alas, but just ask). Keep on trying, it will work some time! 
Fixedly-timed regards
TSD