In the "ACE Protocol" section you mention that "The protocol does not use an IV and only encrypts single blocks in ECB mode." Actually, the client code uses DES with a fixed IV and in cipher block chaining (CBC) mode.
You suggest that for the protocol to work, the F2 function must have the property that it is "infeasible to find two inputs that produce the same output." Since the F2 function is used only to disguise the pseudorandom PASSCODE value, it is not clear what the impact of such a collision would be. Also, inverting the F2 function would provide an attacker with only a single PASSCODE value, not the token's seed.
In the section titled "Attacking the Protocol", you specify a possible method for "spoofing" a legitimate server response to an invalid request. This method could work against old versions (ACE/Server version 1.2.4 and earlier) of the protocol, but all current versions (Version 1.3 and later) have included a defense against this specific attack. The defense consists of the inclusion, in the server's "OK" response, of a value dependent on the client's secret key.
We view this defense and the defense against the "keyboard race" attack described by "PeiterZ" as short-term fixes. For the longer term, SDI and RSA are completely redesigning the ACE Protocol, both to provide stronger, cryptographic authentication, and to provide for additional security services. The new protocol will be based on published algorithms and standards, and will, as you recommend, be made available for peer review.
In the "Future Directions" section, you state that all the algorithms involved are over a decade old.
The card hash was developed in 1986, but both F2 and SDTI_encrypt date to 1991. SDTI_encrypt, the SDI proprietary block cipher was used as an alternative to DES for export purposes. SDI is now licensed to export the DES version of ACE/Server, so the use of SDTI_encrypt is being discontinued.
The token hash will probably be replaced, with a published algorithm, in the next major update of the token firmware. The new ACE Protocol will not use the F2 hash, and may replace it with SHA-1, or another published algorithm.
Security Dynamics hopes to make ACE/Server an important component in the security infrastructure. The new protocol, based on the cryptographic expertise of RSA, is a crucial step in that direction. We hope you will participate in the review of its design, and look forward to your input.
I will also be attending the DIMACS Workshop on Network Threats. I hope we will have a chance to discuss this further at that time. Thank you again, for both the paper and the chance to respond.
Regards, John G. Brainard
Principal Research Engineer