How to Decrypt RSA with SHA512 in .Net for FIPS compliance

We recently installed software at a client that had FIPS compliance enabled on their Windows 7 PCs and our software encountered an error trying to decrypt some of our license keys at the client.

Our fix was simple, but I only figured out how to do it by using Telerik’s JustDecompile on mscorlib.dll to figure out what was happening.

We are using RSACryptoServiceProvider and SHA512.  Our code looked like this:

success = rsa.VerifyData(bytesToVerify, CryptoConfig.MapNameToOID(“SHA512”), signedBytes);

but that does not work because the “SHA512” passed into the MapNameToOID function will return the code for a SHA512 implementation in windows that is not FIPS compliant.  In fact, MapNameToOID does not support any FIPS compliant SHA512 implementation.

The only way we could get the decryption to verify was to pass a string with the full classname as the second parameter instead of using the MapNameToOID method:

success = rsa.VerifyData(bytesToVerify, “System.Security.Cryptography.SHA512CryptoServiceProvider”, signedBytes);

Microsoft has a list of their algorithms here:

http://msdn.microsoft.com/en-us/library/system.security.cryptography.cryptoconfig(v=vs.110).aspx

For more about FIPS I recommend starting with this article:

http://blog.aggregatedintelligence.com/2007/10/fips-validated-cryptographic-algorithms.html