I have a new installation of Vanilla Forums v2.3 with jsConnect.
I set up a connection. SSO works fine in test mode. However, once I take the connection out of test mode I get the red "Invalid Signature" error.
Try enabling debug mode and see what it says.
In test mode, it doesn't validate the signature. So, basically, you're creating the signature wrong.
Verify you're using exactly the same hash method as what you've selected in the Dashboard under the connection's settings. Then make sure you're alphabetically ordering the JSON array before you hash it.