Since RAS v.17 you may switch between multiple farms more efficiently.
First of all you need log in to each Farm and check 'Remember credentials' checkbox. Of course you may not check it, but in that case you will type credentials on every switch. SSO works also. But you must to use the same credentials as for authentication on a local PC where RAS Console is installed.
When all servers are added you can switch between Farms using switch button next to 'Location' string
NOTE: All your farms and console must be the same version and build. Otherwise you will see popped up error message regarding misconfiguration in versions between farm and console.