I think this might be an easy one.
When we have different networks (read subnets in your scenario) we need to provide a way from which each network can reach or know how to reach the other networks.
The problem you are facing is that if you are in a host in the 192.168.3.X subnet neither the host or the router in this subnet know how to route the traffic to let’s say 192.168.4.X.
You say that your router’s are Linksys but you did not mention which model.
So the basic concept here is:
In each router you need to configure a second network address of the other subnet and make a persistent route to that subnet via that address. In a cisco scenario you will create a sub-interface with a network address of the other subnet. After creating this new network address and the persistent route to the other subnet you will be ready to go (in this way). Of course you will have to do exactly the same thing on the other router to achieve the same functionality in the other way around.
Let me know if you have further questions.
The dual-router, dual-IP scenario listed above MAY work, but there is yet a simpler one – if any of your routers support more than one VLAN/subnet, then set up one of your routers with both LANs on it, and ensure that inter-vlan routing is enabled (if the devices you have support it) and cable both of your servers into the one router…